Skip to content
geeksforgeeks
  • Tutorials
    • Python
    • Java
    • Data Structures & Algorithms
    • ML & Data Science
    • Interview Corner
    • Programming Languages
    • Web Development
    • CS Subjects
    • DevOps And Linux
    • School Learning
    • Practice Coding Problems
  • Courses
    • DSA to Development
    • Get IBM Certification
    • Newly Launched!
      • Master Django Framework
      • Become AWS Certified
    • For Working Professionals
      • Interview 101: DSA & System Design
      • Data Science Training Program
      • JAVA Backend Development (Live)
      • DevOps Engineering (LIVE)
      • Data Structures & Algorithms in Python
    • For Students
      • Placement Preparation Course
      • Data Science (Live)
      • Data Structure & Algorithm-Self Paced (C++/JAVA)
      • Master Competitive Programming (Live)
      • Full Stack Development with React & Node JS (Live)
    • Full Stack Development
    • Data Science Program
    • All Courses
  • Data Visualization
  • Statistics in R
  • Machine Learning in R
  • Data Science in R
  • Packages in R
  • Data Types
  • String
  • Array
  • Vector
  • Lists
  • Matrices
  • Oops in R
Open In App
Next Article:
KNN Classifier in R Programming
Next article icon

Support Vector Machine Classifier Implementation in R with Caret package

Last Updated : 28 Apr, 2025
Comments
Improve
Suggest changes
Like Article
Like
Report

One of the most crucial aspects of machine learning that most data scientists run against in their careers is the classification problem. The goal of a classification algorithm is to foretell whether a particular activity will take place or not. Depending on the data available, classification algorithms strive to provide answers to questions like whether a client is likely to leave or not, whether a YouTube viewer is likely to click on the suggested video or not, and if it will rain today or not. In this tutorial, we'll use R programming language to create the Support Vector Machine Classifier, which will help us solve a classification issue.

Support Vector Machine (SVM)

To comprehend the idea behind the support vector machine, it is necessary to know that the algorithm groups the points on either side according to their homogeneous relationships using a line called a hyperplane. These points are said to be linearly separable if a straight line can divide them up. Look at the hyperplane in the following figure that linearly divides the symbols for something like the stars and sun.

Support Vector Machine Classifier Implementation in R with Caret package
 

Looking at the above image, one might be surprised to learn that we can draw a line in a different shape and location to create a border between these two shapes, as shown in the following image. This raises the question of how a support vector classifier machine should handle these conditions in order to model for the specific classification problems that are being presented.

Support Vector Machine Classifier Implementation in R with Caret package
 

The aforementioned issue for support vector classifiers can be resolved by using a maximum margin hyperplane - the lines that produce the greatest gap between the two classes. Look at the image below, which shows the relationships between the shapes and the maximum margin hyperplane.

Support Vector Machine Classifier Implementation in R with Caret package
 

The three blue-filled shapes are what are referred to as support vectors. They serve as a foundation for a potential MMH definition. What happens if the points cannot be separated linearly? How does the support vector machine handle issues of this nature? In the presence of the slack variable, which enables the machine to break some limitations and allow a limited number of points to enter restricted areas, nothing is impossible for the support vector classifier. In this case, the machine worked to reduce the overall cost of an algorithm.

Support Vector Machine Classifier Implementation in R with Caret package
 

Another related issue that a support vector machine could run into when modeling real-world categorization data is the extensive non linearity of one data. The kernel technique, a feature of the support vector classifier that enables us to manipulate those data easily to linearly separable data, is also a solution for this kind of problem from the machine. Look at the illustration below.

Support Vector Machine Classifier Implementation in R with Caret package
 

By stating this, let's examine how this machine learning method will be used in practice. We will delve into R and its package caret, one of the most popular open source programming languages. Caret, which stands for classification and regression training, is a potent capability that enables you to use it to create an enormous number of models. Let's get going.

Importing Packages

All required packages installations and importing can be done using below code,

R 
# Code for installation of all necessary packages install.packages("caret") install.packages("ggplot2") install.packages("GGally") install.packages("psych") install.packages("ggpubr") install.packages("reshape")  # Code for importation of all necessary packages library(caret) library(ggplot2)  library(GGally) library(psych)  library(ggpubr) library(reshape)

Loading Data

Dataset includes the information about 768 women from a community close to Phoenix, Arizona, USA, may be found in the Pima Indian Diabetes Dataset, which was originally created by the National Institute of Diabetes and Digestive and Kidney Diseases. This tutorial is designed to categorize whether the patient outcome is diabetes or not depending on the data in the dataset. 

Let's add the data to the code.

R 
# Code df <- read.csv("data/diabetes.csv") head(df)

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 

To check if there is any null values in our dataset.

R 
# Code sum(is.na(df))

Output:

0

To see the dimensions of our dataset.

R 
# Code dim(df)

Output:

768  9

Data Visualization 

To see the type of our data frame.

R 
# Code sapply(df, class)

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 
R 
# Code summary(df) # to calculate the summary of our dataset
Summary of our Dataset
R 
# Code a <- ggplot(data = df, aes(x = Pregnancies)) +     geom_histogram( color = "red", fill = "blue", alpha = 0.1) +      geom_density()  b <- ggplot(data = df, aes(x = Glucose)) +     geom_histogram( color = "red", fill = "blue", alpha = 0.1) +      geom_density()  c <- ggplot(data = df, aes(x = BloodPressure)) +     geom_histogram( color = "red", fill = "blue", alpha = 0.1) +      geom_density()  d <- ggplot(data = df, aes(x = SkinThickness)) +     geom_histogram( color = "red", fill = "blue", alpha = 0.1) +      geom_density()  e <- ggplot(data = df, aes(x = Insulin)) +     geom_histogram( color = "red", fill = "blue", alpha = 0.1) +     geom_density()  f <- ggplot(data = df, aes(x = BMI)) +     geom_histogram( color = "red", fill = "blue", alpha = 0.1) +      geom_density()  g <- ggplot(data = df, aes(x = DiabetesPedigreeFunction)) +     geom_histogram( color = "red", fill = "blue", alpha = 0.1) +     geom_density()  h <- ggplot(data = df, aes(x = Age)) +     geom_histogram( color = "red", fill = "blue", alpha = 0.1) +geom_density()     ggarrange(a, b, c, d,e,f,g, h + rremove("x.text"),            labels = c("a", "b", "c", "d","e", "f", "g", "h"),           ncol = 3, nrow = 3)

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 
R 
# Code ggplot(data = df, aes(x =Outcome, fill = Outcome)) +     geom_bar()

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 

Code to label our categorical variable as a factor

R 
# Code to label our categorical variable as a factor df$Outcome<- factor(df$Outcome,                    levels = c(0, 1),                    labels = c("Negative", "Positive"))

To select only numerical features from our dataset to plot boxplot.

R 
out <- subset(df,   select = c(Pregnancies,Glucose,             BloodPressure,SkinThickness,             Insulin,BMI,             DiabetesPedigreeFunction,Age))  # Code for boxplot ggplot(data = melt(out),                 aes(x=variable, y=value)) +              geom_boxplot(aes(fill=variable))

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 

To plot the correlation plot by using the psych package

R 
corPlot(df[, 1:8])

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 

Insights 

  • In these findings, the mean of the variables necessary to predict the outcome variable—pregnancies, insulin, glucose, diabetes pedigree function, and age—is greater than the median. The mean is higher than the median because the data seem to be biased to the right.
  • Our data's histogram and boxplots support the conclusions of summary statistics.
  • The correlation map, however, shows that the variables in our data do not strongly correlate with one another.

Based on the foregoing summary, how should we format the data? What would be the next step in this? I'll let you work on this yourself.

Data preparation and Modelling

To split our dataset into training and validation

R 
cutoff <- createDataPartition(df$Outcome, p=0.85, list=FALSE) # select 15% of the data for validation testdf <- df[-cutoff,] # use the remaining 85% of data to training and testing the models traindf <- df[cutoff,]

Let's train the SVM model now.

R 
# Code to train the SVM set.seed(1234) # set the 10 fold crossvalidation with AU  # to pick for us what we call the best model control <- trainControl(method="cv",number=10, classProbs = TRUE) metric <- "Accuracy" model <- train(Outcome ~., data = traindf, method = "svmRadial",                 tuneLength = 8,preProc = c("center","scale"),                     metric=metric, trControl=control)

To print the summary of the model.

R 
# Code for model summary model

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 

Let's plot the accuracy graph.

R 
# Code  plot(model)

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 

Let's now evaluate the precision of our model. The confusion matrix will be used to forecast the accuracy.

R 
# validate our model  predict <- predict(model, newdata = testdf) confusionMatrix(predict, testdf$Outcome)

Output:

Support Vector Machine Classifier Implementation in R with Caret package
 

The accuracy on the validation set is 81.74%, according to the confusion matrix results, which suggests that our model can predict this dataset fairly well. With this, this tutorial comes to a close. I hope you learned something from and found it useful. 


Next Article
KNN Classifier in R Programming

M
michaelyousuf
Improve
Article Tags :
  • R Language

Similar Reads

    Machine Learning with R
    Machine Learning as the name suggests is the field of study that allows computers to learn and take decisions on their own i.e. without being explicitly programmed. These decisions are based on the available data that is available through experiences or instructions. It gives the computer that makes
    2 min read

    Getting Started With Machine Learning In R

    Introduction to Machine Learning in R
    Machine learning in R allows data scientists, analysts and statisticians to build predictive models, uncover patterns and gain insights using powerful statistical techniques combined with modern machine learning algorithms. R provides a comprehensive environment with numerous built-in functions and
    6 min read
    What is Machine Learning?
    Machine learning is a branch of artificial intelligence that enables algorithms to uncover hidden patterns within datasets. It allows them to predict new, similar data without explicit programming for each task. Machine learning finds applications in diverse fields such as image and speech recogniti
    9 min read
    Setting up Environment for Machine Learning with R Programming
    Machine Learning is a subset of Artificial Intelligence (AI) which enables systems to learn and make predictions without explicit programming. In machine learning, algorithms and models are developed to identify patterns and trends within data, allowing systems to predict outcomes based on observed
    2 min read
    Supervised and Unsupervised Learning in R Programming
    Machine Learning (ML) is a subset of Artificial Intelligence (AI) that enables computers to learn from data and improve their performance over time without being explicitly programmed. The choice of ML algorithms depends on the type of data and the task at hand which can be broadly divided into Supe
    3 min read

    Data Processing

    Introduction to Data in Machine Learning
    Data refers to the set of observations or measurements to train a machine learning models. The performance of such models is heavily influenced by both the quality and quantity of data available for training and testing. Machine learning algorithms cannot be trained without data. Cutting-edge develo
    4 min read
    ML | Understanding Data Processing
    In machine learning, data is the most important aspect, but the raw data is messy, incomplete, or unstructured. So, we process the raw data to transform it into a clean, structured format for analysis, and this step in the data science pipeline is known as data processing. Without data processing, e
    5 min read
    ML | Overview of Data Cleaning
    Data cleaning is a important step in the machine learning (ML) pipeline as it involves identifying and removing any missing duplicate or irrelevant data. The goal of data cleaning is to ensure that the data is accurate, consistent and free of errors as raw data is often noisy, incomplete and inconsi
    13 min read
    ML | Feature Scaling - Part 1
    Feature Scaling is a technique to standardize the independent features present in the data in a fixed range. It is performed during the data pre-processing. Working: Given a data-set with features- Age, Salary, BHK Apartment with the data size of 5000 people, each having these independent data featu
    3 min read

    Supervised Learning

    Simple Linear Regression in R
    Regression shows a line or curve that passes through all the data points on the target-predictor graph in such a way that the vertical distance between the data points and the regression line is minimum What is Linear Regression?Linear Regression is a commonly used type of predictive analysis. Linea
    12 min read
    Multiple Linear Regression using R
    Prerequisite: Simple Linear-Regression using RLinear Regression: It is the basic and commonly used type for predictive analysis. It is a statistical approach for modeling the relationship between a dependent variable and a given set of independent variables.These are of two types:   Simple linear Re
    3 min read
    Decision Tree for Regression in R Programming
    Decision tree is a type of algorithm in machine learning that uses decisions as the features to represent the result in the form of a tree-like structure. It is a common tool used to visually represent the decisions made by the algorithm. Decision trees use both classification and regression. Regres
    4 min read
    Decision Tree Classifiers in R Programming
    Classification is the task in which objects of several categories are categorized into their respective classes using the properties of classes. A classification model is typically used to, Predict the class label for a new unlabeled data objectProvide a descriptive model explaining what features ch
    4 min read
    Random Forest Approach in R Programming
    Random Forest is an machine learning algorithm which is used for both regression and classification tasks. It is an ensemble method that creates multiple decision trees and combines their outputs to improve model performance.Key points about Random ForestBagging (Bootstrap Aggregating): This method
    5 min read
    Random Forest Approach for Regression in R Programming
    Random Forest approach is a supervised learning algorithm. It builds the multiple decision trees which are known as forest and glue them together to urge a more accurate and stable prediction. The random forest approach is similar to the ensemble technique called as Bagging. In this approach, multip
    3 min read
    Random Forest Approach for Classification in R Programming
    Random forest approach is supervised nonlinear classification and regression algorithm. Classification is a process of classifying a group of datasets in categories or classes. As random forest approach can use classification or regression techniques depending upon the user and target or categories
    4 min read
    Classifying data using Support Vector Machines(SVMs) in R
    Support Vector Machines (SVM) are supervised learning models mainly used for classification and but can also be used for regression tasks. In this approach, each data point is represented as a point in an n-dimensional space where n is the number of features. The goal is to find a hyperplane that be
    5 min read
    Support Vector Machine Classifier Implementation in R with Caret package
    One of the most crucial aspects of machine learning that most data scientists run against in their careers is the classification problem. The goal of a classification algorithm is to foretell whether a particular activity will take place or not. Depending on the data available, classification algori
    7 min read
    KNN Classifier in R Programming
    K-Nearest Neighbor or KNN is a supervised non-linear classification algorithm. It is also Non-parametric in nature meaning , it doesn't make any assumption about underlying data or its distribution. Algorithm Structure In KNN algorithm, K specifies the number of neighbors and its algorithm is as fol
    4 min read

    Evaluation Metrics

    Precision, Recall and F1-Score using R
    In machine learning, evaluating model performance is critical. Three widely used metrics—Precision, Recall, and F1-Score—help assess the quality of classification models. Here's what each metric represents:Recall: Measures the proportion of actual positive cases correctly identified. Also known as s
    3 min read
    How to Calculate F1 Score in R?
    In this article, we will be looking at the approach to calculate F1 Score using the various packages and their various functionalities in the R language. F1 Score The F-score or F-measure is a measure of a test's accuracy. It is calculated from the precision and recall of the test, where the precisi
    5 min read

    Unsupervised Learning

    K-Means Clustering in R Programming
    K Means Clustering is an unsupervised learning algorithm that groups data into clusters based on similarity. This algorithm divides data into a specified number of clusters, assigning each data point to one. It is used in various fields like banking, healthcare, retail and media. In this article we
    3 min read
    Hierarchical Clustering in R Programming
    Hierarchical clustering in R Programming Language is an Unsupervised non-linear algorithm in which clusters are created such that they have a hierarchy(or a pre-determined ordering). For example, consider a family of up to three generations. A grandfather and mother have their children that become f
    3 min read
    How to Perform Hierarchical Cluster Analysis using R Programming?
    Cluster analysis or clustering is a technique to find subgroups of data points within a data set. The data points belonging to the same subgroup have similar features or properties. Clustering is an unsupervised machine learning approach and has a wide variety of applications such as market research
    5 min read
    Linear Discriminant Analysis in R Programming
    One of the most popular or well established Machine Learning technique is Linear Discriminant Analysis (LDA ). It is mainly used to solve classification problems rather than supervised classification problems. It is basically a dimensionality reduction technique. Using the Linear combinations of pre
    6 min read

    Model Selection and Evaluation

    Cross-Validation in R programming
    The major challenge in designing a machine learning model is to make it work accurately on the unseen data. To know whether the designed model is working fine or not, we have to test it against those data points which were not present during the training of the model. These data points will serve th
    9 min read
    LOOCV (Leave One Out Cross-Validation) in R Programming
    LOOCV (Leave-One-Out Cross-Validation) is a cross-validation technique where each individual observation in the dataset is used once as the validation set, while the remaining observations are used as the training set. This process is repeated for all observations, with each one serving as the valid
    4 min read
    Bias-Variance Trade Off - Machine Learning
    It is important to understand prediction errors (bias and variance) when it comes to accuracy in any machine-learning algorithm. There is a tradeoff between a model’s ability to minimize bias and variance which is referred to as the best solution for selecting a value of Regularization constant. A p
    3 min read

    Reinforcement Learning

    Markov Decision Process
    Markov Decision Process (MDP) is a way to describe how a decision-making agent like a robot or game character moves through different situations while trying to achieve a goal. MDPs rely on variables such as the environment, agent’s actions and rewards to decide the system’s next optimal action. It
    4 min read
    Q-Learning in Reinforcement Learning
    Q-Learning is a popular model-free reinforcement learning algorithm that helps an agent learn how to make the best decisions by interacting with its environment. Instead of needing a model of the environment the agent learns purely from experience by trying different actions and seeing their results
    7 min read
    Deep Q-Learning in Reinforcement Learning
    Deep Q-Learning is a method that uses deep learning to help machines make decisions in complicated situations. It’s especially useful in environments where the number of possible situations called states is very large like in video games or robotics.Before understanding Deep Q-Learning it’s importan
    4 min read

    Dimensionality Reduction

    Introduction to Dimensionality Reduction
    When working with machine learning models, datasets with too many features can cause issues like slow computation and overfitting. Dimensionality reduction helps to reduce the number of features while retaining key information. Techniques like principal component analysis (PCA), singular value decom
    4 min read
    ML | Introduction to Kernel PCA
    PRINCIPAL COMPONENT ANALYSIS: is a tool which is used to reduce the dimension of the data. It allows us to reduce the dimension of the data without much loss of information. PCA reduces the dimension by finding a few orthogonal linear combinations (principal components) of the original variables wit
    6 min read
    Principal Component Analysis with R Programming
    Principal component analysis(PCA) in R programming is an analysis of the linear components of all existing attributes. Principal components are linear combinations (orthogonal transformation) of the original predictor in the dataset. It is a useful technique for EDA(Exploratory data analysis) and al
    3 min read

    Advanced Topics

    Kolmogorov-Smirnov Test in R Programming
    Kolmogorov-Smirnov (K-S) test is a non-parametric test employed to check whether the probability distributions of a sample and a control distribution, or two samples are equal. It is constructed based on the cumulative distribution function (CDF) and calculates the greatest difference between the em
    4 min read
    Moore – Penrose Pseudoinverse in R Programming
    The concept used to generalize the solution of a linear equation is known as Moore – Penrose Pseudoinverse of a matrix. Moore – Penrose inverse is the most widely known type of matrix pseudoinverse. In linear algebra pseudoinverse A^{+}    of a matrix A is a generalization of the inverse matrix. The
    5 min read
    Spearman Correlation Testing in R Programming
    Correlation is a key statistical concept used to measure the strength and direction of the relationship between two variables. Unlike Pearson’s correlation, which assumes a linear relationship and continuous data, Spearman’s rank correlation coefficient is a non-parametric measure that assesses how
    3 min read
    Poisson Functions in R Programming
    The Poisson distribution represents the probability of a provided number of cases happening in a set period of space or time if these cases happen with an identified constant mean rate (free of the period since the ultimate event). Poisson distribution has been named after Siméon Denis Poisson(Frenc
    3 min read
    Feature Engineering in R Programming
    Feature engineering is the process of transforming raw data into features that can be used in a machine-learning model. In R programming, feature engineering can be done using a variety of built-in functions and packages. One common approach to feature engineering is to use the dplyr package to mani
    7 min read
    Adjusted Coefficient of Determination in R Programming
    Prerequisite: Multiple Linear Regression using R A well-fitting regression model produces predicted values close to the observed data values. The mean model, which uses the mean for every predicted value, commonly would be used if there were no informative predictor variables. The fit of a proposed
    3 min read
    Mann Whitney U Test in R Programming
    A popular nonparametric(distribution-free) test to compare outcomes between two independent groups is the Mann Whitney U test. When comparing two independent samples, when the outcome is not normally distributed and the samples are small, a nonparametric test is appropriate. It is used to see the di
    4 min read
    Bootstrap Confidence Interval with R Programming
    Bootstrapping is a statistical method for inference about a population using sample data. It can be used to estimate the confidence interval(CI) by drawing samples with replacement from sample data. Bootstrapping can be used to assign CI to various statistics that have no closed-form or complicated
    5 min read
geeksforgeeks-footer-logo
Corporate & Communications Address:
A-143, 7th Floor, Sovereign Corporate Tower, Sector- 136, Noida, Uttar Pradesh (201305)
Registered Address:
K 061, Tower K, Gulshan Vivante Apartment, Sector 137, Noida, Gautam Buddh Nagar, Uttar Pradesh, 201305
GFG App on Play Store GFG App on App Store
Advertise with us
  • Company
  • About Us
  • Legal
  • Privacy Policy
  • In Media
  • Contact Us
  • Advertise with us
  • GFG Corporate Solution
  • Placement Training Program
  • Languages
  • Python
  • Java
  • C++
  • PHP
  • GoLang
  • SQL
  • R Language
  • Android Tutorial
  • Tutorials Archive
  • DSA
  • Data Structures
  • Algorithms
  • DSA for Beginners
  • Basic DSA Problems
  • DSA Roadmap
  • Top 100 DSA Interview Problems
  • DSA Roadmap by Sandeep Jain
  • All Cheat Sheets
  • Data Science & ML
  • Data Science With Python
  • Data Science For Beginner
  • Machine Learning
  • ML Maths
  • Data Visualisation
  • Pandas
  • NumPy
  • NLP
  • Deep Learning
  • Web Technologies
  • HTML
  • CSS
  • JavaScript
  • TypeScript
  • ReactJS
  • NextJS
  • Bootstrap
  • Web Design
  • Python Tutorial
  • Python Programming Examples
  • Python Projects
  • Python Tkinter
  • Python Web Scraping
  • OpenCV Tutorial
  • Python Interview Question
  • Django
  • Computer Science
  • Operating Systems
  • Computer Network
  • Database Management System
  • Software Engineering
  • Digital Logic Design
  • Engineering Maths
  • Software Development
  • Software Testing
  • DevOps
  • Git
  • Linux
  • AWS
  • Docker
  • Kubernetes
  • Azure
  • GCP
  • DevOps Roadmap
  • System Design
  • High Level Design
  • Low Level Design
  • UML Diagrams
  • Interview Guide
  • Design Patterns
  • OOAD
  • System Design Bootcamp
  • Interview Questions
  • Inteview Preparation
  • Competitive Programming
  • Top DS or Algo for CP
  • Company-Wise Recruitment Process
  • Company-Wise Preparation
  • Aptitude Preparation
  • Puzzles
  • School Subjects
  • Mathematics
  • Physics
  • Chemistry
  • Biology
  • Social Science
  • English Grammar
  • Commerce
  • World GK
  • GeeksforGeeks Videos
  • DSA
  • Python
  • Java
  • C++
  • Web Development
  • Data Science
  • CS Subjects
@GeeksforGeeks, Sanchhaya Education Private Limited, All rights reserved
We use cookies to ensure you have the best browsing experience on our website. By using our site, you acknowledge that you have read and understood our Cookie Policy & Privacy Policy
Lightbox
Improvement
Suggest Changes
Help us improve. Share your suggestions to enhance the article. Contribute your expertise and make a difference in the GeeksforGeeks portal.
geeksforgeeks-suggest-icon
Create Improvement
Enhance the article with your expertise. Contribute to the GeeksforGeeks community and help create better learning resources for all.
geeksforgeeks-improvement-icon
Suggest Changes
min 4 words, max Words Limit:1000

Thank You!

Your suggestions are valuable to us.

What kind of Experience do you want to share?

Interview Experiences
Admission Experiences
Career Journeys
Work Experiences
Campus Experiences
Competitive Exam Experiences