SVM Hyperparameter Tuning using GridSearchCV - ML
Last Updated : 10 May, 2025
Support Vector Machines (SVM) are used for classification tasks but their performance depends on the right choice of hyperparameters like C and gamma. Finding the optimal combination of these hyperparameters can be a issue. GridSearchCV automates the process by systematically testing various combinations of hyperparameters and selecting the best one based on cross-validation results. In this article, we’ll see how to use GridSearchCV with a SVM model to improve its accuracy and increase its predictive capabilities.
Let see various steps which are involved in this implementation.
Step 1: Importing Necessary Libraries
We will be using Pandas, NumPy and Scikit-learn for building and evaluating the model.
Python import pandas as pd import numpy as np from sklearn.metrics import classification_report, confusion_matrix from sklearn.datasets import load_breast_cancer from sklearn.svm import SVC
Step 2: Loading and Printing the Dataset
In this example we will use Breast Cancer dataset from Scikit-learn. This dataset contains data about cell features and their corresponding cancer diagnosis i.e malignant or benign.
Python cancer = load_breast_cancer() df_feat = pd.DataFrame(cancer['data'], columns=cancer['feature_names']) df_target = pd.DataFrame(cancer['target'], columns=['Cancer']) print("Feature Variables: ") print(df_feat.info()) print("Dataframe looks like : ") print(df_feat.head()) Output:
Step 3: Splitting the Data into Training and Testing Sets
We will split the dataset into training (70%) and testing (30%) sets using train_test_split.
Python from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split( df_feat, np.ravel(df_target), test_size = 0.30, random_state = 101)
Step 4: Training an SVM Model without Hyperparameter Tuning
Before tuning the model let’s train a simple SVM classifier without any hyperparameter tuning.
Python model = SVC() model.fit(X_train, y_train) predictions = model.predict(X_test) print(classification_report(y_test, predictions))
Output:
Training without Hyperparameter TuningWhile the accuracy is around 92%, we can improve the model’s performance by tuning the hyperparameters.
Step 5: Hyperparameter Tuning with GridSearchCV
Now let’s use GridSearchCV to find the best combination of C, gamma and kernel hyperparameters for the SVM model.
Python from sklearn.model_selection import GridSearchCV param_grid = {'C': [0.1, 1, 10, 100, 1000], 'gamma': [1, 0.1, 0.01, 0.001, 0.0001], 'kernel': ['rbf']} grid = GridSearchCV(SVC(), param_grid, refit = True, verbose = 3) grid.fit(X_train, y_train) Output:
Hyperparameter Tuning with GridSearchCVStep 6: Get the Best Hyperparameters and Model
After grid search finishes we can check best hyperparameters and the optimized model.
Python print(grid.best_params_) print(grid.best_estimator_)
Output:
Hyperparameters and Model Step 7: Evaluating the Optimized Model
We can evaluate the optimized model on the test dataset.
Python grid_predictions = grid.predict(X_test) print(classification_report(y_test, grid_predictions))
Output:
Model evaluationAfter hyperparameter tuning, the accuracy of the model increased to 94% showing that the tuning process improved the model’s performance. By using this approach, we can improve the model which helps in making it more accurate and reliable.