Skip to content

Hyperparameter tuning using GridSearchCV and KerasClassifier

SVM Hyperparameter Tuning using GridSearchCV – ML

Last Updated : 10 May, 2025

Support Vector Machines (SVM) are used for classification tasks but their performance depends on the right choice of hyperparameters like C and gamma. Finding the optimal combination of these hyperparameters can be a issue. GridSearchCV automates the process by systematically testing various combinations of hyperparameters and selecting the best one based on cross-validation results. In this article, we’ll see how to use GridSearchCV with a SVM model to improve its accuracy and increase its predictive capabilities.

Let see various steps which are involved in this implementation.

Step 1: Importing Necessary Libraries

We will be using Pandas, NumPy and Scikit-learn for building and evaluating the model.

Python

import pandas as pd  import numpy as np  from sklearn.metrics import classification_report, confusion_matrix  from sklearn.datasets import load_breast_cancer  from sklearn.svm import SVC

Step 2: Loading and Printing the Dataset

In this example we will use Breast Cancer dataset from Scikit-learn. This dataset contains data about cell features and their corresponding cancer diagnosis i.e malignant or benign.

Python

cancer = load_breast_cancer()  df_feat = pd.DataFrame(cancer['data'], columns=cancer['feature_names']) df_target = pd.DataFrame(cancer['target'], columns=['Cancer'])  print("Feature Variables: ") print(df_feat.info()) print("Dataframe looks like : ") print(df_feat.head())

Output:

Step 3: Splitting the Data into Training and Testing Sets

We will split the dataset into training (70%) and testing (30%) sets using train_test_split.

Python

from sklearn.model_selection import train_test_split   X_train, X_test, y_train, y_test = train_test_split(  						df_feat, np.ravel(df_target),  				test_size = 0.30, random_state = 101)

Step 4: Training an SVM Model without Hyperparameter Tuning

Before tuning the model let’s train a simple SVM classifier without any hyperparameter tuning.

Python

model = SVC()  model.fit(X_train, y_train)   predictions = model.predict(X_test)  print(classification_report(y_test, predictions))

Output:

svm3

Training without Hyperparameter Tuning

While the accuracy is around 92%, we can improve the model’s performance by tuning the hyperparameters.

Step 5: Hyperparameter Tuning with GridSearchCV

Now let’s use GridSearchCV to find the best combination of C, gamma and kernel hyperparameters for the SVM model.

Python

from sklearn.model_selection import GridSearchCV   param_grid = {'C': [0.1, 1, 10, 100, 1000],  			'gamma': [1, 0.1, 0.01, 0.001, 0.0001],  			'kernel': ['rbf']}   grid = GridSearchCV(SVC(), param_grid, refit = True, verbose = 3)    grid.fit(X_train, y_train)

Output:

svm4-

Hyperparameter Tuning with GridSearchCV

Step 6: Get the Best Hyperparameters and Model

After grid search finishes we can check best hyperparameters and the optimized model.

Python

print(grid.best_params_) print(grid.best_estimator_)

Output:

Hyperparameters and Model

Step 7: Evaluating the Optimized Model

We can evaluate the optimized model on the test dataset.

Python

grid_predictions = grid.predict(X_test) print(classification_report(y_test, grid_predictions))

Output:

svm6

Model evaluation

After hyperparameter tuning, the accuracy of the model increased to 94% showing that the tuning process improved the model’s performance. By using this approach, we can improve the model which helps in making it more accurate and reliable.

Hyperparameter tuning using GridSearchCV and KerasClassifier

T

tyagikartik4282

Improve

Article Tags :

Practice Tags :

Machine Learning

Similar Reads

SVM Hyperparameter Tuning using GridSearchCV | ML

Support Vector Machines (SVM) are used for classification tasks but their performance depends on the right choice of hyperparameters like C and gamma. Finding the optimal combination of these hyperparameters can be a issue. GridSearchCV automates the process by systematically testing various combina

Hyperparameter tuning using GridSearchCV and KerasClassifier

Hyperparameter tuning is done to increase the efficiency of a model by tuning the parameters of the neural network. Some scikit-learn APIs like GridSearchCV and RandomizedSearchCV are used to perform hyper parameter tuning. In this article, you'll learn how to use GridSearchCV to tune Keras Neural N

Hyperparameter tuning SVM parameters using Genetic Algorithm

The performance support Vector Machines (SVMs) are heavily dependent on hyperparameters such as the regularization parameter (C) and the kernel parameters (gamma for RBF kernel). Genetic Algorithms (GAs) leverage evolutionary principles to search for optimal hyperparameter values. This article explo

Hyperparameter Tuning with R

In R Language several techniques and packages can be used to optimize these hyperparameters, leading to better, more reliable models. in this article, we will discuss all the techniques and packages for Hyperparameter Tuning with R. What are Hyperparameters?Hyperparameters are the settings that cont

Hyperparameter tuning

Machine Learning model is defined as a mathematical model with several parameters that need to be learned from the data. By training a model with existing data we can fit the model parameters. However there is another kind of parameter known as hyperparameters which cannot be directly learned from t

How to tune a Decision Tree in Hyperparameter tuning

Decision trees are powerful models extensively used in machine learning for classification and regression tasks. The structure of decision trees resembles the flowchart of decisions helps us to interpret and explain easily. However, the performance of decision trees highly relies on the hyperparamet

Random Forest Hyperparameter Tuning in Python

Random Forest is one of the most popular and powerful machine learning algorithms used for both classification and regression tasks. It works by building multiple decision trees and combining their outputs to improve accuracy and control overfitting. While Random Forest is already a robust model fin

Cross-validation and Hyperparameter tuning of LightGBM Model

In a variety of industries, including finance, healthcare, and marketing, machine learning models have become essential for resolving challenging real-world issues. Gradient boosting techniques have become incredibly popular among the myriad of machine learning algorithms due to their remarkable pre

CatBoost Cross-Validation and Hyperparameter Tuning

CatBoost is a powerful gradient-boosting algorithm of machine learning that is very popular for its effective capability to handle categorial features of both classification and regression tasks. To maximize the potential of CatBoost, it's essential to fine-tune its hyperparameters which can be done

Hyperparameter Tuning in Linear Regression

Linear regression is one of the simplest and most widely used algorithms in machine learning. Despite its simplicity, it can be quite powerful, especially when combined with proper hyperparameter tuning. Hyperparameter tuning is the process of tuning a machine learning model's parameters to achieve