Hyperparameter tuning using GridSearchCV and KerasClassifier

Last Updated : 20 Mar, 2024

Hyperparameter tuning is done to increase the efficiency of a model by tuning the parameters of the neural network. Some scikit-learn APIs like GridSearchCV and RandomizedSearchCV are used to perform hyper parameter tuning.

In this article, you’ll learn how to use GridSearchCV to tune Keras Neural Networks hyper parameters.

Approach:

We will wrap Keras models for use in scikit-learn using KerasClassifier which is a wrapper.
We will use cross validation using KerasClassifier and GridSearchCV
Tune hyperparameters like number of epochs, number of neurons and batch size.

Implementation of the scikit-learn classifier API for Keras:

tf.keras.wrappers.scikit_learn.KerasClassifier(

build_fn=None, **sk_params

)

Code:

   # import the libraries  
import tensorflow as tf 
import pandas as pd 
from sklearn.compose import ColumnTransformer 
from sklearn.preprocessing import OneHotEncoder 
from keras.wrappers.scikit_learn import KerasClassifier 
from sklearn.model_selection import GridSearchCV 
from sklearn.preprocessing import LabelEncoder 
from sklearn.preprocessing import StandardScaler 
 
  

Import the dataset using which we’ll predict if a customer stays or leave.

Code:

   # The last column is a binary value 
dataset = pd.read_csv('Churn_Modelling.csv') 
X = dataset.iloc[:, 3:-1].values 
y = dataset.iloc[:, -1].values 
 
  

Code: Preprocess the data

   le = LabelEncoder() 
X[:, 2] = le.fit_transform(X[:, 2]) 
#perform one hot encoding  
ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [1])], remainder='passthrough') 
X = np.array(ct.fit_transform(X)) 
# perform standardization of the data.  
sc = StandardScaler() 
X = sc.fit_transform(X) 
 
  

To use the KerasClassifier wrapper, we will need to build our model in a function which needs to be passed to the build_fn argument in the KerasClassifier constructor.

Code:

   def build_clf(unit): 
  # creating the layers of the NN 
  ann = tf.keras.models.Sequential() 
  ann.add(tf.keras.layers.Dense(units=unit, activation='relu')) 
  ann.add(tf.keras.layers.Dense(units=unit, activation='relu')) 
  ann.add(tf.keras.layers.Dense(units=1, activation='sigmoid')) 
  ann.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy']) 
  return ann
 
  

Code: create the object of KerasClassifier class

model=KerasClassifier(build_fn=build_clf)

Now we will create the dictionary of the parameters we want to tune and pass as an argument in GridSearchCV.

Code:

   params={'batch_size':[100, 20, 50, 25, 32],  
        'nb_epoch':[200, 100, 300, 400], 
        'unit':[5,6, 10, 11, 12, 15], 
           
        } 
gs=GridSearchCV(estimator=model, param_grid=params, cv=10) 
# now fit the dataset to the GridSearchCV object.  
gs = gs.fit(X, y)
 
  

The best_score_ member gives the best score observed during the optimization procedure and the best_params_ describes the combination of parameters that achieved the best results.

Code:

   best_params=gs.best_params_ 
accuracy=gs.best_score_

Output:

Accuracy: 0.80325

Best Params: {‘batch_size’: 20, ‘nb_epoch’: 200, ‘unit’: 15}

Hyperparameter tuning SVM parameters using Genetic Algorithm

maryamnadeem20

Improve

Article Tags :

Practice Tags :

Support Vector Machines (SVM) are used for classification tasks but their performance depends on the right choice of hyperparameters like C and gamma. Finding the optimal combination of these hyperparameters can be a issue. GridSearchCV automates the process by systematically testing various combina

3 min read

Hyperparameter tuning SVM parameters using Genetic Algorithm

The performance support Vector Machines (SVMs) are heavily dependent on hyperparameters such as the regularization parameter (C) and the kernel parameters (gamma for RBF kernel). Genetic Algorithms (GAs) leverage evolutionary principles to search for optimal hyperparameter values. This article explo

9 min read

How to tune a Decision Tree in Hyperparameter tuning

Decision trees are powerful models extensively used in machine learning for classification and regression tasks. The structure of decision trees resembles the flowchart of decisions helps us to interpret and explain easily. However, the performance of decision trees highly relies on the hyperparamet

14 min read

Cross-validation and Hyperparameter tuning of LightGBM Model

In a variety of industries, including finance, healthcare, and marketing, machine learning models have become essential for resolving challenging real-world issues. Gradient boosting techniques have become incredibly popular among the myriad of machine learning algorithms due to their remarkable pre

14 min read

Hyperparameter Tuning with R

In R Language several techniques and packages can be used to optimize these hyperparameters, leading to better, more reliable models. in this article, we will discuss all the techniques and packages for Hyperparameter Tuning with R. What are Hyperparameters?Hyperparameters are the settings that cont

5 min read

HyperParameter Tuning: Fixing Overfitting in Neural Networks

Overfitting is a pervasive problem in neural networks, where the model becomes too specialized to the training data and fails to generalize well to new, unseen data. This issue can be addressed through hyperparameter tuning, which involves adjusting various parameters to optimize the performance of

6 min read

CatBoost Cross-Validation and Hyperparameter Tuning

CatBoost is a powerful gradient-boosting algorithm of machine learning that is very popular for its effective capability to handle categorial features of both classification and regression tasks. To maximize the potential of CatBoost, it's essential to fine-tune its hyperparameters which can be done

11 min read

Transfer learning & fine-tuning using Keras

Transfer learning is a powerful technique used in deep learning tasks. Here, a model developed for a particular task is reused as a starting point for a model on the second task. Thus, transfer learning uses the knowledge gained from a pre-trained model and allows faster convergence with better perf

7 min read

Sklearn | Model Hyper-parameters Tuning

Hyperparameter tuning is the process of finding the optimal values for the hyperparameters of a machine-learning model. Hyperparameters are parameters that control the behaviour of the model but are not learned during training. Hyperparameter tuning is an important step in developing machine learnin

12 min read

Hyperparameter tuning with Ray Tune in PyTorch

Hyperparameter tuning is a crucial step in the machine learning pipeline that can significantly impact the performance of a model. Choosing the right set of hyperparameters can be the difference between an average model and a highly accurate one. Ray Tune is an industry-standard tool for distributed

8 min read

Hyperparameter tuning using GridSearchCV and KerasClassifier

Approach:

Implementation of the scikit-learn classifier API for Keras:

Output:

Accuracy: 0.80325

Best Params: {‘batch_size’: 20, ‘nb_epoch’: 200, ‘unit’: 15}

Similar Reads