Random Forest Classifier using Scikit-learn
Last Updated : 11 Mar, 2025
Random Forest is a method that combines the predictions of multiple decision trees to produce a more accurate and stable result. It can be used for both classification and regression tasks.
In classification tasks, Random Forest Classification predicts categorical outcomes based on the input data. It uses multiple decision trees and outputs the label that has the maximum votes among all the individual tree predictions and in this article we will learn more about it.
Working of Random Forest Classifier
Random Forest Classification works by creating multiple decision trees each trained on a random subset of data. The process begins with Bootstrap Sampling where random rows of data are selected with replacement to form different training datasets for each tree.
Then where only a random subset of features is used to build each tree ensuring diversity across the models.
During the training phase Feature Sampling is applied to each tree built by recursively partitioning the data based on the features. At each split the algorithm selects the best feature from the random subset optimizing for information gain or Gini impurity. The process continues until a predefined stopping criterion is met such as reaching maximum depth or having a minimum number of samples in each leaf node. After the trees are trained each tree makes a prediction. The final prediction for classification tasks is determined by majority voting.
Random Forest Classifier
Benefits of Random Forest Classification:
- Random Forest can handle large datasets and high-dimensional data.
- By combining predictions from many decision trees it reduces the risk of overfitting compared to a single decision tree.
- It is robust to noisy data and works well with categorical data.
Implementing Random Forest Classification in Python
Before implementing random forest classifier in Python let’s first understand it’s parameters.
- n_estimators: Number of trees in the forest.
- max_depth: Maximum depth of each tree.
- max_features: Number of features considered for splitting at each node.
- criterion: Function used to measure split quality (‘gini’ or ‘entropy’).
- min_samples_split: Minimum samples required to split a node.
- min_samples_leaf: Minimum samples required to be at a leaf node.
- bootstrap: Whether to use bootstrap sampling when building trees (True or False).
Now that we know it’s parameters we can start building it in python.
1. Import Required Libraries
We will be importing Pandas, matplotlib, seaborn and sklearn to build the model.
python import pandas as pd import matplotlib.pyplot as plt import seaborn as sns import sklearn from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.metrics import accuracy_score, confusion_matrix from sklearn.ensemble import RandomForestClassifier from sklearn.datasets import load_iris
2. Import Dataset
For this we’ll use the Iris Dataset which is available within sklearn. This dataset contains information about three types of Iris flowers and their respective features (sepal length, sepal width, petal length and petal width).
python iris = load_iris() df = pd.DataFrame(data=iris.data, columns=iris.feature_names) df['target'] = iris.target df
Output:
3. Data Preparation
Here we will separate the features (X) and the target variable (y).
python X = df.iloc[:, :-1].values y = df.iloc[:, -1].values
4. Splitting the Dataset
We’ll split the dataset into training and testing sets so we can train the model on one part and evaluate it on another.
python X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
5. Feature Scaling
Feature scaling ensures that all the features are on a similar scale which is important for some machine learning models. However Random Forest is not highly sensitive to feature scaling. But it is a good practice to scale when combining models.
python scaler = StandardScaler() X_train = scaler.fit_transform(X_train) X_test = scaler.transform(X_test)
6. Building Random Forest Classifier
We will create the Random Forest Classifier model, train it on the training data and make predictions on the test data.
python classifier = RandomForestClassifier(n_estimators=100, random_state=42) classifier.fit(X_train, y_train) y_pred = classifier.predict(X_test)
7. Evaluation of the Model
We will evaluate the model using the accuracy score and confusion matrix.
python accuracy = accuracy_score(y_test, y_pred) print(f'Accuracy: {accuracy * 100:.2f}%') conf_matrix = confusion_matrix(y_test, y_pred) plt.figure(figsize=(8, 6)) sns.heatmap(conf_matrix, annot=True, fmt='g', cmap='Blues', cbar=False, xticklabels=iris.target_names, yticklabels=iris.target_names) plt.title('Confusion Matrix Heatmap') plt.xlabel('Predicted Labels') plt.ylabel('True Labels') plt.show() Output:
Accuracy: 100.00%
Confusion Matrix
Its perfect accuracy along with the confusion matrix shown by Random Forest Classifier has learned to classify all the instances correctly. However it’s essential to note that the Iris dataset used here is relatively simple and well-known in the machine learning
8. Feature Importance
Random Forest Classifiers also provide insight into which features were the most important in making predictions. We can plot the feature importance.
Python feature_importances = classifier.feature_importances_ plt.barh(iris.feature_names, feature_importances) plt.xlabel('Feature Importance') plt.title('Feature Importance in Random Forest Classifier') plt.show() Output:
From the graph we can see that petal width (cm) is the most important feature followed closely by petal length (cm). The sepal width (cm) and sepal length (cm) have lower importance in determining the model’s predictions. This indicates that the classifier relies more on the petal measurements to make predictions about the flower species.
Random Forest Classifiers are useful for classification tasks offering high accuracy and robustness. They are easy to use, provide insights into feature importance and can handle complex datasets.
Random Forest can also be used for regression problem: Random Forest Regression in Python
Similar Reads
Feature Selection Using Random forest Classifier
Feature selection is a crucial step in the machine learning pipeline that involves identifying the most relevant features for building a predictive model. One effective method for feature selection is using a Random Forest classifier, which provides insights into feature importance. In this article,
5 min read
Multiclass classification using scikit-learn
Multiclass classification is a popular problem in supervised machine learning. Problem - Given a dataset of m training examples, each of which contains information in the form of various features and a label. Each label corresponds to a class, to which the training example belongs. In multiclass cla
5 min read
Classifier Comparison in Scikit Learn
In scikit-learn, a classifier is an estimator that is used to predict the label or class of an input sample. There are many different types of classifiers that can be used in scikit-learn, each with its own strengths and weaknesses. Let's load the iris datasets from the sklearn.datasets and then tr
3 min read
Text Classification using scikit-learn in NLP
The purpose of text classification, a key task in natural language processing (NLP), is to categorise text content into preset groups. Topic categorization, sentiment analysis, and spam detection can all benefit from this. In this article, we will use scikit-learn, a Python machine learning toolkit,
5 min read
Feature Selection Using Random Forest
Feature selection is a crucial step in building machine learning models. It involves selecting the most important features from your dataset that contribute to the predictive power of the model. Random Forest, an ensemble learning method, is widely used for feature selection due to its inherent abil
4 min read
Random Forest for Image Classification Using OpenCV
Random Forest is a machine learning algorithm that uses multiple decision trees to achieve precise results in classification and regression tasks. It resembles the process of choosing the best path amidst multiple options. OpenCV, an open-source library for computer vision and machine learning tasks
8 min read
Implementation of KNN classifier using Scikit - learn - Python
K-Nearest Neighbors is a most simple but fundamental classifier algorithm in Machine Learning. It is under the supervised learning category and used with great intensity for pattern recognition, data mining and analysis of intrusion. It is widely disposable in real-life scenarios since it is non-par
3 min read
ML | Cancer cell classification using Scikit-learn
Machine learning is used in solving real-world problems including medical diagnostics. One such application is classifying cancer cells based on their features and determining whether they are 'malignant' or 'benign'. In this article, we will use Scikit-learn to build a classifier for cancer cell de
4 min read
Interpreting Random Forest Classification Results
Random Forest is a powerful and versatile machine learning algorithm that excels in both classification and regression tasks. It is an ensemble learning method that constructs multiple decision trees during training and outputs the class that is the mode of the classes (for classification) or mean p
6 min read
Hyperparameters of Random Forest Classifier
In this article, we are going to learn about different hyperparameters that exist in a Random Forest Classifier. We have already learnt about the implementation of Random Forest Classifier using scikit-learn library in the article https://www.geeksforgeeks.org/random-forest-classifier-using-scikit-l
4 min read