Decision Tree Regression using sklearn - Python Last Updated : 02 Jun, 2025 Comments Improve Suggest changes Like Article Like Report Decision Tree Regression is a method used to predict continuous values like prices or scores by using a tree-like structure. It works by splitting the data into smaller parts based on simple rules taken from the input features. These splits help reduce errors in prediction. At the end of each branch, called a leaf node the model gives a prediction usually the average value of that group. In the tree:Decision Nodes (shown as diamonds) ask yes/no questions about the data, like “Is age greater than 50?”Leaf Nodes (shown as rectangles) give the final predicted number based on the data that reached that point.Workflow of Decision Tree RegressionBranches connect nodes and represent the outcome of a decision. For example if the answer to a condition is "Yes," you follow one branch; if "No," you follow another. In below example it shows a decision tree that evaluates the smallest of three numbers: Implementation of Decision Tree RegressionFor example we want to predict house prices based on factors like size, location and age. A Decision Tree Regressor can split the data based on these features such as checking the location first, then the size and finally the age. This way it can accurately predicts the price by considering the most impactful factors first making it useful and easy to interpret.Step 1: Importing the required librariesWe will import the following libraries.NumPy: For numerical computations and array handlingMatplotlib: For plotting graphs and visualizationsWe import different modules from scikit-learn (sklearn) for various tasks such as modeling, data splitting, tree visualization, and performance evaluation. Python import numpy as np import matplotlib.pyplot as plt from sklearn.tree import DecisionTreeRegressor, export_text from sklearn.model_selection import train_test_split from sklearn.metrics import mean_squared_error Step 2: Creating a Sample DatasetHere we create a synthetic dataset using numpy library, where the feature values X are randomly sampled and sorted between 0 and 5, and the target y is a noisy sine function of X. The scatter plot visualizes the data points, showing how the target values vary with the feature. Python np.random.seed(42) X = np.sort(5 * np.random.rand(100, 1), axis=0) y = np.sin(X).ravel() + np.random.normal(0, 0.1, X.shape[0]) plt.scatter(X, y, color='red', label='Data') plt.title("Synthetic Dataset") plt.xlabel("Feature") plt.ylabel("Target") plt.legend() plt.show() Output:Non-linear DataStep 3: Splitting the DatasetWe split the dataset into train and test dataset using the train_test_split function into the ratio of 70% training and 30% testing. We also set a random_state=42 to ensure reproducibility. Python X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42) Step 4: Initializing the Decision Tree RegressorHere we used DecisionTreeRegressor method from Sklearn python library to implement Decision Tree Regression. We also define the max_depth as 4 which controls the maximum levels a tree can reach , controlling model complexity. Python regressor = DecisionTreeRegressor(max_depth=4, random_state=42) Step 5: Fiting Decision Tree Regressor ModelWe fit our model using the .fit() method on the X_train and y_train , so that the model can learn the relationships between different variables. Python regressor.fit(X_train, y_train) Output:DecisionTreeRegressor(max_depth=4, random_state=42)Step 6: Predicting a New ValueWe will now predict a new value using our trained model using the predict() function. After that we also calculated the mean squared error (MSE) to check how accurate is our predicted value from the actual value , telling how well the model fits to our training data. Python y_pred = regressor.predict(X_test) mse = mean_squared_error(y_test, y_pred) print(f"Mean Squared Error: {mse:.4f}") Output:Mean Squared Error: 0.0151Step 7: Visualizing the resultWe will visualise the regression line our model has calculated to see how well the decision tree fits the data and captures the underlying pattern, especially showing how the predictions change smoothly or in steps depending on the tree's splits. Python X_grid = np.arange(min(X), max(X), 0.01)[:, np.newaxis] y_grid_pred = regressor.predict(X_grid) plt.figure(figsize=(10, 6)) plt.scatter(X, y, color='red', label='Data') plt.plot(X_grid, y_grid_pred, color='blue', label='Model Prediction') plt.title("Decision Tree Regression") plt.xlabel("Feature") plt.ylabel("Target") plt.legend() plt.show() Output:Decision Tree RegressionStep 8: Export and Show the Tree Structure belowFor better understanding we used plot_tree to visualize the structure of the decision tree to understand how the model splits the feature space, showing the decision rules at each node and how the tree partitions the data to make predictions. Python from sklearn.tree import plot_tree plt.figure(figsize=(20, 10)) plot_tree( regressor, feature_names=["Feature"], filled=True, rounded=True, fontsize=10 ) plt.title("Decision Tree Structure") plt.show() Output: Visualized Decision Tree RegressionDecision Tree Regression is used for predicting continuous values effectively capturing non-linear patterns in data. Its tree-based structure makes model interpretability easy as we can tell why a decision was made and why we get this specific output. This information can further be used to fine tune model based on it flow of working. Comment More infoAdvertise with us Next Article Ensemble Methods in Python A AnkanDas22 Follow Improve Article Tags : Machine Learning AI-ML-DS ML-Regression Sklearn Practice Tags : Machine Learning Similar Reads Machine Learning Algorithms Machine learning algorithms are essentially sets of instructions that allow computers to learn from data, make predictions, and improve their performance over time without being explicitly programmed. Machine learning algorithms are broadly categorized into three types: Supervised Learning: Algorith 8 min read Top 15 Machine Learning Algorithms Every Data Scientist Should Know in 2025 Machine Learning (ML) Algorithms are the backbone of everything from Netflix recommendations to fraud detection in financial institutions. These algorithms form the core of intelligent systems, empowering organizations to analyze patterns, predict outcomes, and automate decision-making processes. Wi 14 min read Linear Model RegressionOrdinary Least Squares (OLS) using statsmodelsOrdinary Least Squares (OLS) is a widely used statistical method for estimating the parameters of a linear regression model. It minimizes the sum of squared residuals between observed and predicted values. In this article we will learn how to implement Ordinary Least Squares (OLS) regression using P 3 min read Linear Regression (Python Implementation)Linear regression is a statistical method that is used to predict a continuous dependent variable i.e target variable based on one or more independent variables. This technique assumes a linear relationship between the dependent and independent variables which means the dependent variable changes pr 14 min read Multiple Linear Regression using Python - MLLinear regression is a statistical method used for predictive analysis. It models the relationship between a dependent variable and a single independent variable by fitting a linear equation to the data. Multiple Linear Regression extends this concept by modelling the relationship between a dependen 4 min read Polynomial Regression ( From Scratch using Python )Prerequisites Linear RegressionGradient DescentIntroductionLinear Regression finds the correlation between the dependent variable ( or target variable ) and independent variables ( or features ). In short, it is a linear model to fit the data linearly. But it fails to fit and catch the pattern in no 5 min read Bayesian Linear RegressionLinear regression is based on the assumption that the underlying data is normally distributed and that all relevant predictor variables have a linear relationship with the outcome. But In the real world, this is not always possible, it will follows these assumptions, Bayesian regression could be the 10 min read How to Perform Quantile Regression in PythonIn this article, we are going to see how to perform quantile regression in Python. Linear regression is defined as the statistical method that constructs a relationship between a dependent variable and an independent variable as per the given set of variables. While performing linear regression we a 4 min read Isotonic Regression in Scikit LearnIsotonic regression is a regression technique in which the predictor variable is monotonically related to the target variable. This means that as the value of the predictor variable increases, the value of the target variable either increases or decreases in a consistent, non-oscillating manner. Mat 6 min read Stepwise Regression in PythonStepwise regression is a method of fitting a regression model by iteratively adding or removing variables. It is used to build a model that is accurate and parsimonious, meaning that it has the smallest number of variables that can explain the data. There are two main types of stepwise regression: F 6 min read Least Angle Regression (LARS)Regression is a supervised machine learning task that can predict continuous values (real numbers), as compared to classification, that can predict categorical or discrete values. Before we begin, if you are a beginner, I highly recommend this article. Least Angle Regression (LARS) is an algorithm u 3 min read Linear Model ClassificationLogistic Regression in Machine LearningLogistic Regression is a supervised machine learning algorithm used for classification problems. Unlike linear regression which predicts continuous values it predicts the probability that an input belongs to a specific class. It is used for binary classification where the output can be one of two po 11 min read Understanding Activation Functions in DepthIn artificial neural networks, the activation function of a neuron determines its output for a given input. This output serves as the input for subsequent neurons in the network, continuing the process until the network solves the original problem. Consider a binary classification problem, where the 6 min read RegularizationImplementation of Lasso Regression From Scratch using PythonLasso Regression (Least Absolute Shrinkage and Selection Operator) is a linear regression technique that combines prediction with feature selection. It does this by adding a penalty term to the cost function shrinking less relevant feature's coefficients to zero. This makes it effective for high-dim 7 min read Implementation of Ridge Regression from Scratch using PythonPrerequisites: Linear Regression Gradient Descent Introduction: Ridge Regression ( or L2 Regularization ) is a variation of Linear Regression. In Linear Regression, it minimizes the Residual Sum of Squares ( or RSS or cost function ) to fit the training examples perfectly as possible. The cost funct 4 min read Implementation of Elastic Net Regression From ScratchPrerequisites: Linear RegressionGradient DescentLasso & Ridge RegressionIntroduction: Elastic-Net Regression is a modification of Linear Regression which shares the same hypothetical function for prediction. The cost function of Linear Regression is represented by J. \frac{1}{m} \sum_{i=1}^{m}\l 5 min read K-Nearest Neighbors (KNN)Implementation of Elastic Net Regression From ScratchPrerequisites: Linear RegressionGradient DescentLasso & Ridge RegressionIntroduction: Elastic-Net Regression is a modification of Linear Regression which shares the same hypothetical function for prediction. The cost function of Linear Regression is represented by J. \frac{1}{m} \sum_{i=1}^{m}\l 5 min read Brute Force Approach and its pros and consIn this article, we will discuss the Brute Force Algorithm and what are its pros and cons. What is the Brute Force Algorithm?A brute force algorithm is a simple, comprehensive search strategy that systematically explores every option until a problem's answer is discovered. It's a generic approach to 3 min read Implementation of KNN classifier using Scikit - learn - PythonK-Nearest Neighbors is a most simple but fundamental classifier algorithm in Machine Learning. It is under the supervised learning category and used with great intensity for pattern recognition, data mining and analysis of intrusion. It is widely disposable in real-life scenarios since it is non-par 3 min read Regression using k-Nearest Neighbors in R ProgrammingMachine learning is a subset of Artificial Intelligence that provides a machine with the ability to learn automatically without being explicitly programmed. The machine in such cases improves from the experience without human intervention and adjusts actions accordingly. It is primarily of 3 types: 5 min read Support Vector MachinesSupport Vector Machine (SVM) AlgorithmSupport Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression tasks. It tries to find the best boundary known as hyperplane that separates different classes in the data. It is useful when you want to do binary classification like spam vs. not spam or 9 min read Classifying data using Support Vector Machines(SVMs) in PythonIntroduction to SVMs: In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. A Support Vector Machine (SVM) is a discriminative classifier 4 min read Support Vector Regression (SVR) using Linear and Non-Linear Kernels in Scikit LearnSupport vector regression (SVR) is a type of support vector machine (SVM) that is used for regression tasks. It tries to find a function that best predicts the continuous output value for a given input value. SVR can use both linear and non-linear kernels. A linear kernel is a simple dot product bet 5 min read Major Kernel Functions in Support Vector Machine (SVM)In previous article we have discussed about SVM(Support Vector Machine) in Machine Learning. Now we are going to learn in detail about SVM Kernel and Different Kernel Functions and its examples.Types of SVM Kernel FunctionsSVM algorithm use the mathematical function defined by the kernel. Kernel Fu 4 min read ML - Stochastic Gradient Descent (SGD) Stochastic Gradient Descent (SGD) is an optimization algorithm in machine learning, particularly when dealing with large datasets. It is a variant of the traditional gradient descent algorithm but offers several advantages in terms of efficiency and scalability, making it the go-to method for many d 8 min read Decision TreeMajor Kernel Functions in Support Vector Machine (SVM)In previous article we have discussed about SVM(Support Vector Machine) in Machine Learning. Now we are going to learn in detail about SVM Kernel and Different Kernel Functions and its examples.Types of SVM Kernel FunctionsSVM algorithm use the mathematical function defined by the kernel. Kernel Fu 4 min read CART (Classification And Regression Tree) in Machine LearningCART( Classification And Regression Trees) is a variation of the decision tree algorithm. It can handle both classification and regression tasks. Scikit-Learn uses the Classification And Regression Tree (CART) algorithm to train Decision Trees (also called “growing†trees). CART was first produced b 11 min read Decision Tree Classifiers in R ProgrammingClassification is the task in which objects of several categories are categorized into their respective classes using the properties of classes. A classification model is typically used to, Predict the class label for a new unlabeled data objectProvide a descriptive model explaining what features ch 4 min read Decision Tree Regression using sklearn - PythonDecision Tree Regression is a method used to predict continuous values like prices or scores by using a tree-like structure. It works by splitting the data into smaller parts based on simple rules taken from the input features. These splits help reduce errors in prediction. At the end of each branch 4 min read Ensemble LearningEnsemble Methods in PythonEnsemble means a group of elements viewed as a whole rather than individually. An Ensemble method creates multiple models and combines them to solve it. Ensemble methods help to improve the robustness/generalizability of the model. In this article, we will discuss some methods with their implementat 11 min read Random Forest Regression in PythonA random forest is an ensemble learning method that combines the predictions from multiple decision trees to produce a more accurate and stable prediction. It is a type of supervised learning algorithm that can be used for both classification and regression tasks.In regression task we can use Random 7 min read ML | Extra Tree Classifier for Feature SelectionPrerequisites: Decision Tree Classifier Extremely Randomized Trees Classifier(Extra Trees Classifier) is a type of ensemble learning technique which aggregates the results of multiple de-correlated decision trees collected in a "forest" to output it's classification result. In concept, it is very si 6 min read Implementing the AdaBoost Algorithm From ScratchAdaBoost means Adaptive Boosting which is a ensemble learning technique that combines multiple weak classifiers to create a strong classifier. It works by sequentially adding classifiers to correct the errors made by previous models giving more weight to the misclassified data points. In this articl 4 min read XGBoostTraditional machine learning models like decision trees and random forests are easy to interpret but often struggle with accuracy on complex datasets. XGBoost short form for eXtreme Gradient Boosting is an advanced machine learning algorithm designed for efficiency, speed and high performance.It is 6 min read CatBoost in Machine LearningWhen working with machine learning we often deal with datasets that include categorical data. We use techniques like One-Hot Encoding or Label Encoding to convert these categorical features into numerical values. However One-Hot Encoding can lead to sparse matrix and cause overfitting. This is where 5 min read LightGBM (Light Gradient Boosting Machine)LightGBM is an open-source high-performance framework developed by Microsoft. It is an ensemble learning framework that uses gradient boosting method which constructs a strong learner by sequentially adding weak learners in a gradient descent manner.It's designed for efficiency, scalability and high 7 min read Stacking in Machine LearningStacking is a ensemble learning technique where the final model known as the “stacked model" combines the predictions from multiple base models. The goal is to create a stronger model by using different models and combining them.Architecture of StackingStacking architecture is like a team of models 3 min read Like