Skip to content
geeksforgeeks
  • Courses
    • DSA to Development
    • Get IBM Certification
    • Newly Launched!
      • Master Django Framework
      • Become AWS Certified
    • For Working Professionals
      • Interview 101: DSA & System Design
      • Data Science Training Program
      • JAVA Backend Development (Live)
      • DevOps Engineering (LIVE)
      • Data Structures & Algorithms in Python
    • For Students
      • Placement Preparation Course
      • Data Science (Live)
      • Data Structure & Algorithm-Self Paced (C++/JAVA)
      • Master Competitive Programming (Live)
      • Full Stack Development with React & Node JS (Live)
    • Full Stack Development
    • Data Science Program
    • All Courses
  • Tutorials
    • Data Structures & Algorithms
    • ML & Data Science
    • Interview Corner
    • Programming Languages
    • Web Development
    • CS Subjects
    • DevOps And Linux
    • School Learning
  • Practice
    • Build your AI Agent
    • GfG 160
    • Problem of the Day
    • Practice Coding Problems
    • GfG SDE Sheet
  • Contests
    • Accenture Hackathon (Ending Soon!)
    • GfG Weekly [Rated Contest]
    • Job-A-Thon Hiring Challenge
    • All Contests and Events
  • Data Science
  • Data Science Projects
  • Data Analysis
  • Data Visualization
  • Machine Learning
  • ML Projects
  • Deep Learning
  • NLP
  • Computer Vision
  • Artificial Intelligence
Open In App
Next Article:
Stacking in Machine Learning
Next article icon

LightGBM (Light Gradient Boosting Machine)

Last Updated : 11 Mar, 2025
Comments
Improve
Suggest changes
Like Article
Like
Report

LightGBM is an open-source high-performance framework developed by Microsoft. It is an ensemble learning framework that uses gradient boosting method which constructs a strong learner by sequentially adding weak learners in a gradient descent manner.

It’s designed for efficiency, scalability and high accuracy particularly with large datasets. It uses decision trees that grow efficiently by minimizing memory usage and optimizing training time. Key innovations like Gradient-based One-Side Sampling (GOSS), histogram-based algorithms and leaf-wise tree growth enable LightGBM to outperform other frameworks in both speed and accuracy.

LightGBM-Geeksforgeeks

Prerequisites

  • Supervised Machine Learning
  • Ensemble Learning
  • Gradient Boosting
  • Tree Based Machine Learning Algorithms

LightGBM Tutorial

In this guide we will go through LightGBM Tutorial and its dependencies.

LightGBM installations

Setting up LightGBM involves installing necessary dependencies like CMake and compilers, cloning the repository and building the framework. Once the framework is set up the Python package can be installed using pip to start utilize LightGBM.

  • How to Install LightGBM on Windows?
  • How to Install LightGBM on Linux?
  • How to Install LightGBM on MacOS?

LightGBM Data Structure

LightGBM Data Structure API refers to the set of functions and methods provided by the framework for handling and manipulating data structures within the context of machine learning tasks. This API includes functions for creating datasets, loading data from different sources, preprocessing features and converting data into formats suitable for training models with LightGBM. It allows users to interact with data efficiently and seamlessly integrate it into the machine learning workflow.

  • lightgbm.Dataset
  • lightgbm.Booster
  • lightgbm.CVBooster
  • lightgbm.Sequence

LightGBM Core Parameters

LightGBM’s performance is heavily influenced by the core parameters that control the structure and optimization of the model. Below are some of the key parameters:

  1. objective: Specifies the loss function to optimize during training. LightGBM supports various objectives such as regression, binary classification and multiclass classification.
  2. task : It specifies the task we wish to perform which is either train or prediction. The default entry is train.
  3. num_leaves: Specifies the maximum number of leaves in each tree. Higher values allow the model to capture more complex patterns but may lead to overfitting.
  4. learning_rate: Determines the step size at each iteration during gradient descent. Lower values result in slower learning but may improve generalization.
  5. max_depth: Sets the maximum depth of each tree.
  6. min_data_in_leaf: Specifies the minimum number of data points required to form a leaf node. Higher values help prevent overfitting but may result in underfitting.
  7. num_iterations : It specifies the number of iterations to be performed. The default value is 100.
  8. feature_fraction: Controls the fraction of features to consider when building each tree. Randomly selecting a subset of features helps improve model diversity and reduce overfitting.
  9. bagging_fraction: Specifies the fraction of data to be used for bagging (sampling data points with replacement) during training.
  10. L1 and L2: Regularization parameters that control L1 and L2 regularization respectively. They penalize large coefficients to prevent overfitting.
  11. min_split_gain: Specifies the minimum gain required to split a node further. It helps control the tree’s growth and prevents unnecessary splits.
  12. categorical_feature : It specifies the categorical feature used for training model.

One who want to study about the applications of these parameters in details they can follow the below article.

  • LightGBM Tree Parameters
  • LightGBM Feature Parameters

LightGBM Tree

A LightGBM tree is a decision tree structure used to predict outcomes. These trees are grown recursively in a leaf-wise manner, maximizing reduction in loss at each step. Key features of LightGBM trees include:

  • LightGBM Leaf-wise tree growth strategy
  • LightGBM Gradient-Based Strategy
  • LightGBM Histogram-Based Learning
  • Handling categorical features efficiently using LightGBM

LightGBM Boosting Algorithms

LightGBM Boosting Algorithms uses:

  • Gradient Boosting Decision Trees (GBDT): builds decision trees sequentially to correct errors iteratively.
  • Gradient-based One-Side Sampling (GOSS): samples instances with large gradients, optimizing efficiency.
  • Exclusive Feature Bundling (EFB): bundles exclusive features to reduce overfitting.
  • Dropouts meet Multiple Additive Regression Trees (DART): introduces dropout regularization to improve model robustness by training an ensemble of diverse models.

These algorithms balance speed, memory usage and accuracy.

LightGBM Examples

  • LightGBM Regression Examples
  • LightGBM for Classifications
    • LightGBM Binary Classifications Example
    • LightGBM Multiclass Classifications Example
  • Time Series Using LightGBM
  • LightGBM for Quantile regression

Training and Evaluation in LightGBM

Training in LightGBM involves fitting a gradient boosting model to a dataset. During training, the model iteratively builds decision trees to minimize a specified loss function, adjusting tree parameters to optimize model performance. Evaluation assesses the trained model’s performance using metrics such as mean squared error for regression tasks or accuracy for classification tasks. Cross-validation techniques may be employed to validate model performance on unseen data and prevent overfitting.

  • Train a model using LightGBM
  • Cross-validation and hyperparameter tuning
  • LightGBM evaluation metrics

LightGBM Hyperparameters Tuning

LightGBM hyperparameter tuning involves optimizing the settings that govern the behavior and performance of the model during training. Techniques like grid search, random search and Bayesian optimization can be used to find the optimal set of hyperparameters for your model.

  • LightGBM key Hyperparameters
    • Learning Rate
    • Number of Leaves
    • Max Depth
    • Feature Fraction
  • LightGBM Regularization parameters
  • LightGBM Learning Control Parameters

LightGBM Parallel and GPU Training

LightGBM supports parallel processing and GPU acceleration which greatly enhances training speed particularly for large-scale datasets. It allows the use of multiple CPU cores or GPUs making it highly scalable.

LightGBM Feature Importance and Visualization

Understanding which features contribute most to your model’s predictions is key. Feature importance can be visualized using techniques like SHAP values (SHapley Additive exPlanations) which provide a unified measure of feature importance. This helps in interpreting the model and guiding future feature engineering efforts.

  • LightGBM Feature Importance and Visualization
  • SHAP (SHapley Additive exPlanations) values for interpretability

Advantages of the LightGBM

LightGBM offers several key benefits:

  • Faster speed and higher accuracy: It outperforms other gradient boosting algorithms on large datasets.
  • Low memory usage: Optimized for memory efficiency and handling large datasets with minimal overhead.
  • Parallel and GPU learning support: Takes advantage of multiple cores or GPUs for faster training.
  • Effective on large datasets: Its optimized techniques such as leaf-wise growth and histogram-based learning make it suitable for big data applications.

LightGBM vs Other Boosting Algorithms

A comparison between LightGBM and other boosting algorithms such as Gradient Boosting, AdaBoost, XGBoost and CatBoost highlights:

  • LightGBM vs XGBOOST
  • GradientBoosting vs AdaBoost vs XGBoost vs CatBoost vs LightGBM

LightGBM is an outstanding choice for solving supervised learning tasks particularly for classification, regression and ranking problems. Its unique algorithms, efficient memory usage, and support for parallel and GPU training give it a distinct advantage over other gradient boosting methods.



Next Article
Stacking in Machine Learning

S
shreyanshisingh28
Improve
Article Tags :
  • AI-ML-DS
  • Computer Subject
  • Machine Learning
  • AI-ML-DS With Python
  • LightGBM
Practice Tags :
  • Machine Learning

Similar Reads

  • Machine Learning Algorithms
    Machine learning algorithms are essentially sets of instructions that allow computers to learn from data, make predictions, and improve their performance over time without being explicitly programmed. Machine learning algorithms are broadly categorized into three types: Supervised Learning: Algorith
    8 min read
  • Top 15 Machine Learning Algorithms Every Data Scientist Should Know in 2025
    Machine Learning (ML) Algorithms are the backbone of everything from Netflix recommendations to fraud detection in financial institutions. These algorithms form the core of intelligent systems, empowering organizations to analyze patterns, predict outcomes, and automate decision-making processes. Wi
    15 min read

    • Linear Model Regression

    • Ordinary Least Squares (OLS) using statsmodels
      Ordinary Least Squares (OLS) is a widely used statistical method for estimating the parameters of a linear regression model. It minimizes the sum of squared residuals between observed and predicted values. In this article we will learn how to implement Ordinary Least Squares (OLS) regression using P
      3 min read
    • Linear Regression (Python Implementation)
      Linear regression is a statistical method that is used to predict a continuous dependent variable i.e target variable based on one or more independent variables. This technique assumes a linear relationship between the dependent and independent variables which means the dependent variable changes pr
      14 min read
    • ML | Multiple Linear Regression using Python
      Linear regression is a fundamental statistical method widely used for predictive analysis. It models the relationship between a dependent variable and a single independent variable by fitting a linear equation to the data. Multiple Linear Regression is an extension of this concept that allows us to
      4 min read
    • Polynomial Regression ( From Scratch using Python )
      Prerequisites Linear RegressionGradient DescentIntroductionLinear Regression finds the correlation between the dependent variable ( or target variable ) and independent variables ( or features ). In short, it is a linear model to fit the data linearly. But it fails to fit and catch the pattern in no
      5 min read
    • Bayesian Linear Regression
      Linear regression is based on the assumption that the underlying data is normally distributed and that all relevant predictor variables have a linear relationship with the outcome. But In the real world, this is not always possible, it will follows these assumptions, Bayesian regression could be the
      11 min read
    • How to Perform Quantile Regression in Python
      In this article, we are going to see how to perform quantile regression in Python. Linear regression is defined as the statistical method that constructs a relationship between a dependent variable and an independent variable as per the given set of variables. While performing linear regression we a
      4 min read
    • Isotonic Regression in Scikit Learn
      Isotonic regression is a regression technique in which the predictor variable is monotonically related to the target variable. This means that as the value of the predictor variable increases, the value of the target variable either increases or decreases in a consistent, non-oscillating manner. Mat
      6 min read
    • Stepwise Regression in Python
      Stepwise regression is a method of fitting a regression model by iteratively adding or removing variables. It is used to build a model that is accurate and parsimonious, meaning that it has the smallest number of variables that can explain the data. There are two main types of stepwise regression: F
      6 min read
    • Least Angle Regression (LARS)
      Regression is a supervised machine learning task that can predict continuous values (real numbers), as compared to classification, that can predict categorical or discrete values. Before we begin, if you are a beginner, I highly recommend this article. Least Angle Regression (LARS) is an algorithm u
      3 min read

    Linear Model Classification

    • Logistic Regression in Machine Learning
      In our previous discussion, we explored the fundamentals of machine learning and walked through a hands-on implementation of Linear Regression. Now, let's take a step forward and dive into one of the first and most widely used classification algorithms — Logistic Regression What is Logistic Regressi
      13 min read
    • Understanding Activation Functions in Depth
      In artificial neural networks, the activation function of a neuron determines its output for a given input. This output serves as the input for subsequent neurons in the network, continuing the process until the network solves the original problem. Consider a binary classification problem, where the
      6 min read

    Regularization

    • Implementation of Lasso Regression From Scratch using Python
      Lasso Regression (Least Absolute Shrinkage and Selection Operator) is a linear regression technique that combines prediction with feature selection. It does this by adding a penalty term to the cost function shrinking less relevant feature's coefficients to zero. This makes it effective for high-dim
      7 min read
    • Implementation of Ridge Regression from Scratch using Python
      Prerequisites: Linear Regression Gradient Descent Introduction: Ridge Regression ( or L2 Regularization ) is a variation of Linear Regression. In Linear Regression, it minimizes the Residual Sum of Squares ( or RSS or cost function ) to fit the training examples perfectly as possible. The cost funct
      4 min read
    • Implementation of Elastic Net Regression From Scratch
      Prerequisites: Linear RegressionGradient DescentLasso & Ridge RegressionIntroduction: Elastic-Net Regression is a modification of Linear Regression which shares the same hypothetical function for prediction. The cost function of Linear Regression is represented by J. [Tex]\frac{1}{m} \sum_{i=1}^
      5 min read

    K-Nearest Neighbors (KNN)

    • Implementation of Elastic Net Regression From Scratch
      Prerequisites: Linear RegressionGradient DescentLasso & Ridge RegressionIntroduction: Elastic-Net Regression is a modification of Linear Regression which shares the same hypothetical function for prediction. The cost function of Linear Regression is represented by J. [Tex]\frac{1}{m} \sum_{i=1}^
      5 min read
    • Brute Force Approach and its pros and cons
      In this article, we will discuss the Brute Force Algorithm and what are its pros and cons. What is the Brute Force Algorithm?A brute force algorithm is a simple, comprehensive search strategy that systematically explores every option until a problem's answer is discovered. It's a generic approach to
      3 min read
    • Implementation of KNN classifier using Scikit - learn - Python
      K-Nearest Neighbors is a most simple but fundamental classifier algorithm in Machine Learning. It is under the supervised learning category and used with great intensity for pattern recognition, data mining and analysis of intrusion. It is widely disposable in real-life scenarios since it is non-par
      3 min read
    • Regression using k-Nearest Neighbors in R Programming
      Machine learning is a subset of Artificial Intelligence that provides a machine with the ability to learn automatically without being explicitly programmed. The machine in such cases improves from the experience without human intervention and adjusts actions accordingly. It is primarily of 3 types:
      5 min read

    Support Vector Machines

    • Support Vector Machine (SVM) Algorithm
      Support Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression tasks. While it can handle regression problems, SVM is particularly well-suited for classification tasks. SVM aims to find the optimal hyperplane in an N-dimensional space to separate data
      10 min read
    • Classifying data using Support Vector Machines(SVMs) in Python
      Introduction to SVMs: In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. A Support Vector Machine (SVM) is a discriminative classifier
      4 min read
    • Support Vector Regression (SVR) using Linear and Non-Linear Kernels in Scikit Learn
      Support vector regression (SVR) is a type of support vector machine (SVM) that is used for regression tasks. It tries to find a function that best predicts the continuous output value for a given input value. SVR can use both linear and non-linear kernels. A linear kernel is a simple dot product bet
      5 min read
    • Major Kernel Functions in Support Vector Machine (SVM)
      In previous article we have discussed about SVM(Support Vector Machine) in Machine Learning. Now we are going to learn  in detail about SVM Kernel and Different Kernel Functions and its examples. Types of SVM Kernel FunctionsSVM algorithm use the mathematical function defined by the kernel. Kernel F
      4 min read
  • ML | Stochastic Gradient Descent (SGD)
    Stochastic Gradient Descent (SGD) is an optimization algorithm in machine learning, particularly when dealing with large datasets. It is a variant of the traditional gradient descent algorithm but offers several advantages in terms of efficiency and scalability, making it the go-to method for many d
    8 min read

    • Decision Tree

    • Major Kernel Functions in Support Vector Machine (SVM)
      In previous article we have discussed about SVM(Support Vector Machine) in Machine Learning. Now we are going to learn  in detail about SVM Kernel and Different Kernel Functions and its examples. Types of SVM Kernel FunctionsSVM algorithm use the mathematical function defined by the kernel. Kernel F
      4 min read
    • CART (Classification And Regression Tree) in Machine Learning
      CART( Classification And Regression Trees) is a variation of the decision tree algorithm. It can handle both classification and regression tasks. Scikit-Learn uses the Classification And Regression Tree (CART) algorithm to train Decision Trees (also called “growing” trees). CART was first produced b
      11 min read
    • Decision Tree Classifiers in R Programming
      Classification is the task in which objects of several categories are categorized into their respective classes using the properties of classes. A classification model is typically used to, Predict the class label for a new unlabeled data objectProvide a descriptive model explaining what features ch
      4 min read
    • Python | Decision Tree Regression using sklearn
      When it comes to predicting continuous values, Decision Tree Regression is a powerful and intuitive machine learning technique. Unlike traditional linear regression, which assumes a straight-line relationship between input features and the target variable, Decision Tree Regression is a non-linear re
      4 min read

    Ensemble Learning

    • Ensemble Methods in Python
      Ensemble means a group of elements viewed as a whole rather than individually. An Ensemble method creates multiple models and combines them to solve it. Ensemble methods help to improve the robustness/generalizability of the model. In this article, we will discuss some methods with their implementat
      11 min read
    • Random Forest Regression in Python
      A random forest is an ensemble learning method that combines the predictions from multiple decision trees to produce a more accurate and stable prediction. It is a type of supervised learning algorithm that can be used for both classification and regression tasks. In regression task we can use Rando
      9 min read
    • ML | Extra Tree Classifier for Feature Selection
      Prerequisites: Decision Tree Classifier Extremely Randomized Trees Classifier(Extra Trees Classifier) is a type of ensemble learning technique which aggregates the results of multiple de-correlated decision trees collected in a "forest" to output it's classification result. In concept, it is very si
      6 min read
    • Implementing the AdaBoost Algorithm From Scratch
      AdaBoost means Adaptive Boosting and it is a is a powerful ensemble learning technique that combines multiple weak classifiers to create a strong classifier. It works by sequentially adding classifiers to correct the errors made by previous models giving more weight to the misclassified data points.
      3 min read
    • XGBoost
      Traditional machine learning models like decision trees and random forests are easy to interpret but often struggle with accuracy on complex datasets. XGBoost, short for eXtreme Gradient Boosting, is an advanced machine learning algorithm designed for efficiency, speed, and high performance. What is
      9 min read
    • CatBoost in Machine Learning
      When working with machine learning, we often deal with datasets that include categorical data. We use techniques like One-Hot Encoding or Label Encoding to convert these categorical features into numerical values. However One-Hot Encoding can lead to sparse matrix and cause overfitting. This is wher
      7 min read
    • LightGBM (Light Gradient Boosting Machine)
      LightGBM is an open-source high-performance framework developed by Microsoft. It is an ensemble learning framework that uses gradient boosting method which constructs a strong learner by sequentially adding weak learners in a gradient descent manner. It's designed for efficiency, scalability and hig
      7 min read
    • Stacking in Machine Learning
      Machine learning uses ensemble methods to combine multiple models and improve overall performance. These methods uses the strengths of various models to create a more robust solution. Some of the most popular ensemble techniques include Bagging and Boosting. Bagging trains multiple similar models an
      2 min read
geeksforgeeks-footer-logo
Corporate & Communications Address:
A-143, 7th Floor, Sovereign Corporate Tower, Sector- 136, Noida, Uttar Pradesh (201305)
Registered Address:
K 061, Tower K, Gulshan Vivante Apartment, Sector 137, Noida, Gautam Buddh Nagar, Uttar Pradesh, 201305
GFG App on Play Store GFG App on App Store
Advertise with us
  • Company
  • About Us
  • Legal
  • Privacy Policy
  • In Media
  • Contact Us
  • Advertise with us
  • GFG Corporate Solution
  • Placement Training Program
  • Languages
  • Python
  • Java
  • C++
  • PHP
  • GoLang
  • SQL
  • R Language
  • Android Tutorial
  • Tutorials Archive
  • DSA
  • Data Structures
  • Algorithms
  • DSA for Beginners
  • Basic DSA Problems
  • DSA Roadmap
  • Top 100 DSA Interview Problems
  • DSA Roadmap by Sandeep Jain
  • All Cheat Sheets
  • Data Science & ML
  • Data Science With Python
  • Data Science For Beginner
  • Machine Learning
  • ML Maths
  • Data Visualisation
  • Pandas
  • NumPy
  • NLP
  • Deep Learning
  • Web Technologies
  • HTML
  • CSS
  • JavaScript
  • TypeScript
  • ReactJS
  • NextJS
  • Bootstrap
  • Web Design
  • Python Tutorial
  • Python Programming Examples
  • Python Projects
  • Python Tkinter
  • Python Web Scraping
  • OpenCV Tutorial
  • Python Interview Question
  • Django
  • Computer Science
  • Operating Systems
  • Computer Network
  • Database Management System
  • Software Engineering
  • Digital Logic Design
  • Engineering Maths
  • Software Development
  • Software Testing
  • DevOps
  • Git
  • Linux
  • AWS
  • Docker
  • Kubernetes
  • Azure
  • GCP
  • DevOps Roadmap
  • System Design
  • High Level Design
  • Low Level Design
  • UML Diagrams
  • Interview Guide
  • Design Patterns
  • OOAD
  • System Design Bootcamp
  • Interview Questions
  • Inteview Preparation
  • Competitive Programming
  • Top DS or Algo for CP
  • Company-Wise Recruitment Process
  • Company-Wise Preparation
  • Aptitude Preparation
  • Puzzles
  • School Subjects
  • Mathematics
  • Physics
  • Chemistry
  • Biology
  • Social Science
  • English Grammar
  • Commerce
  • World GK
  • GeeksforGeeks Videos
  • DSA
  • Python
  • Java
  • C++
  • Web Development
  • Data Science
  • CS Subjects
@GeeksforGeeks, Sanchhaya Education Private Limited, All rights reserved
We use cookies to ensure you have the best browsing experience on our website. By using our site, you acknowledge that you have read and understood our Cookie Policy & Privacy Policy
Lightbox
Improvement
Suggest Changes
Help us improve. Share your suggestions to enhance the article. Contribute your expertise and make a difference in the GeeksforGeeks portal.
geeksforgeeks-suggest-icon
Create Improvement
Enhance the article with your expertise. Contribute to the GeeksforGeeks community and help create better learning resources for all.
geeksforgeeks-improvement-icon
Suggest Changes
min 4 words, max Words Limit:1000

Thank You!

Your suggestions are valuable to us.

What kind of Experience do you want to share?

Interview Experiences
Admission Experiences
Career Journeys
Work Experiences
Campus Experiences
Competitive Exam Experiences