Spring Boot Kafka Consumer Example
Last Updated : 28 Feb, 2022
Spring Boot is one of the most popular and most used frameworks of Java Programming Language. It is a microservice-based framework and to make a production-ready application using Spring Boot takes very less time. Spring Boot makes it easy to create stand-alone, production-grade Spring-based Applications that you can "just run". So some of the main features of Spring boot are listed below.
- Create stand-alone Spring applications
- Embed Tomcat, Jetty, or Undertow directly.
- Provide 'starter' dependencies to simplify the build configuration.
- Configure Spring and 3rd party libraries Automatically whenever possible.
- Provide production-ready features like health checks, metrics, and externalized configuration.
- Almost no code generation and no requirement for XML configuration.
Apache Kafka is a publish-subscribe messaging system. A messaging system lets you send messages between processes, applications, and servers. Broadly Speaking, Apache Kafka is software where topics (A topic might be a category) can be defined and further processed. Applications may connect to this system and transfer a message onto the topic. A message can include any kind of information, from any event on your Personal blog or can be a very simple text message that would trigger any other event. Here we will be discussing how we can consume messages from Kafka topics and display them in our console with Spring Boot where Kafka is a pre-requisite.
Example:
Prerequisite: Make sure you have installed Apache Kafka in your local machine for which one should know How to Install and Run Apache Kafka on Windows?
Step 1: Go to this link and create a Spring Boot project. Add the "Spring for Apache Kafka" dependency to your Spring Boot project.
Step 2: Create a Configuration file named KafkaConfig. Below is the code for the KafkaConfig.java file.
Java // Java Program to Illustrate Kafka Configuration package com.amiya.kafka.apachekafkaconsumer.config; // Importing required classes import java.util.HashMap; import java.util.Map; import org.apache.kafka.clients.consumer.ConsumerConfig; import org.apache.kafka.common.serialization.StringDeserializer; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.kafka.annotation.EnableKafka; import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory; import org.springframework.kafka.core.ConsumerFactory; import org.springframework.kafka.core.DefaultKafkaConsumerFactory; // Annotations @EnableKafka @Configuration // Class public class KafkaConfig { @Bean public ConsumerFactory<String, String> consumerFactory() { // Creating a Map of string-object pairs Map<String, Object> config = new HashMap<>(); // Adding the Configuration config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092"); config.put(ConsumerConfig.GROUP_ID_CONFIG, "group_id"); config.put( ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); config.put( ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); return new DefaultKafkaConsumerFactory<>(config); } // Creating a Listener public ConcurrentKafkaListenerContainerFactory concurrentKafkaListenerContainerFactory() { ConcurrentKafkaListenerContainerFactory< String, String> factory = new ConcurrentKafkaListenerContainerFactory<>(); factory.setConsumerFactory(consumerFactory()); return factory; } } Step 3: Create a Consumer file named KafkaConsumer
Java // Java Program to Illustrate Kafka Consumer package com.amiya.kafka.apachekafkaconsumer.consumer; // Importing required classes import org.springframework.kafka.annotation.KafkaListener; import org.springframework.stereotype.Component; @Component // Class public class KafkaConsumer { @KafkaListener(topics = "NewTopic", groupId = "group_id") // Method public void consume(String message) { // Print statement System.out.println("message = " + message); } } Step 4: Now we have to do the following things in order to consume messages from Kafka topics with Spring Boot
- Run the Apache Zookeeper server
- Run the Apache Kafka server
- Send the messages from Kafka Topics
Run your Apache Zookeeper server by using this command
C:\kafka>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
Similarly, run your Apache Kafka server by using this command
C:\kafka>.\bin\windows\kafka-server-start.bat .\config\server.properties
Run the following command to send the messages from Kafka Topics
C:\kafka>.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic NewTopic
Step 5: Now run your spring boot application. Make sure you have changed the port number in the application.properties file
server.port=8081
Let's run the Spring boot application inside the ApacheKafkaConsumerApplication file
Output: In the output, you can see when you are sending the message from Kafka Topics it is displayed on the console in real-time.
Similar Reads
JSON using Jackson in REST API Implementation with Spring Boot When we build REST APIs with Spring Boot, we need to exclude NULL values from the JSON responses. This is useful when we want to optimize the data being transferred, making the response more compact and easier to process for the client.In this article, we are going to learn the approach that is used
4 min read
How to encrypt passwords in a Spring Boot project using Jasypt In this article, we will learn how to encrypt data in Spring Boot application config files like application.properties or application.yml. Inside those files, we can encrypt username, password, etc. You often come across developing projects where you have to connect to databases like MongoDB, etc, a
4 min read
Containerizing Spring Boot Application Java applications, known for their robustness sometimes become difficult to deploy and manage. This is where containerization comes into the picture. Packaging your Java app into a lightweight and self-contained independent unit can provide many benefits to the developers: PortabilityScalabilityFast
3 min read
How to Send Images in Spring Boot without using Servlet and Redundant JSP Codes? Open IntelliJ IDE and in the Controller section of your project inside the Controller class you need to build the Image serve method. The Image serve method looks like below and as you have to collect the Image response use @GetMapping Annotation to collect the image. ImageHandler Controller Method
2 min read
How to Create Todo List API using Spring Boot and MySQL? Spring Boot is built on the top of the spring and contains all the features of spring. And is becoming a favorite of developers these days because of its rapid production-ready environment which enables the developers to directly focus on the logic instead of struggling with the configuration and se
6 min read
Spring Boot - Transaction Management Using @Transactional Annotation The @Transactional annotation is the metadata used for managing transactions in the Spring Boot application. To configure Spring Transaction, this annotation can be applied at the class level or method level. In an enterprise application, a transaction is a sequence of actions performed by the appli
9 min read
Spring Boot - Map Entity to DTO using ModelMapper In enterprise applications, we use RESTful services to establish the communication between client and server. The general idea is that the client sends the request to the server and the server responds to that request with some response. Generally, we have three different layers in most of the appli
8 min read
Spring Boot | How to consume JSON messages using Apache Kafka Apache Kafka is a stream processing system that lets you send messages between processes, applications, and servers. In this article, we will see how to publish JSON messages on the console of a Spring boot application using Apache Kafka. In order to learn how to create a Spring Boot project, refer
3 min read
Spring Boot | How to consume string messages using Apache Kafka Apache Kafka is a publish-subscribe messaging queue used for real-time streams of data. A messaging queue lets you send messages between processes, applications, and servers. In this article we will see how to send string messages from apache kafka to the console of a spring boot application. Appro
3 min read
Spring Boot | How to publish String messages on Apache Kafka Apache Kafka is a publish-subscribe messaging system. A messaging queue lets you send messages between processes, applications, and servers. In this article, we will see how to send string messages to Apache Kafka in a spring boot application. In order to learn how to create a spring boot project, r
2 min read