Setup & Auto-Configuration

Spring for Apache Kafka turns the low-level producer and consumer clients into idiomatic Spring beans you can inject and test like any other dependency. The real productivity win, though, comes from Spring Boot’s auto-configuration: add one dependency, set a handful of spring.kafka.* properties, and Boot wires up a ready-to-use KafkaTemplate, ConsumerFactory, and listener container infrastructure for you. This page walks through adding the dependency, configuring brokers and (de)serializers in application.yml, and understanding exactly which beans Boot creates so you know what you can override. The stack is Spring Boot 3.x, Spring for Apache Kafka, and Java 17+.

Adding the dependency

Spring Boot does not ship a separate spring-boot-starter-kafka; instead you add spring-kafka directly and Boot’s dependency management pins a compatible version through the parent BOM. With the spring-boot-starter-parent (or the Boot Gradle plugin) in place, you omit the version entirely.

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>

For Gradle:

implementation 'org.springframework.kafka:spring-kafka'

This brings in kafka-clients transitively, so you do not declare it yourself. If you also write tests against an embedded broker, add spring-kafka-test with <scope>test</scope>.

Always let the Spring Boot BOM choose the spring-kafka and kafka-clients versions. Mixing a hand-picked kafka-clients with a Boot-managed spring-kafka is the most common source of NoSuchMethodError and ClassNotFoundException at startup.

Configuring brokers, producers, and consumers

All Spring Kafka settings live under the spring.kafka prefix. The single mandatory value is bootstrap-servers — the broker addresses the client uses for the initial cluster discovery. Everything else has sensible defaults, but in practice you set serializers, a consumer group id, and an offset reset policy explicitly.

spring:
  kafka:
    bootstrap-servers: localhost:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
      acks: all
    consumer:
      group-id: order-service
      auto-offset-reset: earliest
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
      properties:
        spring.json.trusted.packages: "com.example.events"

A few of these properties carry the most weight:

Property	Purpose	Common values
`bootstrap-servers`	Initial broker list for cluster discovery	`host1:9092,host2:9092`
`consumer.group-id`	Consumer group for offset tracking and partition assignment	any logical app name
`consumer.auto-offset-reset`	Where to start when no committed offset exists	`earliest`, `latest`, `none`
`producer.key-serializer` / `value-serializer`	Converts keys/values to bytes	`StringSerializer`, `JsonSerializer`
`consumer.key-deserializer` / `value-deserializer`	Converts bytes back to objects	`StringDeserializer`, `JsonDeserializer`
`producer.acks`	Durability guarantee for writes	`all`, `1`, `0`

The auto-offset-reset value only applies the first time a group reads a partition (or after its committed offset has expired). earliest replays the whole topic, latest skips everything written before the consumer started, and none throws if no offset exists — choose deliberately. The nested properties: block passes any raw Kafka client property that Boot does not expose as a first-class key; above, spring.json.trusted.packages whitelists the packages the JsonDeserializer is allowed to deserialize into.

What Boot auto-configures

When spring-kafka is on the classpath, KafkaAutoConfiguration reads your spring.kafka.* properties (bound into a KafkaProperties object) and creates these beans, each marked @ConditionalOnMissingBean so you can replace any of them:

ProducerFactory — a DefaultKafkaProducerFactory built from spring.kafka.producer.*.
KafkaTemplate — the high-level send API, backed by that producer factory.
ConsumerFactory — a DefaultKafkaConsumerFactory built from spring.kafka.consumer.*.
ConcurrentKafkaListenerContainerFactory — the factory @KafkaListener methods use to build their message-listener containers.
KafkaAdmin — registers NewTopic beans against the broker on startup.

Because these beans exist, a service only needs to inject KafkaTemplate to produce, and annotate a method with @KafkaListener to consume — no manual factory wiring.

import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;

public record OrderPlaced(String orderId, int quantity) {}

@Service
public class OrderProducer {

    private final KafkaTemplate<String, OrderPlaced> kafkaTemplate;

    public OrderProducer(KafkaTemplate<String, OrderPlaced> kafkaTemplate) {
        this.kafkaTemplate = kafkaTemplate;
    }

    public void publish(OrderPlaced event) {
        kafkaTemplate.send("orders", event.orderId(), event);
    }
}

On startup with logging.level.org.apache.kafka.clients=INFO, the clients log the resolved configuration so you can confirm Boot applied your YAML:

Output:

INFO o.a.k.c.producer.ProducerConfig : ProducerConfig values:
    acks = all
    bootstrap.servers = [localhost:9092]
    key.serializer = class org.apache.kafka.common.serialization.StringSerializer
    value.serializer = class org.springframework.kafka.support.serializer.JsonSerializer
INFO o.a.k.clients.consumer.ConsumerConfig : ConsumerConfig values:
    auto.offset.reset = earliest
    group.id = order-service
    bootstrap.servers = [localhost:9092]

Overriding the defaults

When the property surface is not enough — for example, you need two producer factories with different serializers — declare the bean yourself and Boot backs off. You can still read the bound properties through KafkaProperties.

@Configuration
public class KafkaConfig {

    @Bean
    public ProducerFactory<String, byte[]> byteProducerFactory(KafkaProperties props) {
        return new DefaultKafkaProducerFactory<>(props.buildProducerProperties());
    }
}

Best Practices

Pin only spring-kafka and let the Boot BOM manage kafka-clients to avoid version drift.
Set bootstrap-servers, consumer.group-id, and auto-offset-reset explicitly in every environment — never rely on implicit defaults for the group id.
Set acks: all on producers for durable writes; pair it with enable-idempotence: true for exactly-once-style delivery semantics.
Restrict spring.json.trusted.packages to your event packages rather than * to avoid deserializing untrusted types.
Externalize broker addresses and credentials with profiles or environment variables instead of hardcoding them in application.yml.
Define topics as NewTopic beans so KafkaAdmin creates them with the right partition and replication counts, instead of relying on broker auto-creation.