Producer Configuration (Spring)

Every KafkaTemplate you inject is backed by a ProducerFactory, and that factory holds the configuration map that controls durability, throughput, and ordering guarantees of your messages. Spring Boot can build the factory for you from application.yml, or you can declare a DefaultKafkaProducerFactory bean and own the configuration map directly. Getting these settings right — acks, enable.idempotence, batching, and compression — is the difference between a producer that silently drops data under load and one that delivers exactly-once-per-partition with high throughput.

Configuring via application.yml

The fastest path is to let Spring Boot’s auto-configuration read the spring.kafka.producer.* properties. These map onto the underlying org.apache.kafka.clients.producer.ProducerConfig keys, and Spring wires the resulting ProducerFactory and a KafkaTemplate bean automatically. Anything that does not have a dedicated property — such as enable.idempotence — goes under the free-form properties block, where keys use the raw dotted client names.

spring:
  kafka:
    bootstrap-servers: localhost:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
      acks: all
      retries: 2147483647
      batch-size: 32768          # 32 KB per partition batch
      buffer-memory: 67108864    # 64 MB total send buffer
      compression-type: lz4
      properties:
        enable.idempotence: true
        linger.ms: 20
        max.in.flight.requests.per.connection: 5
        delivery.timeout.ms: 120000
        spring.json.add.type.headers: false

With enable.idempotence: true, the broker deduplicates retries so that an at-least-once retry never produces a duplicate on the partition. Note that idempotence requires acks=all, retries > 0, and max.in.flight.requests.per.connection <= 5; Spring (and the client) will fail fast at startup if you contradict these.

Setting acks=all and enable.idempotence=true is the recommended baseline for any producer carrying business data. The throughput cost is small, and it eliminates both silent data loss and duplicates caused by retries.

Key producer properties

The table below summarizes the settings you will tune most often. The middle column shows the application.yml location relative to spring.kafka.producer.

Property	YAML key	What it controls
`acks`	`acks`	Durability: `0` (none), `1` (leader), `all` (full ISR). Use `all`.
`enable.idempotence`	`properties.enable.idempotence`	Dedupe retries per partition.
`retries`	`retries`	Number of retry attempts on transient errors.
`batch.size`	`batch-size`	Max bytes batched per partition before sending.
`linger.ms`	`properties.linger.ms`	Wait time to fill a batch, trading latency for throughput.
`compression.type`	`compression-type`	`none`, `gzip`, `snappy`, `lz4`, or `zstd`.
`buffer.memory`	`buffer-memory`	Total memory for unsent records before blocking.
`delivery.timeout.ms`	`properties.delivery.timeout.ms`	Upper bound on the time `send()` may take, including retries.

Configuring via a @Bean

When you need full programmatic control — multiple factories, runtime-computed brokers, per-environment serializers, or distinct templates for different value types — declare the factory yourself. You build a Map<String, Object> of ProducerConfig keys, pass it to DefaultKafkaProducerFactory, and expose a KafkaTemplate on top of it. This bean takes precedence over the auto-configured one.

package com.devcraftly.kafka.config;

import java.util.HashMap;
import java.util.Map;

import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;
import org.springframework.kafka.support.serializer.JsonSerializer;

import com.devcraftly.kafka.OrderEvent;

@Configuration
public class KafkaProducerConfig {

    @Bean
    public ProducerFactory<String, OrderEvent> orderProducerFactory() {
        Map<String, Object> props = new HashMap<>();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
        props.put(ProducerConfig.ACKS_CONFIG, "all");
        props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);
        props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE);
        props.put(ProducerConfig.BATCH_SIZE_CONFIG, 32 * 1024);
        props.put(ProducerConfig.LINGER_MS_CONFIG, 20);
        props.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "lz4");
        props.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, 5);
        // Don't emit Spring's __TypeId__ header for cross-language consumers.
        props.put(JsonSerializer.ADD_TYPE_INFO_HEADERS, false);
        return new DefaultKafkaProducerFactory<>(props);
    }

    @Bean
    public KafkaTemplate<String, OrderEvent> orderKafkaTemplate(
            ProducerFactory<String, OrderEvent> orderProducerFactory) {
        return new KafkaTemplate<>(orderProducerFactory);
    }
}

The DTO carried by this template is a simple immutable record, serialized as JSON by JsonSerializer.

package com.devcraftly.kafka;

import java.math.BigDecimal;
import java.time.Instant;

public record OrderEvent(
        String orderId,
        String customerId,
        BigDecimal amount,
        Instant occurredAt) {
}

Verifying the effective configuration

The producer logs its resolved configuration at INFO on first use, which is the quickest way to confirm that idempotence and acks actually took effect — values copied incorrectly under properties are easy to miss otherwise.

ProducerConfig values:
    acks = -1
    batch.size = 32768
    compression.type = lz4
    enable.idempotence = true
    linger.ms = 20
    max.in.flight.requests.per.connection = 5
    retries = 2147483647

Note that acks = -1 is the wire representation of acks=all — they are identical.

Best practices

Default to acks=all with enable.idempotence=true for all data-carrying producers; reserve acks=1/acks=0 for disposable telemetry only.
Tune linger.ms (10-50 ms) together with batch.size to amortize network round-trips; larger batches plus compression dramatically improve throughput.
Prefer lz4 or zstd compression — they offer a strong ratio with low CPU cost and shrink both broker storage and network usage.
Keep max.in.flight.requests.per.connection at 5 or below when idempotence is on, so retries cannot reorder records within a partition.
Use a @Bean factory when you need distinct serializers or broker sets per template; otherwise keep configuration in application.yml for clarity.
Set delivery.timeout.ms deliberately — it bounds the total send time and should exceed linger.ms + request.timeout.ms with margin for retries.