Producer Configuration

A Kafka producer is configured through a flat map of string-keyed properties, and the handful of settings you choose determine your durability guarantees, throughput, ordering, and end-to-end latency. Most defaults are sensible for a quick start, but production workloads almost always tune acks, batching, idempotence, and timeouts deliberately. This page is a reference for the configs that matter most, what they do, and the defaults shipped in modern kafka-clients.

The core configuration map

Three properties are mandatory: the bootstrap servers and the two serializers. Everything else has a default. You build the producer by handing this map to the KafkaProducer constructor.

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.common.serialization.StringSerializer;
import java.util.Properties;

Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "broker1:9092,broker2:9092");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
props.put(ProducerConfig.ACKS_CONFIG, "all");
props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);
props.put(ProducerConfig.LINGER_MS_CONFIG, 10);
props.put(ProducerConfig.BATCH_SIZE_CONFIG, 32 * 1024);
props.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "lz4");

try (Producer<String, String> producer = new KafkaProducer<>(props)) {
    // send records...
}

Prefer the ProducerConfig constants over raw strings — they are compile-time checked and document the exact key names.

Configuration reference

The table below covers the configs you will touch most often. Defaults reflect current Kafka client releases.

Property	Default	What it controls
`bootstrap.servers`	(required)	Comma-separated `host:port` list used to discover the full cluster. Two or three brokers is enough for bootstrapping.
`key.serializer`	(required)	Class that turns the record key into bytes, e.g. `StringSerializer`.
`value.serializer`	(required)	Class that turns the record value into bytes, e.g. `JsonSerializer` or a schema-aware serializer.
`acks`	`all`	How many in-sync replicas must acknowledge a write: `0`, `1`, or `all` (`-1`). Drives durability.
`retries`	`2147483647`	Number of automatic resend attempts for retriable errors. Effectively bounded by `delivery.timeout.ms`.
`delivery.timeout.ms`	`120000`	Hard upper bound on the time from `send()` returning until success or failure, covering all retries.
`batch.size`	`16384`	Maximum bytes per partition batch. Larger batches improve throughput and compression.
`linger.ms`	`0`	How long to wait for more records before sending a batch. A few ms greatly improves batching.
`buffer.memory`	`33554432`	Total bytes available for buffering unsent records. When exhausted, `send()` blocks.
`max.in.flight.requests.per.connection`	`5`	Unacknowledged requests allowed per connection. With idempotence, `<= 5` preserves ordering.
`compression.type`	`none`	Batch compression codec: `none`, `gzip`, `snappy`, `lz4`, or `zstd`.
`enable.idempotence`	`true`	Prevents duplicate and out-of-order writes on retries. Requires `acks=all`.

Durability: acks and idempotence

acks is the single most important durability knob. acks=0 is fire-and-forget with no delivery guarantee; acks=1 waits only for the partition leader; acks=all waits for every in-sync replica and is the only setting that survives a leader failure without data loss. Pair acks=all with a broker-side min.insync.replicas=2 so a write is rejected if too few replicas are available.

Idempotence is enabled by default in modern clients. It deduplicates records on retry using a producer ID and sequence numbers, so you get exactly-once semantics per partition without code changes. It requires acks=all, retries > 0, and max.in.flight.requests.per.connection <= 5. If you explicitly set conflicting values, the client throws a ConfigException at startup.

Do not set enable.idempotence=false to “fix” a config error. Idempotence is nearly free and protects against silent duplicates caused by retried network timeouts.

Throughput: batching, linger, and compression

Throughput comes from sending fewer, larger requests. The producer groups records destined for the same partition into batches up to batch.size bytes. linger.ms tells the producer to wait briefly so more records can join a batch instead of sending one record at a time. The defaults (linger.ms=0) optimize for latency; setting linger.ms=5 to 20 and bumping batch.size to 32–64 KB dramatically improves throughput under load with minimal added latency.

Compression amplifies these gains because it operates on the whole batch. lz4 and zstd give the best ratio-to-CPU trade-off for most workloads. Use the same codec consistently so brokers can store batches as-is.

# Spring Boot application.yml
spring:
  kafka:
    bootstrap-servers: broker1:9092,broker2:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
      acks: all
      properties:
        enable.idempotence: true
        linger.ms: 10
        compression.type: lz4
      batch-size: 32768
      buffer-memory: 67108864

Backpressure and timeouts

buffer.memory caps how much unsent data the producer holds. When the application produces faster than the network can drain, the buffer fills and send() blocks for up to max.block.ms (default 60 s) before throwing TimeoutException. delivery.timeout.ms then bounds the lifetime of an accepted record across all retries. Because retries defaults to effectively infinite, delivery.timeout.ms is your real “give up” deadline.

You can confirm an applied configuration by logging the producer at startup.

INFO  o.a.k.clients.producer.ProducerConfig - ProducerConfig values:
        acks = all
        batch.size = 32768
        compression.type = lz4
        enable.idempotence = true
        linger.ms = 10

Best Practices

Always set bootstrap.servers to two or three brokers, not one, so bootstrapping survives a single broker outage.
Keep acks=all with broker min.insync.replicas=2 for any data you cannot afford to lose.
Leave enable.idempotence=true; it eliminates duplicates from retries at negligible cost.
Tune linger.ms (5–20 ms) and batch.size (32–64 KB) together to trade a little latency for large throughput gains.
Choose lz4 or zstd compression to shrink network and storage usage on text-heavy payloads.
Treat delivery.timeout.ms, not retries, as the knob that controls how long a record may be retried.
Size buffer.memory to absorb expected traffic bursts so producers do not block under spikes.