Consumer Configuration

A Kafka consumer is configured through a flat map of string-keyed properties, and the settings you pick determine which records you receive, how offsets are committed, how the group stays balanced, and how throughput trades off against latency. The defaults work for a demo, but production consumers almost always tune offset commit behavior, poll sizing, and the timeouts that govern group membership. This page is a reference for the configs that matter most, what each one does, and the defaults shipped in modern kafka-clients.

The core configuration map

Four properties anchor every consumer: the bootstrap servers, a group.id, and the two deserializers that mirror the producer’s serializers. Everything else has a default. You build the consumer by handing this map to the KafkaConsumer constructor and then subscribing to one or more topics.

import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.consumer.Consumer;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.common.serialization.StringDeserializer;
import java.util.List;
import java.util.Properties;

Properties props = new Properties();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "broker1:9092,broker2:9092");
props.put(ConsumerConfig.GROUP_ID_CONFIG, "order-processor");
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, 500);

try (Consumer<String, String> consumer = new KafkaConsumer<>(props)) {
    consumer.subscribe(List.of("orders"));
    // poll loop...
}

Prefer the ConsumerConfig constants over raw strings — they are compile-time checked and document the exact key names.

Configuration reference

The table below covers the configs you will touch most often. Defaults reflect current Kafka client releases.

Property	Default	What it controls
`bootstrap.servers`	(required)	Comma-separated `host:port` list used to discover the full cluster. Two or three brokers is enough for bootstrapping.
`group.id`	(none)	Identifies the consumer group. Members sharing an ID split the topic’s partitions among themselves.
`key.deserializer`	(required)	Class that turns the record key’s bytes back into an object, e.g. `StringDeserializer`.
`value.deserializer`	(required)	Class that turns the record value’s bytes back into an object, e.g. `JsonDeserializer`.
`enable.auto.commit`	`true`	Whether the client commits offsets automatically on a timer. Set `false` for at-least-once control.
`auto.commit.interval.ms`	`5000`	How often offsets are committed when auto-commit is on. Only applies if `enable.auto.commit=true`.
`auto.offset.reset`	`latest`	Where to start when no committed offset exists: `earliest`, `latest`, or `none` (throw).
`max.poll.records`	`500`	Maximum records returned by a single `poll()` call. Caps the batch your code processes per loop.
`max.poll.interval.ms`	`300000`	Max time allowed between `poll()` calls before the broker assumes the consumer is dead and rebalances.
`fetch.min.bytes`	`1`	Minimum data the broker waits to accumulate before answering a fetch. Higher values batch more.
`fetch.max.wait.ms`	`500`	Max time the broker waits to satisfy `fetch.min.bytes` before responding with whatever it has.
`session.timeout.ms`	`45000`	How long the group coordinator waits without a heartbeat before evicting the member.
`heartbeat.interval.ms`	`3000`	How often the consumer sends heartbeats. Keep it well below `session.timeout.ms` (≈ one third).
`isolation.level`	`read_uncommitted`	`read_committed` hides records from aborted transactions; required for exactly-once reads.

Group identity and offset reset

group.id is what makes a consumer part of a group. Every member with the same ID cooperatively divides the subscribed partitions, and Kafka tracks committed offsets per group. Change the ID and you effectively start a brand-new subscription with no committed position.

When a group has no committed offset for a partition — a fresh group, or one whose offsets have aged out — auto.offset.reset decides where to begin. earliest replays from the start of the log, latest (the default) skips everything written before the consumer joined, and none throws NoOffsetForPartitionException so you can handle the case explicitly. The choice is silent but consequential: a defaulted latest is the classic reason a new consumer “misses” historical data.

auto.offset.reset only fires when there is no valid committed offset. It does not control behavior on normal restarts — those resume from the last commit. Do not rely on it as a general “start position” knob.

Offset commits: auto vs manual

With enable.auto.commit=true, the client commits the offsets it last returned every auto.commit.interval.ms during poll(). This is convenient but at-most-once-ish: offsets can be committed before your processing actually finishes, so a crash loses in-flight records. For at-least-once delivery, set enable.auto.commit=false and commit yourself after the work succeeds.

while (running) {
    var records = consumer.poll(java.time.Duration.ofMillis(1000));
    for (var record : records) {
        process(record); // your business logic
    }
    consumer.commitSync(); // commit only after the batch is fully processed
}

Poll sizing and liveness

max.poll.records bounds how many records one poll() returns, while max.poll.interval.ms bounds how long your code may spend processing them before the next poll(). If processing a batch takes longer than max.poll.interval.ms, the coordinator considers the consumer stuck and triggers a rebalance — a frequent cause of “consumer keeps leaving the group” symptoms. The fix is to lower max.poll.records, speed up processing, or raise the interval.

Liveness is also tracked by heartbeats on a background thread: heartbeat.interval.ms sets the cadence and session.timeout.ms sets the eviction deadline. Fetch tuning shapes throughput — raising fetch.min.bytes (with fetch.max.wait.ms as a ceiling) lets the broker return fuller responses, cutting request overhead at the cost of a little latency.

# Spring Boot application.yml
spring:
  kafka:
    bootstrap-servers: broker1:9092,broker2:9092
    consumer:
      group-id: order-processor
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
      auto-offset-reset: earliest
      enable-auto-commit: false
      max-poll-records: 500
      fetch-min-size: 65536
      properties:
        isolation.level: read_committed
        max.poll.interval.ms: 300000
    listener:
      ack-mode: manual

You can confirm an applied configuration by logging the consumer at startup.

INFO  o.a.k.clients.consumer.ConsumerConfig - ConsumerConfig values:
        auto.offset.reset = earliest
        enable.auto.commit = false
        group.id = order-processor
        isolation.level = read_committed
        max.poll.records = 500

Best Practices

Set a stable, descriptive group.id; treat changing it as starting a new subscription.
Choose auto.offset.reset deliberately — use earliest when missing historical data is unacceptable.
Turn off enable.auto.commit and commit after processing for reliable at-least-once delivery.
Keep batch processing time under max.poll.interval.ms; lower max.poll.records if loops run long.
Set heartbeat.interval.ms to roughly one third of session.timeout.ms so transient pauses do not evict members.
Raise fetch.min.bytes to batch reads on high-throughput topics, bounding the wait with fetch.max.wait.ms.
Use isolation.level=read_committed whenever producers write transactionally and you need exactly-once reads.