Unclean Leader Election

Every Kafka partition has one leader that handles all reads and writes, and a set of followers that replicate it. When the leader fails, Kafka must promote one of its replicas. Unclean leader election is the controversial case where every in-sync replica is gone and Kafka is forced to choose between two bad options: stay offline until an up-to-date replica returns, or promote a stale replica and silently discard the records it never received. Understanding this knob is essential because it is the single setting that converts a durability guarantee into an availability gamble.

In-sync replicas and the ISR

Each partition tracks an in-sync replica set (ISR) — the replicas that are fully caught up with the leader. A follower stays in the ISR only while it keeps fetching within replica.lag.time.max.ms (default 30 s). Producers using acks=all are acknowledged only after every member of the ISR has the record, so anything acknowledged is durable across the whole ISR.

The ISR can shrink. If brokers crash or fall behind, replicas drop out, and in the worst case the ISR collapses to just the leader. If that leader then dies too, there are no in-sync replicas left to promote. This is the exact moment unclean leader election decides Kafka’s behaviour.

What unclean election actually does

The controller’s choice depends on unclean.leader.election.enable:

`unclean.leader.election.enable`	Behaviour when ISR is empty	Outcome
`false` (default)	Partition stays offline; waits for a former ISR member to return	No data loss, reduced availability
`true`	Promotes an out-of-sync replica as the new leader	Availability restored, unreplicated records lost

When unclean election is enabled and a stale replica is promoted, every record the old leader had acknowledged but not yet replicated to that follower is gone permanently. Worse, the new leader’s log becomes the source of truth, so when the old, more-complete leader rejoins, it truncates its log to match the new leader — actively deleting committed data. This breaks the durability contract that acks=all appears to promise.

Warning: Unclean leader election can violate at-least-once delivery even though producers received successful acknowledgements. A 200-style ack from Kafka does not survive an unclean election. Treat enabling it as accepting silent data loss.

Configuring the behaviour

The setting is a broker default and can be overridden per topic. The default is false, and modern Kafka (KRaft mode) keeps it that way.

# Broker-level default (server.properties)
unclean.leader.election.enable=false

Override or inspect it per topic with the admin CLI:

# Force durability for a critical topic
kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type topics --entity-name payments \
  --alter --add-config unclean.leader.election.enable=false

# Inspect the effective value
kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type topics --entity-name payments --describe

Output:

Dynamic configs for topic payments are:
  unclean.leader.election.enable=false sensitive=false synonyms={DYNAMIC_TOPIC_CONFIG:unclean.leader.election.enable=false, DEFAULT_CONFIG:unclean.leader.election.enable=false}

You can set the same property when creating a topic programmatically with the Java AdminClient:

import org.apache.kafka.clients.admin.Admin;
import org.apache.kafka.clients.admin.NewTopic;
import java.util.Map;
import java.util.Properties;

public final class TopicProvisioner {

    public static void createDurableTopic(String bootstrapServers) throws Exception {
        Properties props = new Properties();
        props.put("bootstrap.servers", bootstrapServers);

        try (Admin admin = Admin.create(props)) {
            NewTopic topic = new NewTopic("payments", 6, (short) 3)
                .configs(Map.of(
                    "min.insync.replicas", "2",
                    "unclean.leader.election.enable", "false"
                ));
            admin.createTopics(java.util.List.of(topic)).all().get();
        }
    }
}

The availability-vs-durability trade-off

This setting is a direct expression of the CAP-style trade-off, scoped to a single partition:

false (durability first): No acknowledged record is ever lost to a leader change. The price is that a partition with a wiped-out ISR becomes unavailable for reads and writes until a sufficiently up-to-date broker recovers. Pair this with min.insync.replicas=2 and acks=all for a strong durability baseline.
true (availability first): The partition recovers as soon as any replica is alive, accepting that recently acknowledged data may vanish. This only makes sense when stale-but-available data beats no data.

For most systems — payments, orders, the source of truth for an event-sourced domain — durability wins and you keep the default false.

When `true` might be acceptable

There are narrow cases where availability genuinely outranks perfect durability:

High-volume, low-value telemetry or metrics where a brief gap is tolerable and downtime is not.
Caches or materialised views that can be rebuilt from an upstream source of truth.
Approximate analytics streams where losing a few seconds of events does not change conclusions.

Even then, prefer fixing the root cause — too few replicas, undersized brokers, or rack/AZ skew that lets a single failure domain take out the whole ISR. Reaching for unclean election is usually a symptom that replication was under-provisioned.

Best practices

Keep unclean.leader.election.enable=false (the default) for any topic where acknowledged data must not be lost.
Combine acks=all, replication.factor=3, and min.insync.replicas=2 so the ISR can survive one broker failure without going empty.
Spread replicas across racks/availability zones with broker.rack so a single failure domain cannot collapse the entire ISR.
Only enable unclean election per topic, never as a blanket broker default, and document explicitly why each such topic accepts data loss.
Alert on UnderReplicatedPartitions and OfflinePartitionsCount so you address a shrinking ISR before it forces the unclean-vs-offline decision.
Treat a partition stuck offline as an operational incident to recover the right broker — not a reason to flip the switch to true under pressure.