Partitions and replication

The producer sends data directly to the broker that is the leader for a partition; any Kafka node can answer a metadata request about which servers are alive and where each partition's leader currently is. The client controls which partition a message goes to: at random for load balancing, or by a key hashed to a partition — for example, a user id key sends all data for that user to the same partition — and the partition function can be overridden. At its heart a partition is a replicated log: the leader chooses the ordering of values and followers copy it. The guarantee a replication algorithm must provide is that once a client is told a message is committed, any newly elected leader must also have it. Rather than majority vote, Kafka dynamically maintains a set of in-sync replicas (ISR) caught up to the leader; a write is committed only when all in-sync replicas have received it, only ISR members are eligible for election as leader, and with f+1 replicas a topic tolerates f failures without losing committed messages. Scope: Apache Kafka. Reviewed October 4, 2026.