What are the reasons of Kafka rebalancing?

Let us consider every topic in Kafka is a table. The table has partitions similarly the topic also has partitions. Every consumer is assigned one or more topic partitions exclusively.

What is a rebalance in Kafka?

The rebalance in Kafka is the re-assignment of partitions among consumers. In a consumer group, the consumer is assigned a partition. When the rebalance happens the group co-ordinator revokes the partitions from the consumer and assigns new partitions.

When a rebalance happens?

  • When a consumer JOINS the group
  • When a consumer is DEAD
  • When a consumer is considered DEAD by the group coordinator. This may happen after a crash or when the consumer is busy with a long-running processing, which means that no heartbeats has been sent in the meanwhile by the consumer to the group coordinator within the configured session interval
  • When new partitions are added
  • Topic is created or deleted

Kafka consumer group triggers a rebalance
What does rebalance mean in Kafka