What is a consumer group in Kafka?

Consumer group

A consumer group in Kafka composes of one or more consumers where each consumer reads from different partitions of a topic. As each consumer reads from separate partitions, this guarantees that the message will not be duplicated.

What is a consumer group co-ordinator?

One of the Kafka brokers gets elected as a group co-ordinator. The group co-ordinator makes the group leader and group members. When a group member leaves or a new member joins the group co-ordinator initiate the rebalance of partitions to the group members.

  • Manage a list of consumer group members
  • Initiate the rebalance activity
  • Block the consumer reading when the rebalance happening
  • Communicate with the consumer about the new partition assignment

What is a consumer group leader?

The first member joins in a consumer group becomes the consumer group leader. The group co-ordinator manages that. The consumer group leader executes rebalance activities.

  • Execute the rebalance activities
  • Sends the new partition assignment to the group co-ordinator