A consumer group in Kafka composes of one or more consumers where each consumer reads from different partitions of a topic. As each consumer reads from separate partitions, this guarantees that the message will not be duplicated.
What is a consumer group co-ordinator?
One of the Kafka brokers gets elected as a group co-ordinator. The group co-ordinator makes the group leader and group members. When a group member leaves or a new member joins the group co-ordinator initiate the rebalance of partitions to the group members.
- Manage a list of consumer group members
- Initiate the rebalance activity
- Block the consumer reading when the rebalance happening
- Communicate with the consumer about the new partition assignment
What is a consumer group leader?
The first member joins in a consumer group becomes the consumer group leader. The group co-ordinator manages that. The consumer group leader executes rebalance activities.
- Execute the rebalance activities
- Sends the new partition assignment to the group co-ordinator