Can multiple Kafka consumers read the same message from a partition?

Let’s redefine the question a little bit. Can multiple Kafka consumers from a consumer group read the same message from a partition?

The short answer to the above redefined question is No.

How is it possible? – Consumer offset

Consumer offset is used to track the messages that are consumed by consumers in a consumer group. A topic can be consumed by many consumer groups and each consumer group will have many consumers. A topic is divided into multiple partitions.

A consumer in a consumer group is assigned to a partition. Only one consumer is assigned to a partition. A consumer can be assigned to consume multiple partitions.

Consumer offset is managed at the partition level per consumer group. In summary –

• Topic has many partitions
• Topic can have many consumer groups
• Only one consumer in a consumer group can be assigned to consume messages from a partition
• A consumer offset is managed at the partition level per consumer group

So the rule in Kafka is only one consumer in a consumer group can be assigned to consume messages from a partition in a topic and hence multiple Kafka consumers from a consumer group can not read the same message from a partition.

Interested in learning more about consumer offset? Check out this post.