[
https://issues.apache.org/jira/browse/KAFKA-12498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olumide Ajiboye updated KAFKA-12498:
------------------------------------
Description:
I have two Kafka clusters in Active\Passive replication using MM 2.0.
When I produce messages to a topic and try to read from it in the same
cluster, the consumer group lag is already set to zero. The replica topic also
has the same lag and log-end-offset.
MM2 is using a superuser client and I am using a separate consumer-group with
separate credentials and with permission just to read and write to this topic.
Steps to reproduce:
* Setup cluster and DR cluster with MM2 in uni-directional replication.
* Create a topic.
* Write 10 messages to the topic without any active consumers but with MM2
replicating the topic to a passive Cluster.
* Attempt to read 3 messages from the topic, this creates my consumer group
and adds an active consumer. The result is no messages are read
* Describe the consumer group, the result shows Log-End-Offset with correct
number of messages, but Lag shows 0
* Attempt to read 3 messages from passive cluster using same consumer-group.
Result: no messages consumed, Lag shows 0, Log-End-Offset shows correct number
of messages (i.e. same as active cluster)
* Keep consumer running
* Write a few more messages.
* Consumer is now reading latest messages
* Stop consumer
* Keep writing new messages
* Lag shows correct value.
In the absence of MM2, Kafka operation is as expected.
was:
I have two Kafka clusters in Active\Passive replication using MM 2.0.
When I produce messages to a topic and try to read from it in the same cluster,
the consumer group lag is already set to zero. The replica topic also has the
same lag and log-end-offset.
My MM2 is using a superuser account and I am using a separate consumer-group
with permission just to read and write to this topic.
Just to elaborate further, * Write 10 messages to a topic without any active
consumers but with MM2 replicating the topic to a passive Cluster.
* Attempt to read 3 messages from the topic, this creates my consumer group
and adds an active consumer. The result is no messages are read
* Describe the consumer group, the result shows Log-End-Offset with correct
number of messages, but Lag shows 0
* Attempt to read 3 messages from passive cluster using same consumer-group.
Result: no messages consumed, Lag shows 0, Log-End-Offset shows correct number
of messages (i.e. same as active cluster)
* Keep consumer running
* Write a few more messages.
* Consumer is now reading latest messages
* Stop consumer
* Keep writing new messages
* Lag shows correct value.
In the absence of MM2, Kafka operation is as expected.
> Consumer Group Lag is zero when MirrorMaker 2.0 is running
> ----------------------------------------------------------
>
> Key: KAFKA-12498
> URL: https://issues.apache.org/jira/browse/KAFKA-12498
> Project: Kafka
> Issue Type: Bug
> Components: consumer, replication
> Affects Versions: 2.7.0
> Environment: Kubernetes 1.19.7
> Strimzi 0.21.1
> Reporter: Olumide Ajiboye
> Priority: Major
>
> I have two Kafka clusters in Active\Passive replication using MM 2.0.
> When I produce messages to a topic and try to read from it in the same
> cluster, the consumer group lag is already set to zero. The replica topic
> also has the same lag and log-end-offset.
> MM2 is using a superuser client and I am using a separate consumer-group with
> separate credentials and with permission just to read and write to this
> topic.
> Steps to reproduce:
> * Setup cluster and DR cluster with MM2 in uni-directional replication.
> * Create a topic.
> * Write 10 messages to the topic without any active consumers but with MM2
> replicating the topic to a passive Cluster.
> * Attempt to read 3 messages from the topic, this creates my consumer group
> and adds an active consumer. The result is no messages are read
> * Describe the consumer group, the result shows Log-End-Offset with correct
> number of messages, but Lag shows 0
> * Attempt to read 3 messages from passive cluster using same consumer-group.
> Result: no messages consumed, Lag shows 0, Log-End-Offset shows correct
> number of messages (i.e. same as active cluster)
> * Keep consumer running
> * Write a few more messages.
> * Consumer is now reading latest messages
> * Stop consumer
> * Keep writing new messages
> * Lag shows correct value.
> In the absence of MM2, Kafka operation is as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
