[
https://issues.apache.org/jira/browse/KAFKA-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bale updated KAFKA-13932:
-------------------------
Description:
Kafka version: 2.8.0
Replication flow: A -> B
Topic: DL2 -> A.DL2
I use mm2 to replicate cluster A data to cluster B. In some cases, I found
consumer group offset is bigger than partition offset in cluster B which will
cause data loss of the mirror topic.
[Steps]
1. Start mm2 with the attachment: connect-mirror-maker.properties.
2.Create topic DL2, push 5 messages and consume with group g1. Offset info of
DL2 and A.DL2：please see attachment offset1.png
3.Stop mm2 and push 5 messages to the topic DL2. Offset info of DL2 and A.DL2：
please see attachment offset2.png
4. After 1 hour, the data of topic DL2 is deleted.
5.Restart mm2 and sync consumer group offset, found the consume offset is
bigger than partiton offset. Offset info of DL2 and A.DL2: please see
attachment offset3.png
was:
Kafka version: 2.8.0
Replication flow: A -> B
Topic: DL2 -> A.DL2
I use mm2 to replicate cluster A data to cluster B. In some cases, I found
consumer group offset is bigger than partition offset in cluster B which will
cause data loss of the mirror topic.
[Steps]
1. Start mm2 with the attachment: connect-mirror-maker.properties.
2.Create topic DL2, push 5 messages and consume with group g1. offset
info：offset1.png
3.Stop mm2 and push 5 messages to the topic DL2.
4. After 1 hour, the data of topic DL2 is deleted.
5.Restart mm2 and sync consumer group offset, found the consume offset is
bigger than server offset
> Replication data loss in some cases
> -----------------------------------
>
> Key: KAFKA-13932
> URL: https://issues.apache.org/jira/browse/KAFKA-13932
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Affects Versions: 2.8.0
> Reporter: Bale
> Priority: Minor
> Attachments: connect-mirror-maker.properties, offset1.png,
> offset2.png, offset3.png
>
>
> Kafka version: 2.8.0
> Replication flow: A -> B
> Topic: DL2 -> A.DL2
>
> I use mm2 to replicate cluster A data to cluster B. In some cases, I found
> consumer group offset is bigger than partition offset in cluster B which will
> cause data loss of the mirror topic.
>
> [Steps]
> 1. Start mm2 with the attachment: connect-mirror-maker.properties.
> 2.Create topic DL2, push 5 messages and consume with group g1. Offset info of
> DL2 and A.DL2：please see attachment offset1.png
> 3.Stop mm2 and push 5 messages to the topic DL2. Offset info of DL2 and
> A.DL2： please see attachment offset2.png
> 4. After 1 hour, the data of topic DL2 is deleted.
> 5.Restart mm2 and sync consumer group offset, found the consume offset is
> bigger than partiton offset. Offset info of DL2 and A.DL2: please see
> attachment offset3.png
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
