[
https://issues.apache.org/jira/browse/KAFKA-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794433#comment-17794433
]
Greg Harris commented on KAFKA-15985:
-------------------------------------
Hi [~Reamer] The default value of `offset.lag.max` is 100, which could cause
the 31 < 100 lag you're seeing. You can work-around this by setting
`offset.lag.max` to `0`.
We're exploring a fix to this in KAFKA-15906.
> Mirrormaker 2 offset sync is incomplete
> ---------------------------------------
>
> Key: KAFKA-15985
> URL: https://issues.apache.org/jira/browse/KAFKA-15985
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 3.5.1
> Reporter: Philipp Dallig
> Priority: Major
>
> We are currently trying to migrate between two Kafka clusters using
> Mirrormaker2
> new kafka cluster version: 7.5.2-ccs
> old kafka cluster version: kafka_2.13-2.8.0
> The Mirrormaker 2 process runs on the new cluster (target cluster).
> My main problem: The lag in the target cluster is not the same as in the
> source cluster.
> I have set up a producer and consumer against the old Kafka cluster. If I
> stop both. The lag in the old Kafka cluster is 0, while it is > 0 in the new
> Kafka cluster.
> target cluster
> {code}
> GROUP TOPIC PARTITION CURRENT-OFFSET
> LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
> test-sync-5 kafka-replication-test-5 0 3637 3668
> 31 - - -
> {code}
> source cluster
> {code}
> GROUP TOPIC PARTITION CURRENT-OFFSET
> LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
> test-sync-5 kafka-replication-test-5 0 3668 3668
> 0 - - -
> {code}
> MM2 configuration without connection properties.
> {code}
> t-kafka->t-extkafka.enabled = true
> t-kafka->t-extkafka.topics = ops_filebeat, kafka-replication-.*
> t-kafka->t-extkafka.sync.topic.acls.enabled = false
> t-kafka->t-extkafka.sync.group.offsets.enabled = true
> t-kafka->t-extkafka.sync.group.offsets.interval.seconds = 30
> t-kafka->t-extkafka.refresh.groups.interval.seconds = 30
> t-kafka->t-extkafka.offset-syncs.topic.location = target
> t-kafka->t-extkafka.emit.checkpoints.interval.seconds = 30
> t-kafka->t-extkafka.replication.policy.class =
> org.apache.kafka.connect.mirror.IdentityReplicationPolicy
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
