[
https://issues.apache.org/jira/browse/KAFKA-20643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jheng-Sing Chen updated KAFKA-20643:
------------------------------------
Description:
OffsetSyncWriter.PartitionState currently determines whether the translated
downstream offset is too stale using:
downstreamOffset - (lastSyncDownstreamOffset + 1) >= maxOffsetLag
The +1 is required because OffsetSyncStore.translateDownstream translates
consumer group offsets beyond the latest sync to at most one downstream offset
past the sync.
This relationship is currently enforced only by a comment/TODO in
OffsetSyncWriter. If the translation logic in OffsetSyncStore changes without a
matching update in OffsetSyncWriter, stale offset sync detection may become
inconsistent.
We should extract the shared downstream translation boundary logic into a
common helper and use it from both OffsetSyncStore.translateDownstream and
OffsetSyncWriter.PartitionState.update.
TODO link:
(https://github.com/apache/kafka/blob/trunk/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/OffsetSyncWriter.java#L172-L174)
was:
OffsetSyncWriter.PartitionState currently determines whether the translated
downstream offset is too stale using:
downstreamOffset - (lastSyncDownstreamOffset + 1) >= maxOffsetLag
The +1 is required because OffsetSyncStore.translateDownstream translates
consumer group offsets beyond the latest sync to at most one downstream offset
past the sync.
This relationship is currently enforced only by a comment/TODO in
OffsetSyncWriter. If the translation logic in OffsetSyncStore changes without a
matching update in OffsetSyncWriter, stale offset sync detection may become
inconsistent.
We should extract the shared downstream translation boundary logic into a
common helper and use it from both OffsetSyncStore.translateDownstream and
OffsetSyncWriter.PartitionState.update.
> Share common implementation for downstream offset translation boundary in
> MirrorMaker offset syncs
> --------------------------------------------------------------------------------------------------
>
> Key: KAFKA-20643
> URL: https://issues.apache.org/jira/browse/KAFKA-20643
> Project: Kafka
> Issue Type: Improvement
> Components: connect, mirrormaker
> Reporter: Jheng-Sing Chen
> Assignee: Jheng-Sing Chen
> Priority: Minor
>
> OffsetSyncWriter.PartitionState currently determines whether the translated
> downstream offset is too stale using:
> downstreamOffset - (lastSyncDownstreamOffset + 1) >= maxOffsetLag
> The +1 is required because OffsetSyncStore.translateDownstream translates
> consumer group offsets beyond the latest sync to at most one downstream
> offset past the sync.
> This relationship is currently enforced only by a comment/TODO in
> OffsetSyncWriter. If the translation logic in OffsetSyncStore changes without
> a matching update in OffsetSyncWriter, stale offset sync detection may become
> inconsistent.
> We should extract the shared downstream translation boundary logic into a
> common helper and use it from both OffsetSyncStore.translateDownstream and
> OffsetSyncWriter.PartitionState.update.
> TODO link:
> (https://github.com/apache/kafka/blob/trunk/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/OffsetSyncWriter.java#L172-L174)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
