Greg Harris created KAFKA-16364: ----------------------------------- Summary: MM2 High-Resolution Offset Translation Key: KAFKA-16364 URL: https://issues.apache.org/jira/browse/KAFKA-16364 Project: Kafka Issue Type: New Feature Components: mirrormaker Reporter: Greg Harris
The current OffsetSyncStore implementation [https://github.com/apache/kafka/blob/8b72a2c72f09838fdd2e7416c98d30fe876b4078/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/OffsetSyncStore.java#L57] stores a sparse index of offset syncs. This attempts to strike a balanced default behavior between offset translation availability, memory usage, and throughput on the offset syncs topic. However, this balanced default behavior is not good enough in all circumstances. When precise offset translation is needed away from the end of the topic, such as for consumer groups with persistent lag, offset translation can be more precise. Users should have a way to configure high-precision offset translation, either through additional memory usage or re-reading the offset syncs topic. -- This message was sent by Atlassian Jira (v8.20.10#820010)