Kamal Chandraprakash created KAFKA-15168:
--------------------------------------------
Summary: Handle overlapping remote log segments in
RemoteLogMetadata cache
Key: KAFKA-15168
URL: https://issues.apache.org/jira/browse/KAFKA-15168
Project: Kafka
Issue Type: Sub-task
Reporter: Kamal Chandraprakash
Assignee: Kamal Chandraprakash
For a partition p0 and a given leader epoch, the remote log manager can upload
duplicate segments due to leader change. RemoteLogMetadata cache should handle
the duplicate segments which may affect the follower and consumer.
(eg)
L0 uploaded the segment PZyYVdsJQWeBAdBDPqkcVA at t0 for offset range 10 - 90
L1 uploads the segment L5Ufv71IToiZYKgsluzcyA at t1 for offset range 5 - 100
In the RemoteLogLeaderEpochState class, the {{offsetToId}} is a navigable map.
It sorts the entries by start-offset which keeps the state as:
{code:java}
(5 - 100) -> L5Ufv71IToiZYKgsluzcyA (T1)
(10 - 90) -> PZyYVdsJQWeBAdBDPqkcVA (T0){code}
For a fetch request with fetch-offset as 92, the RemoteLogLeaderEpochState will
return the segment PZyYVdsJQWeBAdBDPqkcVA instead of L5Ufv71IToiZYKgsluzcyA,
which doesn't have the respective offset and throws error back to the caller.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
