[ 
https://issues.apache.org/jira/browse/KAFKA-21450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Jacot resolved KAFKA-21450.
---------------------------------
    Fix Version/s: 4.1.3
                   4.0.3
                   4.5.0
                   4.4.1
                   4.3.3
                   4.2.3
       Resolution: Fixed

>  Iterating a TimelineHashMap at a snapshot returns duplicate or missing 
> entries after a revert
> ----------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-21450
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21450
>             Project: Kafka
>          Issue Type: Bug
>          Components: group-coordinator, kraft
>    Affects Versions: 2.8.0
>            Reporter: David Jacot
>            Assignee: David Jacot
>            Priority: Major
>             Fix For: 4.1.3, 4.0.3, 4.5.0, 4.4.1, 4.3.3, 4.2.3
>
>
> After SnapshotRegistry#revertToSnapshot(S), iterating a TimelineHashMap or a 
> TimelineHashSet at S, or at an older snapshot (entrySet(epoch), 
> values(epoch), keySet(epoch), iterator(epoch)), can return an entry twice or 
> skip one, until S is deleted. Lookups by key (get(key, epoch)), size(epoch) 
> and reads at the latest epoch are not affected.
> Minimal reproduction:
> {code:java}
> SnapshotRegistry registry = new SnapshotRegistry(new LogContext());
> TimelineHashMap<String, Integer> map = new TimelineHashMap<>(registry, 1);
> map.put("a", 1);
> registry.getOrCreateSnapshot(0);
> map.put("a", 2);
> registry.revertToSnapshot(0);
> map.entrySet(0); // returns [a=1, a=1]
> {code}
> Cause: revertToSnapshot(S) copies the elements of S's tier back into the 
> current table in SnapshottableHashTable#executeRevert but leaves them in the 
> tier. The historical iterator at an epoch <= S reads both the current table 
> and the tiers, so every restored element is returned twice. If a restored 
> element is then removed or replaced again while S is still the latest 
> snapshot, it goes back into the same tier. The tier's table can then grow 
> past the current table, and the iterator skips slots, so entries are missed 
> as well.
> Effect: the group coordinator reverts to the last written offset when a write 
> fails, and serves reads at the last committed offset. Until the next write is 
> committed, ConsumerGroupDescribe, ShareGroupDescribe, StreamsGroupDescribe, 
> OffsetFetch for all the partitions of a group, and ListGroups can return a 
> member, a partition or a group twice, or miss one. In the KRaft controller, 
> the reads at the last stable offset collect most results into maps. 
> ListPartitionReassignments without topics could list a topic twice if a 
> controller that has just resigned answers before its next commit.
> Fix: once the elements of S's tier are back in the current table, drop them 
> from the tier.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to