David Jacot created KAFKA-21450:
-----------------------------------

             Summary:  Iterating a TimelineHashMap at a snapshot returns 
duplicate or missing entries after a revert
                 Key: KAFKA-21450
                 URL: https://issues.apache.org/jira/browse/KAFKA-21450
             Project: Kafka
          Issue Type: Bug
          Components: group-coordinator, kraft
    Affects Versions: 2.8.0
            Reporter: David Jacot
            Assignee: David Jacot


After SnapshotRegistry#revertToSnapshot(S), iterating a TimelineHashMap or a 
TimelineHashSet at S, or at an older snapshot (entrySet(epoch), values(epoch), 
keySet(epoch), iterator(epoch)), can return an entry twice or skip one, until S 
is deleted. Lookups by key (get(key, epoch)), size(epoch) and reads at the 
latest epoch are not affected.

Minimal reproduction:

{code:java}
SnapshotRegistry registry = new SnapshotRegistry(new LogContext());
TimelineHashMap<String, Integer> map = new TimelineHashMap<>(registry, 1);
map.put("a", 1);
registry.getOrCreateSnapshot(0);
map.put("a", 2);
registry.revertToSnapshot(0);
map.entrySet(0); // returns [a=1, a=1]
{code}

Cause: revertToSnapshot(S) copies the elements of S's tier back into the 
current table in SnapshottableHashTable#executeRevert but leaves them in the 
tier. The historical iterator at an epoch <= S reads both the current table and 
the tiers, so every restored element is returned twice. If a restored element 
is then removed or replaced again while S is still the latest snapshot, it goes 
back into the same tier. The tier's table can then grow past the current table, 
and the iterator skips slots, so entries are missed as well.

Effect: the group coordinator reverts to the last written offset when a write 
fails, and serves reads at the last committed offset. Until the next write is 
committed, ConsumerGroupDescribe, ShareGroupDescribe, StreamsGroupDescribe, 
OffsetFetch for all the partitions of a group, and ListGroups can return a 
member, a partition or a group twice, or miss one. In the KRaft controller, the 
reads at the last stable offset collect most results into maps. 
ListPartitionReassignments without topics could list a topic twice if a 
controller that has just resigned answers before its next commit.

Fix: once the elements of S's tier are back in the current table, drop them 
from the tier.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to