[
https://issues.apache.org/jira/browse/KAFKA-21450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Jacot resolved KAFKA-21450.
---------------------------------
Fix Version/s: 4.1.3
4.0.3
4.5.0
4.4.1
4.3.3
4.2.3
Resolution: Fixed
> Iterating a TimelineHashMap at a snapshot returns duplicate or missing
> entries after a revert
> ----------------------------------------------------------------------------------------------
>
> Key: KAFKA-21450
> URL: https://issues.apache.org/jira/browse/KAFKA-21450
> Project: Kafka
> Issue Type: Bug
> Components: group-coordinator, kraft
> Affects Versions: 2.8.0
> Reporter: David Jacot
> Assignee: David Jacot
> Priority: Major
> Fix For: 4.1.3, 4.0.3, 4.5.0, 4.4.1, 4.3.3, 4.2.3
>
>
> After SnapshotRegistry#revertToSnapshot(S), iterating a TimelineHashMap or a
> TimelineHashSet at S, or at an older snapshot (entrySet(epoch),
> values(epoch), keySet(epoch), iterator(epoch)), can return an entry twice or
> skip one, until S is deleted. Lookups by key (get(key, epoch)), size(epoch)
> and reads at the latest epoch are not affected.
> Minimal reproduction:
> {code:java}
> SnapshotRegistry registry = new SnapshotRegistry(new LogContext());
> TimelineHashMap<String, Integer> map = new TimelineHashMap<>(registry, 1);
> map.put("a", 1);
> registry.getOrCreateSnapshot(0);
> map.put("a", 2);
> registry.revertToSnapshot(0);
> map.entrySet(0); // returns [a=1, a=1]
> {code}
> Cause: revertToSnapshot(S) copies the elements of S's tier back into the
> current table in SnapshottableHashTable#executeRevert but leaves them in the
> tier. The historical iterator at an epoch <= S reads both the current table
> and the tiers, so every restored element is returned twice. If a restored
> element is then removed or replaced again while S is still the latest
> snapshot, it goes back into the same tier. The tier's table can then grow
> past the current table, and the iterator skips slots, so entries are missed
> as well.
> Effect: the group coordinator reverts to the last written offset when a write
> fails, and serves reads at the last committed offset. Until the next write is
> committed, ConsumerGroupDescribe, ShareGroupDescribe, StreamsGroupDescribe,
> OffsetFetch for all the partitions of a group, and ListGroups can return a
> member, a partition or a group twice, or miss one. In the KRaft controller,
> the reads at the last stable offset collect most results into maps.
> ListPartitionReassignments without topics could list a topic twice if a
> controller that has just resigned answers before its next commit.
> Fix: once the elements of S's tier are back in the current table, drop them
> from the tier.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)