Joel Koshy created KAFKA-2163:
---------------------------------
Summary: Offsets manager cache should prevent stale-offset-cleanup
while an offset load is in progress; otherwise we can lose consumer offsets
Key: KAFKA-2163
URL: https://issues.apache.org/jira/browse/KAFKA-2163
Project: Kafka
Issue Type: Bug
Reporter: Joel Koshy
Fix For: 0.8.3
When leadership of an offsets partition moves, the new leader loads offsets
from that partition into the offset manager cache.
Independently, the offset manager has a periodic cleanup task for stale offsets
that removes old offsets from the cache and appends tombstones for those. If
the partition happens to contain much older offsets (earlier in the log) and
inserts those into the cache; the cleanup task may run and see those offsets
(which it deems to be stale) and proceeds to remove from the cache and append a
tombstone to the end of the log. The tombstone will override the true latest
offset and a subsequent offset fetch request will return no offset.
We just need to prevent the cleanup task from running during an offset load.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
