Shanthoosh Venkataraman created SAMZA-1083:
----------------------------------------------
Summary: Do not load task stores which are older than delete
tombstones during container startup
Key: SAMZA-1083
URL: https://issues.apache.org/jira/browse/SAMZA-1083
Project: Samza
Issue Type: New Feature
Reporter: Shanthoosh Venkataraman
Assignee: Shanthoosh Venkataraman
Every local task store is backed up by a kafka changelog topic. Due to log
compaction, delete tombstones of the changelog topic have a ttl of
delete.retention.ms. Replaying the events from the changelog that has missing
delete tombstones, would result in creation of an inconsistent local store(due
to the missing of some delete events). This patch deletes the local stores in
which difference between current time and last modified time of the offset file
is greater than delete.retention.ms during the container startup.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
