[
https://issues.apache.org/jira/browse/SAMZA-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15858265#comment-15858265
]
Jake Maes commented on SAMZA-1083:
----------------------------------
[~spvenkat], no need to open a PR. I remember the old commit routine.
I had a couple minor feedback items on the review. Address those and make sure
this patch is rebased with the latest. Then I'll commit as soon as I can.
> Do not load task stores which are older than delete tombstones during
> container startup
> ---------------------------------------------------------------------------------------
>
> Key: SAMZA-1083
> URL: https://issues.apache.org/jira/browse/SAMZA-1083
> Project: Samza
> Issue Type: New Feature
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Attachments:
> 0001-SAMZA-1083-Do-not-load-task-stores-which-are-older-t.patch
>
>
> Every local task store is backed up by a kafka changelog topic. Due to log
> compaction, delete tombstones of the changelog topic have a ttl of
> delete.retention.ms. Replaying the events from the changelog that has missing
> delete tombstones, would result in creation of an inconsistent local
> store(due to the missing of some delete events). This patch deletes the local
> stores in which difference between current time and last modified time of the
> offset file is greater than delete.retention.ms during the container startup.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
