[
https://issues.apache.org/jira/browse/SAMZA-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Kumarasubramanian updated SAMZA-2578:
---------------------------------------------
Description:
Samza 1.5 enabled transaction state feature by default for all samza jobs.
We uncovered a bug related to reverting changelog state to last checkpoint
(trimming), which resulted in container stuck in the restoration phase forever.
This happened due to the trimming phase of state restore: when uncheckpointed
messages in the changelog have their values reverted according to the job's
last checkpoint. If a job needed to trim a non-zero number of messages, these
trimmed messages would be repeatedly read and re-written by the restore process
infinitely preventing the job from completing startup.
> Excessive trimming during transactional state restore
> -----------------------------------------------------
>
> Key: SAMZA-2578
> URL: https://issues.apache.org/jira/browse/SAMZA-2578
> Project: Samza
> Issue Type: Improvement
> Reporter: Brett Konold
> Assignee: Brett Konold
> Priority: Major
> Fix For: 1.5.1
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Samza 1.5 enabled transaction state feature by default for all samza jobs.
> We uncovered a bug related to reverting changelog state to last checkpoint
> (trimming), which resulted in container stuck in the restoration phase
> forever. This happened due to the trimming phase of state restore: when
> uncheckpointed messages in the changelog have their values reverted according
> to the job's last checkpoint. If a job needed to trim a non-zero number of
> messages, these trimmed messages would be repeatedly read and re-written by
> the restore process infinitely preventing the job from completing startup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
