[
https://issues.apache.org/jira/browse/SAMZA-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Kumarasubramanian reassigned SAMZA-2531:
------------------------------------------------
Assignee: Daniel Chen
Can you evaluate this ticket as part of your async state checkpointing work?
> Async Commit is Blocking for Window, Timer & End of Stream
> ----------------------------------------------------------
>
> Key: SAMZA-2531
> URL: https://issues.apache.org/jira/browse/SAMZA-2531
> Project: Samza
> Issue Type: Task
> Reporter: Bharath Kumarasubramanian
> Assignee: Daniel Chen
> Priority: Major
>
> We introduced non-blocking commit feature as part of SAMZA-1140. Currently
> the commit is only non-blocking and allows process to be run concurrently.
> As part of this task, we will need to evaluate what it takes to make it run
> concurrently along with window, timer & end of stream and if it is necessary
> in the first place. If not, rename the feature to explicitly callout process
> exclusive commit instead of async commit.
> Some context:
> * Transactional state currently doesn't work well w/ async commit turned on.
> i.e. Even though async commit, uses low watermark approach to find the offset
> to safely commit, it still doesn't guarantee the state snapshots happen
> during the commit correspond to the offset that is committed to checkpoint
> topic.
> * Window, timer could potentially mutate the state in the same manner as
> process; So it might be okay to phase out (if we decide to support) the
> changes to introduce concurrency within window & timer first and follow it up
> with synchronizing the state w/ the offset that is committed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
