[
https://issues.apache.org/jira/browse/SAMZA-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinyu Liu resolved SAMZA-2118.
------------------------------
Resolution: Fixed
Fix Version/s: 1.1
> Improve the shutdown sequence of AsyncRunLoop.
> ----------------------------------------------
>
> Key: SAMZA-2118
> URL: https://issues.apache.org/jira/browse/SAMZA-2118
> Project: Samza
> Issue Type: Improvement
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Priority: Major
> Fix For: 1.1
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> A samza container is comprised of multiple tasks and event-loop. Event loop
> is the main orchestration layer between the different stages of a
> task(process, commit, window) and delivers messages to task by polling them
> from different consumers.
> Currently in samza, the event loop dispatches messages to the tasks as per
> user-defined configuration `task.max.concurrency`. After dispatching messages
> with count equal to `task.max.concurrency` to the tasks, the event loop is
> blocked until the processing of at-least a single dispatched message
> finishes(aka the task implementation invokes `taskCallBack.complete()` for
> at-least a dispatched message).
> When the event loop is blocked by this, the user thread would sometimes try
> to shutdown the event loop. This shutdown intent marked by the user-thread
> will take effect only when the processing of a single message completes. In
> some-cases, when the task implementation does retry on failure for a message,
> this could prolong the shutdown of the samza event loop.
> This introduces tight coupling between the task processing time and the event
> loop shutdown time.
> This behavior was observed predominantly in standalone applications where the
> samza container is shutdown as a part of the re-balancing phase and task
> processing time was significantly larger.
> To alleviate this problem, as a part of this ticket, we propose to wake-up
> the samza event-loop thread when the user-thread marks the intent to shutdown
> the event-loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
