[
https://issues.apache.org/jira/browse/SAMZA-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shanthoosh Venkataraman updated SAMZA-2118:
-------------------------------------------
Description:
A samza container is comprised of multiple tasks and event-loop. Event loop is
the main orchestration layer between the different stages of a task(process,
commit, window) and delivers messages to task by polling them from different
consumers.
Currently in samza, the event loop dispatches messages to the tasks as per
user-defined configuration `task.max.concurrency`. After dispatching messages
with count equal to `task.max.concurrency` to the tasks, the event loop is
blocked until the processing of at-least a single dispatched message
finishes(aka the task implementation invokes `taskCallBack.complete()` for
at-least a dispatched message).
When the event loop is blocked by this, the user thread would sometimes try to
shutdown the event loop. This shutdown intent marked by the user-thread will
take effect only when the processing of a single message completes. In
some-cases, when the task implementation does retry on failure for a message,
this could prolong the shutdown of the samza event loop.
This introduces tight coupling between the task processing time and the event
loop shutdown time. This behavior was observed predominantly in standalone
application where the samza container is shutdown as a part of the re-balancing
phase.
To alleviate this problem, as a part of this ticket, we propose to wake-up the
samza event-loop thread when the user-thread marks the intent to shutdown the
event-loop.
was:
A samza container is comprised of multiple tasks and event-loop. Event loop is
the main orchestration layer between the different stages of a task(process,
commit, window) and delivers messages to task by polling them from different
consumers.
Currently in samza, the event loop dispatches messages to the tasks as per
user-defined configuration `task.max.concurrency`. After dispatching messages
with count equal to `task.max.concurrency` to the tasks, the event loop is
blocked until the processing of at-least a single dispatched message
finishes(aka the task implementation invokes `taskCallBack.complete()` for
at-least a dispatched message).
When the event loop is blocked by this, the user thread would sometimes try to
shutdown the event loop. This shutdown intent marked by the user-thread will
take effect only when the processing of a single message completes. In
some-cases, when the task implementation does retry on failure for a message,
this could prolong the shutdown of the samza event loop.
This introduces tight coupling between the task processing time and the event
loop shutdown time. This behavior was observed predominantly in standalone
application where the samza container is shutdown as a part of the re-balancing
phase.
To alleviate this problem, as a part of this ticket, we propose to wake-up the
samza event-loop thread when the user-thread marks the intent to shutdown the
event-loop.
> Improve the shutdown sequence of AsyncRunLoop.
> ----------------------------------------------
>
> Key: SAMZA-2118
> URL: https://issues.apache.org/jira/browse/SAMZA-2118
> Project: Samza
> Issue Type: Improvement
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Priority: Major
>
> A samza container is comprised of multiple tasks and event-loop. Event loop
> is the main orchestration layer between the different stages of a
> task(process, commit, window) and delivers messages to task by polling them
> from different consumers.
> Currently in samza, the event loop dispatches messages to the tasks as per
> user-defined configuration `task.max.concurrency`. After dispatching messages
> with count equal to `task.max.concurrency` to the tasks, the event loop is
> blocked until the processing of at-least a single dispatched message
> finishes(aka the task implementation invokes `taskCallBack.complete()` for
> at-least a dispatched message).
> When the event loop is blocked by this, the user thread would sometimes try
> to shutdown the event loop. This shutdown intent marked by the user-thread
> will take effect only when the processing of a single message completes. In
> some-cases, when the task implementation does retry on failure for a message,
> this could prolong the shutdown of the samza event loop.
> This introduces tight coupling between the task processing time and the event
> loop shutdown time. This behavior was observed predominantly in standalone
> application where the samza container is shutdown as a part of the
> re-balancing phase.
> To alleviate this problem, as a part of this ticket, we propose to wake-up
> the samza event-loop thread when the user-thread marks the intent to shutdown
> the event-loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
