[
https://issues.apache.org/jira/browse/TEZ-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14386486#comment-14386486
]
Jeff Zhang commented on TEZ-2224:
---------------------------------
Upload new patch, [~hitesh] Please help review.
bq. Is there a reason why we want to prevent new events from being processed on
a shutdown?
No special reason for that, just borrow the code from AsynDispatcher. But I
think you are right, AsyncDispatcher is for general use case (maybe it is to
prevent the case of unlimited events), here for RecoveringService, we can
ensure the case of unlimited events should not happen and need to handle events
even when it is stopped.
bq. TEZ_AM_RECOVERY_HANDLE_REMAINING_EVENT_WHEN_STOPPED_DEFAULT being false by
default is probably wrong. For a real-world scenario, as many pending events
that are seen and can be processed, should be processed.
I make it true in the new patch. But previously we won't process the pending
events by default when there's no such flag. Not sure whether we have any
other consideration at that time, maybe for performance consideration, like in
non-session mode that dag is finished while recovery events handling is not
completed but is not necessary.
> EventQueue empty doesn't mean events are consumed in RecoveryService
> --------------------------------------------------------------------
>
> Key: TEZ-2224
> URL: https://issues.apache.org/jira/browse/TEZ-2224
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Attachments: TEZ-2224-1.patch, TEZ-2224-2.patch
>
>
> If the event queue is empty, the event may still been processing. Should fix
> it like AsyncDispatcher
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
