[
https://issues.apache.org/jira/browse/TEZ-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259255#comment-14259255
]
Bikas Saha commented on TEZ-1867:
---------------------------------
Dont want to pass in a dispatcher. Tests dont need this as of now and would
like to keep this localized within the current scope. The tests are primarily
senders of events and thus oblivious to this change (as intended in the design).
Ideally we should not have cross-locks (and we mostly dont) in a messaging
based system like we have. There are many cases where large numbers of some
events (eg. task attempt events during job start for a large map) that prevent
other events from being processed in a timely manner (eg. vertex events during
the backlog of attempt events) So we should definitely try out offloading
events to separate handlers. The good thing is that if there are many issues
after that then its an easy change to revert after this patch. Just the setup
of the dispatchers can be fixed in DAGAppMaster. Given that, trying it out
seems reasonable cost wise.
Will update the patch with the other comments.
> Create new central dispatcher for Tez AM
> ----------------------------------------
>
> Key: TEZ-1867
> URL: https://issues.apache.org/jira/browse/TEZ-1867
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-1867.1.patch
>
>
> The dispatcher should be able to federate events across multiple event
> specific dispatchers for AM control plane scalability. Add dispatcher
> specific thread names for easier debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
