[
https://issues.apache.org/jira/browse/TEZ-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262003#comment-14262003
]
Bikas Saha commented on TEZ-1897:
---------------------------------
bq. So at least there should be only one thread for the central dispatcher, the
multiple thread can only been used internally in registerAndCreateDispatcher
That is the way it is. Central dispatcher has 1 thread and uses the original
constructor. If numThreads is not specified then it defaults to 1. The new
constructor is package private. I can change the public to package.
bq. This patch cause
TestExceptionPropagation#testExceptionPropagationNonSession fail.
Yes. I noticed this failure too. The javadoc says that it will wait for timeout
or completion of existing tasks, whichever is smaller. However, what is
happening is that when there are 0 pending tasks then it waits for the timeout
period. I am going to match existing behavior by removing the
awaitTermination().
What is interesting is that the AM has unregistered from the RM in both cases.
That is where the diagnostic is sent to the RM. Not sure how sleeping after
that will change the result of the test querying the RM for that final
status/diagnostic.
bq. that AsyncDispatcher would sleep for 1 second when draining event,
That is existing logic that I have not changed. In real execution, we do not
set drain events to true. So that code does not get exercised.
Attaching updated patch. Please review. Thanks!
> Allow higher concurrency in AsyncDispatcher
> -------------------------------------------
>
> Key: TEZ-1897
> URL: https://issues.apache.org/jira/browse/TEZ-1897
> Project: Apache Tez
> Issue Type: Task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-1897.1.patch, TEZ-1897.2.patch
>
>
> Currently, it processes events on a single thread. For events that can be
> executed in parallel, e.g. vertex manager events, allowing higher concurrency
> may be beneficial.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
