[
https://issues.apache.org/jira/browse/TEZ-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14076250#comment-14076250
]
Chen He commented on TEZ-707:
-----------------------------
Thank you for the comments, [~sseth].
I set the maximium queue size is 2 * number of allowed executions to make the
queue little bit larger than the allowed executions. It is used to deal with
following corner case:
Assume the executor's pool size is N (maximium is 2N), if an event launched
more tasks than N. the queue in the thread executor can hold them for a given
period (configurable) without losing track of those tasks that not allowed to
run. If one task finishes, executor will automatically release one waiting task
from the queue and let it run. The maximium queue size is used as buffer for
those waiting tasks. If we set maximium queue size = pool size, we may lose
some task and cause DAG fail.
I agree with you on the second suggestion.
> Create LocalContainerLauncher
> -----------------------------
>
> Key: TEZ-707
> URL: https://issues.apache.org/jira/browse/TEZ-707
> Project: Apache Tez
> Issue Type: Sub-task
> Affects Versions: 0.3.0
> Reporter: Chen He
> Assignee: Chen He
> Priority: Blocker
> Attachments: TEZ-707-2014-7015.patch, TEZ-707-2014-7015.patch.review,
> TEZ-707-v3.patch, TEZ-707-v4.patch, TEZ-707.patch, TEZ-707.patch,
> Tez-707.patch.v2, tez-707.patch
>
>
> Create LocalContainerLauncher and make it work for a single stage DAG. The
> TaskSchedulerEventHandler still asks RM for new container but
> LocalContainerLauncher will run TezTask in form of thread instead of using
> this container from yarn.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
