[
https://issues.apache.org/jira/browse/TEZ-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075775#comment-14075775
]
Chen He commented on TEZ-707:
-----------------------------
I am still confusing about why we need future type of tasks running in local
mode after I investigate the Callback and ListeningServiceFuture mechanism.
Callback mechanism allows us to get return of task, ListeningServiceFuture
provides manageable async way to run tasks. We can get return of each running
task by using above mechanisms. However, In current version of
LocalContainerLauncher, threadspool is used to control the parallelism of
running tasks. We do blocking if running tasks reach the poolsize and sleep
TezSubTaskRunner. Actually, tasks come and go, we care the parallelism more
than the return of each task. Since a task failed, it will record failure in
log (since we are running in a single process).
Please correct me if there is misunderstanding. What you feel, [~sseth]?
> Create LocalContainerLauncher
> -----------------------------
>
> Key: TEZ-707
> URL: https://issues.apache.org/jira/browse/TEZ-707
> Project: Apache Tez
> Issue Type: Sub-task
> Affects Versions: 0.3.0
> Reporter: Chen He
> Assignee: Chen He
> Priority: Blocker
> Attachments: TEZ-707-2014-7015.patch, TEZ-707-2014-7015.patch.review,
> TEZ-707-v3.patch, TEZ-707-v4.patch, TEZ-707.patch, TEZ-707.patch,
> Tez-707.patch.v2, tez-707.patch
>
>
> Create LocalContainerLauncher and make it work for a single stage DAG. The
> TaskSchedulerEventHandler still asks RM for new container but
> LocalContainerLauncher will run TezTask in form of thread instead of using
> this container from yarn.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
