[
https://issues.apache.org/jira/browse/TEZ-684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14251131#comment-14251131
]
Chen He edited comment on TEZ-684 at 12/18/14 3:53 AM:
-------------------------------------------------------
Thank you for the explanation, [~bikassaha].
Since Jan 2014, I started to create and drive TEZ-684. I came up with basic
design. [~jlowe], [~jeagles], [~bikassaha], [~sseth], [~hitesh],
[~pramachandran] and [~rohini] contributed to this JIRA. I really appreciate
their ideas and works. I came up with the first workable version 14/Jul/14
based on Tez-0.4.2. Contributed 10 JIRAs during the process (some of them are
replace by other code like local fetcher)I am keep helping incoming
contributors [~zjffdu] to understand current Local mode memory issue (TEZ-1482)
and continue to work on this JIRA.
I just want TEZ to be more and more popular and accepted as well as MapReduce
even there many competitors like Impala, Spark, etc. The more people (from
different companies and fields) contribute to TEZ, the better it will be
accepted and sophisticated as a general DAG processing platform.
was (Author: airbots):
Thank you for the explanation, [~bikassaha].
Since Jan 2014, I started to create and drive TEZ-684. I came up with basic
design. [~jlowe], [~jeagles], [~bikassaha], [~sseth], [~hitesh],
[~pramachandran] and [~rohini] contributed to this JIRA. I really appreciate
their ideas and works. I came up with the first workable version 14/Jul/14
based on Tez-0.4.2. Contributed 10 JIRAs during the process (some of them are
replace by other code like local fetcher)I am keep helping incoming
contributors [~zjffdu] to understand current Local mode memory issue (TEZ-1482)
and continue to work on this JIRA.
I just want to TEZ to be more and more popular and accepted as well as
MapReduce even there many competitors like Impala, Spark, etc. The more people
(from different companies and fields) contribute to TEZ, the better it will be
accepted and sophisticated as a general DAG processing platform.
> Uber/Local modes for Tez
> ------------------------
>
> Key: TEZ-684
> URL: https://issues.apache.org/jira/browse/TEZ-684
> Project: Apache Tez
> Issue Type: New Feature
> Reporter: Chen He
> Assignee: Chen He
> Fix For: 0.5.0
>
> Attachments: TEZ-684-2014-7-21.patch, TEZ-684.patch, TEZ-684.patch,
> TEZ-684.patch, TEZ-684.patch, TEZ-684.patch, TEZ-684.patch, TEZ-684.patch,
> TEZ-684.patch, TEZ-684.patch, TEZ-684.patch, TEZ-684.patch, TEZ-684.patch,
> TezUberModeDesignDraft.png
>
>
> Similarly to MapReduce Uber-mode in Yarn, we plan to create the Uber-mode for
> Tez. It runs all tasks in local in one process.
> Our target is to start DAGAppMaster in local JVM and let it run all tasks in
> one process.
> Here is my design:
> Once user submits a DAG, Tez starts a instance of DAGAppMaster. This
> DAGAppMaster will check TezConfiguration before instantiate
> ContainerLauncher. If "is_Uber" is true, DAGAppMaster creates a
> LocalContainerLauncher. LocalTaskScheduler and LocalTaskSchedulerEventHandler
> will call LocalContainerLauncher to run all tasks one by one in a single JVM.
> Communications between ResourceManager and local classes (DAGAppMaster,
> LocalContainerLauncher, LocalTaskScheduler, and
> LocalTaskSchedulerEventHandler) are muted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
