[
https://issues.apache.org/jira/browse/YARN-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284446#comment-14284446
]
Jian Fang commented on YARN-1039:
---------------------------------
The duration concept comes with a good intention, but what I really am afraid
of is that it could introduce a huge complex to YARN if it is not designed
properly.
First, there are so many moving parts under the hook for the estimation, for
example, the time of a 30 node cluster may be significantly different from the
one of a 300 node cluster. Getting into the measurement and estimation business
is very much like walking into benchmark comparison business, which is very
hard in reality.
Secondly, the duration probably relies on hadoop customers to provide a proper
value for it if YARN is not smart enough to derive the value by itself, which
could be impractical for many customers. Remember that many hadoop users are
not even developers. Many of them rely on high level components such as pig and
hive to run hadoop jobs. They probably don't know or care about the estimation.
As a result, at least, the duration should only be an enhancement if the value
is provided. YARN should still work properly without such a value.
> Add parameter for YARN resource requests to indicate "long lived"
> -----------------------------------------------------------------
>
> Key: YARN-1039
> URL: https://issues.apache.org/jira/browse/YARN-1039
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 3.0.0, 2.1.1-beta
> Reporter: Steve Loughran
> Assignee: Craig Welch
> Attachments: YARN-1039.1.patch, YARN-1039.2.patch, YARN-1039.3.patch
>
>
> A container request could support a new parameter "long-lived". This could be
> used by a scheduler that would know not to host the service on a transient
> (cloud: spot priced) node.
> Schedulers could also decide whether or not to allocate multiple long-lived
> containers on the same node
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
