[
https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587146#comment-14587146
]
Wangda Tan commented on YARN-1197:
----------------------------------
[~sandyr],
Thanks for replying,
bq. Why does the AM need to poll the NM about increase status before taking
action? Does the NM need to do anything other than update its tracking of the
resources allotted to the container?
Yes, NM only needs to update tracking of the resource and cgroups. We cannot
assume this can happen immediately, so we cannot put "container increased" to
the same RPC. This is same as startContainer, even if launching a container is
fast in most cases, AM needs to poll NM after invoked startContainer.
bq. Would option (b) ever be able to achieve this kind of latency?
If you consider all now/future optimizations, such as continous-scheduling /
scheduler make decision at same AM-RM heart-beat. (b) needs one more NM-RM
heart-beat interval. I agree with you, it could be hundreds of milli-seconds
(a) vs. multi-seconds (b). when the cluster is idle.
But I'm wondering do we really need add these complexity to AM before we have
mature optimizatons listed above? And also, if the cluster is busier, we cannot
expect the delay as well. I tend to do (b) now since it's simpler to app
developer to use this feature, I'm open to add AM->NM channel if we have YARN
scheduler supports fast scheduling better.
> Support changing resources of an allocated container
> ----------------------------------------------------
>
> Key: YARN-1197
> URL: https://issues.apache.org/jira/browse/YARN-1197
> Project: Hadoop YARN
> Issue Type: Task
> Components: api, nodemanager, resourcemanager
> Affects Versions: 2.1.0-beta
> Reporter: Wangda Tan
> Attachments: YARN-1197 old-design-docs-patches-for-reference.zip,
> YARN-1197_Design.pdf
>
>
> The current YARN resource management logic assumes resource allocated to a
> container is fixed during the lifetime of it. When users want to change a
> resource
> of an allocated container the only way is releasing it and allocating a new
> container with expected size.
> Allowing run-time changing resources of an allocated container will give us
> better control of resource usage in application side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
