[
https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588254#comment-14588254
]
MENG DING commented on YARN-1197:
---------------------------------
Thanks guys for all the comments! I think we all agreed that container decrease
request should go through RM, and decrease action will be triggered with RM->NM
heartbeat.
For increase request and action, theoretically option (a) will have better
performance. but we are incurring extra complexity for both YARN and
application writers. I was wondering if we can consider option (c) which sorts
of meet (a) and (b) in the middle:
1) AM sends increase request to RM
2) RM allocates the resource and sends the increase token to NM.
3) RM sends response to AM right away, instead of waiting for NM to confirm
that the increase action has been completed.
4) Upon receiving the response (which indicates that the increase has been
triggered), AM should first poll the container status to make sure that the
increase is done before taking action to allocate new tasks.
Option (c) will save one NM-RM heartbeat cycle, and since both option (a) and
(c) need to poll container status, their performance will be very close.
We can have option (b) enabled by default, and use a configuration parameter to
turn on option (c) for framework like Spark.
Do you think if this is worth considering?
> Support changing resources of an allocated container
> ----------------------------------------------------
>
> Key: YARN-1197
> URL: https://issues.apache.org/jira/browse/YARN-1197
> Project: Hadoop YARN
> Issue Type: Task
> Components: api, nodemanager, resourcemanager
> Affects Versions: 2.1.0-beta
> Reporter: Wangda Tan
> Attachments: YARN-1197 old-design-docs-patches-for-reference.zip,
> YARN-1197_Design.pdf
>
>
> The current YARN resource management logic assumes resource allocated to a
> container is fixed during the lifetime of it. When users want to change a
> resource
> of an allocated container the only way is releasing it and allocating a new
> container with expected size.
> Allowing run-time changing resources of an allocated container will give us
> better control of resource usage in application side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
