[ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619298#comment-13619298 ]
Alejandro Abdelnur commented on YARN-291: ----------------------------------------- bq. How so? In fact ... Changes in NM capacity triggered from outside of the regular scheduling would unbalance existing distribution of allocations potentially triggering preemption. You'd need to handle this specially in the RM/scheduler to handle such scenarios. bq. First this requires modifying existing applications to support new request API... Not really, the API would be augmented in a compatible way to support what I'm describing, existing applications should not be affected as they don't use this new functionality. bq. Second, RM scheduling is fundamentally high latency .... It depends how you design you AM that handles unmanaged containers. You could request several small resources on peak and then release them as you don't need them. bq. Note, the existing simple patch doesn't change existing API/protocols. It is adding a new one, that is a change. > Dynamic resource configuration > ------------------------------ > > Key: YARN-291 > URL: https://issues.apache.org/jira/browse/YARN-291 > Project: Hadoop YARN > Issue Type: New Feature > Components: nodemanager, scheduler > Reporter: Junping Du > Assignee: Junping Du > Labels: features > Attachments: Elastic Resources for YARN-v0.2.pdf, > YARN-291-AddClientRMProtocolToSetNodeResource-03.patch, > YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch, > YARN-291-JMXInterfaceOnNM-02.patch, > YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch, > YARN-291-YARNClientCommandline-04.patch > > > The current Hadoop YARN resource management logic assumes per node resource > is static during the lifetime of the NM process. Allowing run-time > configuration on per node resource will give us finer granularity of resource > elasticity. This allows Hadoop workloads to coexist with other workloads on > the same hardware efficiently, whether or not the environment is virtualized. > About more background and design details, please refer: HADOOP-9165. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira