Junping Du commented on YARN-8118:

Thanks for contributing your idea and code, [~Karthik Palaniappan]! 
As Jason mentioned above, our main goal here is to remove decommissioning nodes 
from service ASAP with least price of interrupting existing progress that 
applications already made (existing containers running). in my opinion, in most 
cases, there is no significant difference between containers to be scheduled by 
existing applications or new applications. If there are any, the right solution 
should be via priority/preemption mechanism between applications. In another 
word, we don't have assumption on priority differences between existing and new 
applications in our typical decommissioning cases.
However, in a pure cloud environment (like EMR, etc.), the scenario could be 
different - what I can imagine (please correct me if I am wrong) is: user(also 
an admin in yarn prospective) drop most workloads to a dedicated yarn cluster 
and wish the cluster can shrink to some minimal size later when applications 
get finished. If this is the case that current design and code want to target, 
then we should take Jason's suggestion above to have a new configure for 
cluster or a new parameter for graceful decommission CLI. 
We need to be careful here as previous decommissioning nodes operation is 
idempotent, here we need to figure out what means if new applications get 
submitted between multiple operations and how to track them - I don't think the 
current code provide a way.

> Better utilize gracefully decommissioning node managers
> -------------------------------------------------------
>                 Key: YARN-8118
>                 URL: https://issues.apache.org/jira/browse/YARN-8118
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>    Affects Versions: 2.8.2
>         Environment: * Google Compute Engine (Dataproc)
>  * Java 8
>  * Hadoop 2.8.2 using client-mode graceful decommissioning
>            Reporter: Karthik Palaniappan
>            Priority: Major
>         Attachments: YARN-8118-branch-2.001.patch
> Proposal design doc with background + details (please comment directly on 
> doc): 
> [https://docs.google.com/document/d/1hF2Bod_m7rPgSXlunbWGn1cYi3-L61KvQhPlY9Jk9Hk/edit#heading=h.ab4ufqsj47b7]
> tl;dr Right now, DECOMMISSIONING nodes must wait for in-progress applications 
> to complete before shutting down, but they cannot run new containers from 
> those in-progress applications. This is wasteful, particularly in 
> environments where you are billed by resource usage (e.g. EC2).
> Proposal: YARN should schedule containers from in-progress applications on 
> DECOMMISSIONING nodes, but should still avoid scheduling containers from new 
> applications. That will make in-progress applications complete faster and let 
> nodes decommission faster. Overall, this should be cheaper.
> I have a working patch without unit tests that's surprisingly just a few real 
> lines of code (patch 001). If folks are happy with the proposal, I'll write 
> unit tests and also write a patch targeted at trunk.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to