[
https://issues.apache.org/jira/browse/YARN-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429629#comment-16429629
]
Junping Du commented on YARN-8118:
----------------------------------
Thanks for contributing your idea and code, [~Karthik Palaniappan]!
As Jason mentioned above, our main goal here is to remove decommissioning nodes
from service ASAP with least price of interrupting existing progress that
applications already made (existing containers running). in my opinion, in most
cases, there is no significant difference between containers to be scheduled by
existing applications or new applications. If there are any, the right solution
should be via priority/preemption mechanism between applications. In another
word, we don't have assumption on priority differences between existing and new
applications in our typical decommissioning cases.
However, in a pure cloud environment (like EMR, etc.), the scenario could be
different - what I can imagine (please correct me if I am wrong) is: user(also
an admin in yarn prospective) drop most workloads to a dedicated yarn cluster
and wish the cluster can shrink to some minimal size later when applications
get finished. If this is the case that current design and code want to target,
then we should take Jason's suggestion above to have a new configure for
cluster or a new parameter for graceful decommission CLI.
We need to be careful here as previous decommissioning nodes operation is
idempotent, here we need to figure out what means if new applications get
submitted between multiple operations and how to track them - I don't think the
current code provide a way.
> Better utilize gracefully decommissioning node managers
> -------------------------------------------------------
>
> Key: YARN-8118
> URL: https://issues.apache.org/jira/browse/YARN-8118
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Affects Versions: 2.8.2
> Environment: * Google Compute Engine (Dataproc)
> * Java 8
> * Hadoop 2.8.2 using client-mode graceful decommissioning
> Reporter: Karthik Palaniappan
> Priority: Major
> Attachments: YARN-8118-branch-2.001.patch
>
>
> Proposal design doc with background + details (please comment directly on
> doc):
> [https://docs.google.com/document/d/1hF2Bod_m7rPgSXlunbWGn1cYi3-L61KvQhPlY9Jk9Hk/edit#heading=h.ab4ufqsj47b7]
> tl;dr Right now, DECOMMISSIONING nodes must wait for in-progress applications
> to complete before shutting down, but they cannot run new containers from
> those in-progress applications. This is wasteful, particularly in
> environments where you are billed by resource usage (e.g. EC2).
> Proposal: YARN should schedule containers from in-progress applications on
> DECOMMISSIONING nodes, but should still avoid scheduling containers from new
> applications. That will make in-progress applications complete faster and let
> nodes decommission faster. Overall, this should be cheaper.
> I have a working patch without unit tests that's surprisingly just a few real
> lines of code (patch 001). If folks are happy with the proposal, I'll write
> unit tests and also write a patch targeted at trunk.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
