[
https://issues.apache.org/jira/browse/YARN-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404441#comment-15404441
]
Junping Du commented on YARN-4676:
----------------------------------
Below transitions for RMNode doesn't sound correct:
{noformat}
+ .addTransition(NodeState.DECOMMISSIONED, NodeState.RUNNING,
+ RMNodeEventType.RECOMMISSION,
+ new RecommissionNodeTransition(NodeState.RUNNING))
+ .addTransition(NodeState.DECOMMISSIONED, NodeState.DECOMMISSIONED,
+ RMNodeEventType.CLEANUP_CONTAINER, new CleanUpContainerTransition())
+ .addTransition(NodeState.DECOMMISSIONED, NodeState.LOST,
+ RMNodeEventType.EXPIRE, new DeactivateNodeTransition(NodeState.LOST))
+ .addTransition(NodeState.DECOMMISSIONED, NodeState.DECOMMISSIONED,
+ RMNodeEventType.CLEANUP_APP, new CleanUpAppTransition())
{noformat}
1. RMNode in DECOMMISSIONED status shouldn't transit to RUNNING. The only way
to make a decommissioned node to be active again is through two steps: 1) put
this node back to RM include-node list and call refreshNode CLI, 2) restart the
NM to register to RM again. The different with DECOMMISSIONING node is:
decommissioning node is still running so step 2 is not needed. For
DECOMMISSIONED node, we never know when step 2 will happen so we shouldn't mark
node as RUNNING.
2. Transmit node from DECOMMISSIONED to LOST is not necessary. Node in
DECOMMISSIONED is already down and won't have heartbeat to RM again so we stop
heartbeat monitor against this node and no chance for EXPIRE event get sent.
3. For CleanUpContainerTransition(), CleanUpAppTransition() (and
AddContainersToBeRemovedFromNMTransition() in existing code base), I don't
think this is necessary as NM get decommissioned should already be clean up in
state - this is different with NM shutdown where we keep container running and
keeping NM state up-to-date.
> Automatic and Asynchronous Decommissioning Nodes Status Tracking
> ----------------------------------------------------------------
>
> Key: YARN-4676
> URL: https://issues.apache.org/jira/browse/YARN-4676
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.8.0
> Reporter: Daniel Zhi
> Assignee: Daniel Zhi
> Labels: features
> Attachments: GracefulDecommissionYarnNode.pdf,
> GracefulDecommissionYarnNode.pdf, YARN-4676.004.patch, YARN-4676.005.patch,
> YARN-4676.006.patch, YARN-4676.007.patch, YARN-4676.008.patch,
> YARN-4676.009.patch, YARN-4676.010.patch, YARN-4676.011.patch,
> YARN-4676.012.patch, YARN-4676.013.patch, YARN-4676.014.patch,
> YARN-4676.015.patch, YARN-4676.016.patch, YARN-4676.017.patch,
> YARN-4676.018.patch, YARN-4676.019.patch
>
>
> YARN-4676 implements an automatic, asynchronous and flexible mechanism to
> graceful decommission
> YARN nodes. After user issues the refreshNodes request, ResourceManager
> automatically evaluates
> status of all affected nodes to kicks out decommission or recommission
> actions. RM asynchronously
> tracks container and application status related to DECOMMISSIONING nodes to
> decommission the
> nodes immediately after there are ready to be decommissioned. Decommissioning
> timeout at individual
> nodes granularity is supported and could be dynamically updated. The
> mechanism naturally supports multiple
> independent graceful decommissioning “sessions” where each one involves
> different sets of nodes with
> different timeout settings. Such support is ideal and necessary for graceful
> decommission request issued
> by external cluster management software instead of human.
> DecommissioningNodeWatcher inside ResourceTrackingService tracks
> DECOMMISSIONING nodes status automatically and asynchronously after
> client/admin made the graceful decommission request. It tracks
> DECOMMISSIONING nodes status to decide when, after all running containers on
> the node have completed, will be transitioned into DECOMMISSIONED state.
> NodesListManager detect and handle include and exclude list changes to kick
> out decommission or recommission as necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
