Jim Brennan created YARN-10475:
----------------------------------
Summary: Scale RM-NM heartbeat interval based on node utilization
Key: YARN-10475
URL: https://issues.apache.org/jira/browse/YARN-10475
Project: Hadoop YARN
Issue Type: Improvement
Components: yarn
Affects Versions: 2.10.1, 3.4.1
Reporter: Jim Brennan
Assignee: Jim Brennan
Add the ability to scale the RM-NM heartbeat interval based on node cpu
utilization compared to overall cluster cpu utilization. If a node is
over-utilized compared to the rest of the cluster, it's heartbeat interval
slows down. If it is under-utilized compared to the rest of the cluster, it's
heartbeat interval speeds up.
This is a feature we have been running with internally in production for
several years. It was developed by [~nroberts], based on the observation that
larger faster nodes on our cluster were under-utilized compared to smaller
slower nodes.
This feature is dependent on [YARN-10450], which added cluster-wide utilization
metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
