Tao Yang created YARN-8692:
------------------------------
Summary: Support node utilization metrics for SLS
Key: YARN-8692
URL: https://issues.apache.org/jira/browse/YARN-8692
Project: Hadoop YARN
Issue Type: Improvement
Components: scheduler-load-simulator
Affects Versions: 3.2.0
Reporter: Tao Yang
Assignee: Tao Yang
Attachments: image-2018-08-21-17-50-04-011.png
The distribution of node utilization is an important healthy factor for the
YARN cluster, related metrics in SLS can be used to evaluate the scheduling
effects and optimize related configurations.
To implement this improvement, we need to do things as below:
(1) Add input configurations (contain avg and stddev for cpu/memory utilization
ratio) and generate utilization samples for tasks, not include AM container
cause I think it's negligible. (2) Simulate containers and node utilization
within node status.
(3) calculate and generate the distribution metrics and use standard deviation
metric (stddev for short) to evaluate the effects(smaller is better).
(4) show these metrics on SLS simulator page like this:
!image-2018-08-21-17-50-04-011.png!
For Node memory/CPU utilization distribution graphs, Y-axis is nodes number,
and P0 represents 0%~9% utilization ratio(containers-utilization /
node-total-resource), P1 represents 10%~19% utilization ratio, P2 represents
20%~29% utilization ratio, ..., at last P9 represents 90%~100% utilization
ratio.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
