[
https://issues.apache.org/jira/browse/YARN-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589680#comment-16589680
]
Weiwei Yang commented on YARN-8692:
-----------------------------------
+1 for the idea, it will be very helpful for testing load distribution. I am
curious how node memory/cpu is calculated here? Is it based on the allocated
memory/cpu?
> Support node utilization metrics for SLS
> ----------------------------------------
>
> Key: YARN-8692
> URL: https://issues.apache.org/jira/browse/YARN-8692
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler-load-simulator
> Affects Versions: 3.2.0
> Reporter: Tao Yang
> Assignee: Tao Yang
> Priority: Major
> Attachments: image-2018-08-21-18-04-22-749.png
>
>
> The distribution of node utilization is an important healthy factor for the
> YARN cluster, related metrics in SLS can be used to evaluate the scheduling
> effects and optimize related configurations.
> To implement this improvement, we need to do things as below:
> (1) Add input configurations (contain avg and stddev for cpu/memory
> utilization ratio) and generate utilization samples for tasks, not include AM
> container cause I think it's negligible.
> (2) Simulate containers and node utilization within node status.
> (3) calculate and generate the distribution metrics and use standard
> deviation metric (stddev for short) to evaluate the effects(smaller is
> better).
> (4) show these metrics on SLS simulator page like this:
> !image-2018-08-21-18-04-22-749.png!
> For Node memory/CPU utilization distribution graphs, Y-axis is nodes number,
> and P0 represents 0%~9% utilization ratio(containers-utilization /
> node-total-resource), P1 represents 10%~19% utilization ratio, P2 represents
> 20%~29% utilization ratio, ..., at last P9 represents 90%~100% utilization
> ratio.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
