[
https://issues.apache.org/jira/browse/YARN-4678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209365#comment-15209365
]
Wangda Tan commented on YARN-4678:
----------------------------------
Actually I think we may need to consider this task to be 3 separate tasks:
1) Understand why reserved resource + allocated resource could excess queue's
max capacity, maybe we can add a test to make sure it won't happen
2) If we simply deduct reserved resources from used and show on the UI, user
could find cluster utilization is < 100 in most of the time, and it gonna be
hard to explain the reason of why it cannot reach 100%. The ideal solution is
that we can show reserved and allocated resources on the same bar with
different color.
3) Record reserved resources in ResourceUsage and QueueCapacities separately.
Thoughts?
> Cluster used capacity is > 100 when container reserved
> -------------------------------------------------------
>
> Key: YARN-4678
> URL: https://issues.apache.org/jira/browse/YARN-4678
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Brahma Reddy Battula
> Assignee: Sunil G
> Attachments: 0001-YARN-4678.patch, 0002-YARN-4678.patch,
> 0003-YARN-4678.patch
>
>
> *Scenario:*
> * Start cluster with Three NM's each having 8GB (cluster memory:24GB).
> * Configure queues with elasticity and userlimitfactor=10.
> * disable pre-emption.
> * run two job with different priority in different queue at the same time
> ** yarn jar hadoop-mapreduce-examples-2.7.2.jar pi -Dyarn.app.priority=LOW
> -Dmapreduce.job.queuename=QueueA -Dmapreduce.map.memory.mb=4096
> -Dyarn.app.mapreduce.am.resource.mb=1536
> -Dmapreduce.job.reduce.slowstart.completedmaps=1.0 10 1000000000000
> ** yarn jar hadoop-mapreduce-examples-2.7.2.jar pi -Dyarn.app.priority=HIGH
> -Dmapreduce.job.queuename=QueueB -Dmapreduce.map.memory.mb=4096
> -Dyarn.app.mapreduce.am.resource.mb=1536 3 1000000000000
> * observe the cluster capacity which was used in RM web UI
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
