[
https://issues.apache.org/jira/browse/YARN-4678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136594#comment-15136594
]
Brahma Reddy Battula commented on YARN-4678:
--------------------------------------------
*As reserved memory is include in the used resources. queue capacity > 100*
..Please let me know any thoughts on this..
2016-02-22 01:15:09,359 | INFO | ResourceManager Event Processor |
completedContainer queue=root usedCapacity=0.9583333
absoluteUsedCapacity=0.9583333 *{color:red}used=<memory:23552, vCores:7>
cluster=<memory:24576, vCores:24>{color}* | ParentQueue.java:631
2016-02-22 01:15:09,359 | INFO | ResourceManager Event Processor | Re-sorting
completed queue: root.QueueA stats: QueueA: capacity=0.4, absoluteCapacity=0.4,
usedResources=<memory:17920, vCores:5>, usedCapacity=1.8229909,
absoluteUsedCapacity=0.7291667, numApps=1, numContainers=5 |
ParentQueue.java:648
2016-02-22 01:15:09,359 | INFO | ResourceManager Event Processor | Application
attempt appattempt_1456061422894_0003_000001 released container
container_e05_1456061422894_0003_01_004504 on *{color:blue}node: host:
vm-3:26009{color}* #containers=1 available=<memory:6656, vCores:7>
used=<memory:1536, vCores:1> with event: FINISHED | CapacityScheduler.java:1452
2016-02-22 01:15:09,359 | INFO | ResourceManager Event Processor | Trying to
fulfill reservation for application application_1456061422894_0003 on node:
vm-3:26009 | CapacityScheduler.java:1110
2016-02-22 01:15:09,359 | INFO | ResourceManager Event Processor | Application
application_1456061422894_0003 *{color:blue}unreserved on node host:
vm-3:26009{color}* #containers=1 available=<memory:6656, vCores:7>
used=<memory:1536, vCores:1>, currently has 0 at priority 20;
currentReservation <memory:0, vCores:0> on node-label= |
FiCaSchedulerApp.java:229
2016-02-22 01:15:09,359 | INFO | ResourceManager Event Processor |
container_e05_1456061422894_0003_01_004505 Container Transitioned from NEW to
ALLOCATED | RMContainerImpl.java:419
2016-02-22 01:15:09,359 | INFO | ResourceManager Event Processor | Assigned
container container_e05_1456061422894_0003_01_004505 of capacity <memory:4096,
vCores:1> on *{color:blue}host vm-3:26009{color}* , which has 2 containers,
<memory:5632, vCores:2> used and <memory:2560, vCores:6> available after
allocation | SchedulerNode.java:154
2016-02-22 01:15:09,359 | INFO | ResourceManager Event Processor |
assignedContainer application attempt=appattempt_1456061422894_0003_000001
container=Container: [ContainerId: container_e05_1456061422894_0003_01_004505,
NodeId: vm-3:26009, NodeHttpAddress: vm-3:26010, Resource: <memory:4096,
vCores:1>, Priority: 20, Token: null, ] queue=QueueA: capacity=0.4,
absoluteCapacity=0.4, usedResources=<memory:17920, vCores:5>,
usedCapacity=1.8229909, absoluteUsedCapacity=0.7291667, numApps=1,
numContainers=5 clusterResource=<memory:24576, vCores:24> | LeafQueue.java:1731
2016-02-22 01:15:09,360 | INFO | ResourceManager Event Processor |
container_e05_1456061422894_0003_01_004508 Container Transitioned from
*{color:blue}NEW to RESERVED{color}* | RMContainerImpl.java:419
2016-02-22 01:15:09,360 | INFO | ResourceManager Event Processor | *Reserved
container application=application_1456061422894_0003 resource=<memory:4096,
vCores:1>* queue=QueueA: capacity=0.4, absoluteCapacity=0.4,
usedResources=<memory:17920, vCores:5>, usedCapacity=1.8229909,
absoluteUsedCapacity=0.7291667, numApps=1, numContainers=5
usedCapacity=1.8229909 absoluteUsedCapacity=0.7291667 used=<memory:17920,
vCores:5> cluster=<memory:24576, vCores:24> | LeafQueue.java:1764
2016-02-22 01:15:09,360 | INFO | ResourceManager Event Processor | Re-sorting
assigned queue: root.QueueA stats: QueueA: capacity=0.4, absoluteCapacity=0.4,
usedResources=<memory:22016, vCores:6>, usedCapacity=2.2396743,
absoluteUsedCapacity=0.8958333, numApps=1, numContainers=6 |
ParentQueue.java:585
2016-02-22 01:15:09,360 | INFO | ResourceManager Event Processor |
*assignedContainer queue=root {color:red}usedCapacity=1.125
absoluteUsedCapacity=1.125
used=<memory:27648, vCores:8>
cluster=<memory:24576, vCores:24>{color}* | ParentQueue.java:465
> Cluster used capacity is > 100 when container reserved
> -------------------------------------------------------
>
> Key: YARN-4678
> URL: https://issues.apache.org/jira/browse/YARN-4678
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Brahma Reddy Battula
> Assignee: Brahma Reddy Battula
>
> *Scenario:*
> * Start cluster with Three NM's each having 8GB (cluster memory:24GB).
> * Configure queues with elasticity and userlimitfactor=10.
> * disable pre-emption.
> * run two job with different priority in different queue at the same time
> ** yarn jar hadoop-mapreduce-examples-2.7.2.jar pi -Dyarn.app.priority=LOW
> -Dmapreduce.job.queuename=QueueA -Dmapreduce.map.memory.mb=4096
> -Dyarn.app.mapreduce.am.resource.mb=1536
> -Dmapreduce.job.reduce.slowstart.completedmaps=1.0 10 1000000000000
> ** yarn jar hadoop-mapreduce-examples-2.7.2.jar pi -Dyarn.app.priority=HIGH
> -Dmapreduce.job.queuename=QueueB -Dmapreduce.map.memory.mb=4096
> -Dyarn.app.mapreduce.am.resource.mb=1536 3 1000000000000
> * observe the cluster capacity which was used in RM web UI
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
