[
https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-4606:
-----------------------------
Summary: CapacityScheduler: applications could get starved because
#activeUsers considers pending apps (was: Sometimes Fairness inconjuncttions
with UserLimitPercent and UserLimitFactor in queue leads to situation where it
appears that applications in queue are getting starved or stuck)
> CapacityScheduler: applications could get starved because #activeUsers
> considers pending apps
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-4606
> URL: https://issues.apache.org/jira/browse/YARN-4606
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, capacityscheduler
> Affects Versions: 2.8.0, 2.7.1
> Reporter: Karam Singh
> Assignee: Wangda Tan
>
> Encountered while studying behaviour fairness with UserLimitPercent and
> UserLimitFactor during following test:
> Ran GridMix with Queue settings: Capacity=10, MaxCap=80, UserLimit=25
> UserLimitFactor=32, FairOrderingPolicy only. Encountered a application
> starving situation where 33 application (190 apps completed out of 761 apps,
> queue can 345 containers) are running with total of 45 containers running,
> and that 12 extra only one app(the app was having around 18000 tasks) , all
> other apps were having AM running only no other containers were given any
> apps. After that app finished, there were 32 AMs that kept running without
> any containers for task being launched
> GridMix was run with following settings:
> gridmix.client.pending.queue.depth=10, gridmix.job-submission.policy=REPLAY,
> gridmix.client.submit.threads=5, gridmix.submit.multiplier=0.0001,
> gridmix.job.type=SLEEPJOB, mapreduce.framework.name=yarn,
> mapreduce.job.queuename=hive1, mapred.job.queue.name=hive1,
> gridmix.sleep.max-map-time=5000, gridmix.sleep.max-reduce-time=5000,
> gridmix.user.resolve.class=org.apache.hadoop.mapred.gridmix.RoundRobinUserResolver
> With Users file containing 4 users for RoundRobinUserResolver
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
