Johan Gustavsson created YARN-4066: -------------------------------------- Summary: Large number of queues choke fair scheduler Key: YARN-4066 URL: https://issues.apache.org/jira/browse/YARN-4066 Project: Hadoop YARN Issue Type: Bug Components: fairscheduler Affects Versions: 2.7.1 Reporter: Johan Gustavsson
Due to synchronization and all the loops performed during queue creation, setting a large amount of queues (12000+) will completely choke the scheduler. To deal with this some optimization to "QueueManager.updateAllocationConfiguration(AllocationConfiguration queueConf)" should be done to reduce the amount of unnesecary loops. The attached patch have been tested to work with atleast 96000 queues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)