[ https://issues.apache.org/jira/browse/YARN-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Gustavsson updated YARN-4066: ----------------------------------- Attachment: YARN-4066-3.patch The formatting issues have been taken care of, sorry about that. > Large number of queues choke fair scheduler > ------------------------------------------- > > Key: YARN-4066 > URL: https://issues.apache.org/jira/browse/YARN-4066 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler > Affects Versions: 2.7.1 > Reporter: Johan Gustavsson > Assignee: Johan Gustavsson > Attachments: YARN-4066-2.patch, YARN-4066-3.patch, yarn-4066-1.patch > > > Due to synchronization and all the loops performed during queue creation, > setting a large amount of queues (12000+) will completely choke the > scheduler. To deal with this some optimization to > "QueueManager.updateAllocationConfiguration(AllocationConfiguration > queueConf)" should be done to reduce the amount of unnesecary loops. The > attached patch have been tested to work with atleast 96000 queues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)