[
https://issues.apache.org/jira/browse/YARN-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17911018#comment-17911018
]
ASF GitHub Bot commented on YARN-11752:
---------------------------------------
shameersss1 opened a new pull request, #7277:
URL: https://github.com/apache/hadoop/pull/7277
### Description of PR
After every 2 container allocation, The global scheduling strategy for
capacity scheduler waits for 5 milliseconds. This affects the container
allocation time as well the throughput. This commit tries to fix the same by
following the same model as that of async scheduling where containers are
allocated as many times as the number of the nodes in one wave before waiting
for 5 milliseconds.
Similarly for global scheduling, all nodes in all partitions are iterated
and the containers are allocated as many times as the number of the nodes in
all the partition in one wave before waiting for 5 milliseconds
### How was this patch tested?
1. Existing unit test
2. Manual verification using YARN SLS benchmark
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Global Scheduler : Improve the container allocation time
> --------------------------------------------------------
>
> Key: YARN-11752
> URL: https://issues.apache.org/jira/browse/YARN-11752
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Syed Shameerur Rahman
> Assignee: Syed Shameerur Rahman
> Priority: Major
>
> As part of YARN-10380, The allocation strategy for multi-mode/global
> scheduler was changed from "node" list based to "partition" based, Due to
> this change, After every two container allocation, the scheduler waits for *5
> milli seconds* by default as mentioned
> [here|[https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java#L627].]
> This significantly affects the container throughput. YARN SLS benchmark shows
> Async scheduling was *6x* better than Global scheduling in terms of container
> throughput for scheduling 1600 container with 50 nodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
