[
https://issues.apache.org/jira/browse/YARN-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043321#comment-16043321
]
Wangda Tan commented on YARN-6679:
----------------------------------
[~daryn]/[~templedf],
I just tried to test the patch using SLS: (a generated trace which consists of
8K nodes, each node has 128GB memory, 400 concurrent running apps, size of each
app varies).
>From my test, YARN scheduler can allocate 2700-3100 containers / sec on latest
>trunk with and without the patch, I didn't see obvious regression however I
>didn't see obvious improvement either. Not sure if it is the same for
>branch-2.8 as we have already fixed some performance issues on trunk.
I don't have chance to look into the issue since theoretically this patch
should improve performance. Not sure if we hit any other bottleneck, [~daryn],
could you share size / allocation-throughput of your cluster?
> Reduce Resource instance overhead via non-PBImpl
> ------------------------------------------------
>
> Key: YARN-6679
> URL: https://issues.apache.org/jira/browse/YARN-6679
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.8.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 2.9.0, 3.0.0-alpha4
>
> Attachments: YARN-6679.2.branch-2.patch, YARN-6679.2.trunk.patch,
> YARN-6679.3.branch-2.patch, YARN-6679.3.trunk.patch,
> YARN-6679.branch-2.patch, YARN-6679.trunk.patch
>
>
> Creating and using transient PB-based Resource instances during scheduling is
> very expensive. The overhead can be transparently reduced by internally
> using lightweight non-PB based instances.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
