[
https://issues.apache.org/jira/browse/YARN-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035505#comment-16035505
]
Daryn Sharp commented on YARN-6680:
-----------------------------------
# Maybe it was the same before, I just attacked the top few hot spots in a
profile to get us unblocked. I think Nathan measured a ~10% performance
reduction but it was enough to push the RM off a cliff. I overall achieved a
2X performance increase from my patches under this umbrella so maybe this was
low hanging fruit.
# Ah yes. There are other (unnecessary) RW lock hotspots, but the label
manager map uses locks to protect the concurrent map for essentially making a
consistent copy. A concurrent map can be iterated w/o CME but isn't guaranteed
to visit every entry hence why I think the locks are there.
# Yes. The goal is optimize the common case.
# Cloning the resource would be very bad. Resource object allocation overhead
is already very high.
> Avoid locking overhead for NO_LABEL lookups
> -------------------------------------------
>
> Key: YARN-6680
> URL: https://issues.apache.org/jira/browse/YARN-6680
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.8.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: YARN-6680.patch
>
>
> Labels are managed via a hash that is protected with a read lock. The lock
> acquire and release are each just as expensive as the hash lookup itself -
> resulting in a 3X slowdown.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
