[
https://issues.apache.org/jira/browse/HADOOP-16641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946946#comment-16946946
]
Daryn Sharp commented on HADOOP-16641:
--------------------------------------
Is this a profile of a protocol using the essentially deprecated
WritableRpcEngine?
> RPC: Heavy contention on Configuration.getClassByNameOrNull
> ------------------------------------------------------------
>
> Key: HADOOP-16641
> URL: https://issues.apache.org/jira/browse/HADOOP-16641
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Gopal Vijayaraghavan
> Priority: Major
> Labels: performance
> Attachments: config-get-class-by-name.png, llap-rpc-locks.svg
>
>
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L2589
> {code}
> map = Collections.synchronizedMap(
> new WeakHashMap<String, WeakReference<Class<?>>>());
> {code}
> This synchronizes all lookups across the same class-loader across all threads
> & yields rpc threads.
> !config-get-class-by-name.png!
> When reading from HDFS with good locality, this fills up the contended lock
> profile with almost no other contributors to the locking - see
> [^llap-rpc-locks.svg]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
