[
https://issues.apache.org/jira/browse/HADOOP-15598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540359#comment-16540359
]
Steve Loughran commented on HADOOP-15598:
-----------------------------------------
added. FWIW, we might want to add some class which does it once, and then is
referenced: I see a number of uses of "os.type". We could also think about
scanning for uses of common properties (I'm thinking user.name, classpath, ...)
and again, having static fields somewhere, because they'll end up being used
where its expensive.
Thanks for this insight; hadn't realised it was so expensive to use
> DataChecksum calculate checksum is contented on hashtable synchronization
> -------------------------------------------------------------------------
>
> Key: HADOOP-15598
> URL: https://issues.apache.org/jira/browse/HADOOP-15598
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
> Affects Versions: 3.2.0, 3.1.1
> Reporter: Prasanth Jayachandran
> Priority: Minor
> Attachments: HADOOP-15598.1.patch, HADOOP-15598.1.patch, Screen Shot
> 2018-07-11 at 1.45.06 AM.png, Screen Shot 2018-07-11 at 2.01.54 AM.png
>
>
> When profiling a multi-threaded hive streaming ingest, observed lock
> contention on java.util.Properties getProperty() to check if os is "sparc".
> java.util.Properties internally uses HashTable. HashTable.get() is
> synchronized method. In the test application, on a 30s profile with 64
> threads ~40% CPU time is spent on getProperty() contention. See attached
> snapshot.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
