[
https://issues.apache.org/jira/browse/MAPREDUCE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906977#comment-14906977
]
Kuhu Shukla commented on MAPREDUCE-6473:
----------------------------------------
Thank you [~jlowe] .
I thought of making the list volatile. Here is a scenario which I felt could be
a problem.
Thread T1 comes into initialize() , calls initProviderList(), checks that the
list is null, which is true the first(few) times. It gets into the sync block,
since no one is in there. T1 hasn't finished populating the list but has
initialized it (so now the list == null is false) and right then another Thread
T2, calls initProviderList() checks that the list is still null, gets false and
returns to initialize() , now it starts iterating over the list which is still
being written by T1. This case would happen right at the beginning but since I
suspected it being ideal for a concurrency exception, I went with another
volatile object. If we remove the first (list == null) check before the sync
block and have only the one inside it, then every thread has to contend for the
sync block even if it is to check whether the list is initialized or not. If
the latter case is not a concern then making the list volatile works.
> Job submission can take a long time during Cluster initialization
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-6473
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6473
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: performance
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: 99%ile.png, MAPREDUCE-6473-PerfTest.txt,
> MAPREDUCE-6473-v1.patch, MAPREDUCE-6473-v2.patch, MAPREDUCE-6473-v3.patch,
> MAPREDUCE-6473-v4.patch, MAPREDUCE-6473-v5.patch, avgtime.png
>
>
> During initialization in Cluster.java, the framework provider classes are
> loaded inside a sync block which can considerably increase job submission
> time when the number of submissions are high. The motive is to reduce time
> spent in this sync block safely to improve performance.
> {noformat}
> synchronized (frameworkLoader) {
> for (ClientProtocolProvider provider : frameworkLoader) {
> LOG.debug("Trying ClientProtocolProvider : "
> + provider.getClass().getName());
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
