[
https://issues.apache.org/jira/browse/HDFS-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876498#comment-16876498
]
Anu Engineer commented on HDFS-14610:
-------------------------------------
[~paulward24] Thank you for the contribution. And welcome to Hadoop. Since you
like to post patches via Github, can I please show you a process which might
make your life easier?
The following links contain the information for Github and Hadoop.
[https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Contributor+Guide]
[https://cwiki.apache.org/confluence/display/HADOOP/Using+Github+for+Ozone+development]
If you push a patch with right Jira number, like if you commits have HDFS-14610
etc in the subject tag, then the link will be added automatically to this JIRA.
For example; if you look at how I have committed your patch. it starts with
"HDFS-14610. HashMap is not thread safe"
If you like, I can setup a video conference and help you understand the
contribution process. I am presuming that all these issues are being discovered
by some tool, and you will be posting more of these race conditions and
solutions for them. Just wanted to make sure you are comfortable with your
contributions.
As always, Thank you for your contribution to Hadoop, and I have committed this
patch to the trunk. Feel free to tag me or send me an email if you need my
attention on any of these JIRAS.
> HashMap is not thread safe. Field storageMap is typically synchronized by
> storageMap. However, in one place, field storageMap is not protected with
> synchronized.
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-14610
> URL: https://issues.apache.org/jira/browse/HDFS-14610
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Paul Ward
> Assignee: Paul Ward
> Priority: Critical
> Labels: fix-provided, patch-available
> Attachments: addingSynchronization.patch
>
>
> I submitted a CR for this issue at:
> [https://github.com/apache/hadoop/pull/1015]
> The field *storageMap* (a *HashMap*)
> [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L155]
> is typically protected by synchronization on *storageMap*, e.g.,
> [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L294]
> [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L443]
> [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L484]
> For a total of 9 locations.
> The reason is because *HashMap* is not thread safe.
> However, here:
> [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L455]
> {{DatanodeStorageInfo storage =}}
> {{ storageMap.get(report.getStorage().getStorageID());}}
> It is not synchronized.
> Note that in the same method:
> [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L484]
> *storageMap* is again protected by synchronization:
> {{synchronized (storageMap) {}}
> {{ storageMapSize = storageMap.size();}}
> {{}}}
>
> The CR I inlined above protected the above instance (line 455 ) with
> synchronization
> like in line 484 and in all other occurrences.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
