Paul Ward created HDFS-14610:
--------------------------------
Summary: HashMap is not thread safe. Field storageMap is typically
synchronized by storageMap. However, in one place, field storageMap is not
protected with synchronized.
Key: HDFS-14610
URL: https://issues.apache.org/jira/browse/HDFS-14610
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Paul Ward
The field *storageMap* (a *HashMap*)
[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L155]
is typically protected by synchronization on *storageMap*, e.g.,
[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L294]
[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L443]
[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L484]
For a total of 9 locations.
The reason is because *HashMap* is not thread safe.
However, here:
[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L455]
{{ DatanodeStorageInfo storage =}}
{{ storageMap.get(report.getStorage().getStorageID());}}
It is not synchronized.
Note that in the same method:
[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java#L484]
*storageMap* is again protected by synchronization:
{{ synchronized (storageMap) {}}
{{ storageMapSize = storageMap.size();}}
{{ }}}
I submitted a CR:
[https://github.com/apache/hadoop/pull/1015]
This CR protected the above instance (line 455 ) with synchronization
like in line 484 and in all other occurrences.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
