[
https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Ward updated HDFS-14618:
-----------------------------
Description:
I submitted a CR for this issue at:
https://github.com/apache/hadoop/pull/1030
The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe):
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70
is protected by synchronization on itself ({{timedOutItems}}):
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178
However, in one place:
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135
it is (trying to be) protected by synchronized using {{pendingReconstructions}}
--- but this cannot protect {{timedOutItems}}.
Synchronized on different objects does not ensure mutual exclusion with the
other locations.
I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and the
other by {{timedOutItems}} can still executed concurrently.
This CR adds the synchronized on {{timedOutItems}}.
Note that this CR keeps the synchronized on {{pendingReconstructions}}, which
is needed for a different purpose (protect {{pendingReconstructions}})
was:
I submitted a CR for this issue at:
https://github.com/apache/hadoop/pull/1030
The field ```timedOutItems``` (an ```ArrayList```, i.e., not thread safe):
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70
is protected by synchronization on itself (```timedOutItems```):
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178
However, in one place:
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135
it is (trying to be) protected by synchronized using
```pendingReconstructions``` --- but this cannot protect ```timedOutItems```.
Synchronized on different objects does not ensure mutual exclusion
with the other locations.
I.e., 2 code locations, one synchronized by
```pendingReconstructions``` and the other by ```timedOutItems``` can
still executed concurrently.
This CR adds the synchronized on ```timedOutItems```.
Note that this CR keeps the synchronized on
```pendingReconstructions```, which is needed for a different purpose
(protect ```pendingReconstructions```)
> Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
> --------------------------------------------------------------------------
>
> Key: HDFS-14618
> URL: https://issues.apache.org/jira/browse/HDFS-14618
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Paul Ward
> Priority: Critical
>
> I submitted a CR for this issue at:
> https://github.com/apache/hadoop/pull/1030
> The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe):
> https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70
> is protected by synchronization on itself ({{timedOutItems}}):
> https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168
> https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268
> https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178
> However, in one place:
> https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135
> it is (trying to be) protected by synchronized using
> {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}.
> Synchronized on different objects does not ensure mutual exclusion with the
> other locations.
> I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and
> the other by {{timedOutItems}} can still executed concurrently.
> This CR adds the synchronized on {{timedOutItems}}.
> Note that this CR keeps the synchronized on {{pendingReconstructions}}, which
> is needed for a different purpose (protect {{pendingReconstructions}})
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
