[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14618: --- Fix Version/s: 3.2.2 3.1.4 > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Assignee: Paul Ward >Priority: Critical > Labels: fix-provided, patch-available > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: race.patch > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-14618: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Assignee: Paul Ward >Priority: Critical > Labels: fix-provided, patch-available > Fix For: 3.3.0 > > Attachments: race.patch > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Ward updated HDFS-14618: - Attachment: (was: race.patch) > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Assignee: Paul Ward >Priority: Critical > Labels: fix-provided, patch-available > Attachments: race.patch > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Ward updated HDFS-14618: - Attachment: race.patch Status: Patch Available (was: In Progress) > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Assignee: Paul Ward >Priority: Critical > Labels: fix-provided, patch-available > Attachments: race.patch, race.patch > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Ward updated HDFS-14618: - Status: In Progress (was: Patch Available) > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Assignee: Paul Ward >Priority: Critical > Labels: fix-provided, patch-available > Attachments: race.patch > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Ward updated HDFS-14618: - Labels: fix-provided patch-available (was: ) > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Priority: Critical > Labels: fix-provided, patch-available > Attachments: race.patch > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Ward updated HDFS-14618: - Attachment: (was: race.diff) > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Priority: Critical > Attachments: race.patch > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Ward updated HDFS-14618: - Attachment: race.patch > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Priority: Critical > Attachments: race.patch > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Ward updated HDFS-14618: - Attachment: race.diff Status: Patch Available (was: Open) This diff is the same from the Pull request I inlined > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Priority: Critical > Attachments: race.diff > > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 > it is (trying to be) protected by synchronized using > {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. > Synchronized on different objects does not ensure mutual exclusion with the > other locations. > I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and > the other by {{timedOutItems}} can still executed concurrently. > This CR adds the synchronized on {{timedOutItems}}. > Note that this CR keeps the synchronized on {{pendingReconstructions}}, which > is needed for a different purpose (protect {{pendingReconstructions}}) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Ward updated HDFS-14618: - Description: I submitted a CR for this issue at: https://github.com/apache/hadoop/pull/1030 The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 is protected by synchronization on itself ({{timedOutItems}}): https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 However, in one place: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 it is (trying to be) protected by synchronized using {{pendingReconstructions}} --- but this cannot protect {{timedOutItems}}. Synchronized on different objects does not ensure mutual exclusion with the other locations. I.e., 2 code locations, one synchronized by {{pendingReconstructions}} and the other by {{timedOutItems}} can still executed concurrently. This CR adds the synchronized on {{timedOutItems}}. Note that this CR keeps the synchronized on {{pendingReconstructions}}, which is needed for a different purpose (protect {{pendingReconstructions}}) was: I submitted a CR for this issue at: https://github.com/apache/hadoop/pull/1030 The field ```timedOutItems``` (an ```ArrayList```, i.e., not thread safe): https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 is protected by synchronization on itself (```timedOutItems```): https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 However, in one place: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135 it is (trying to be) protected by synchronized using ```pendingReconstructions``` --- but this cannot protect ```timedOutItems```. Synchronized on different objects does not ensure mutual exclusion with the other locations. I.e., 2 code locations, one synchronized by ```pendingReconstructions``` and the other by ```timedOutItems``` can still executed concurrently. This CR adds the synchronized on ```timedOutItems```. Note that this CR keeps the synchronized on ```pendingReconstructions```, which is needed for a different purpose (protect ```pendingReconstructions```) > Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). > -- > > Key: HDFS-14618 > URL: https://issues.apache.org/jira/browse/HDFS-14618 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Paul Ward >Priority: Critical > > I submitted a CR for this issue at: > https://github.com/apache/hadoop/pull/1030 > The field {{timedOutItems}} (an {{ArrayList}}, i.e., not thread safe): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70 > is protected by synchronization on itself ({{timedOutItems}}): > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268 > https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178 > However, in one place: >