[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076943#comment-17076943 ] Hudson commented on HDFS-15249: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18122 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18122/]) HDFS-15249 ThrottledAsyncChecker is not thread-safe. (#1922) (github: rev c12ddbd1de40b32bbe2f6a3e484abf843d6d92ae) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/checker/ThrottledAsyncChecker.java > ThrottledAsyncChecker is not thread-safe. > - > > Key: HDFS-15249 > URL: https://issues.apache.org/jira/browse/HDFS-15249 > Project: Hadoop HDFS > Issue Type: Bug > Components: federation >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.4.0 > > > ThrottledAsyncChecker should be thread-safe because it can be used by > multiple threads when we have multiple namespaces. > *checksInProgress* and *completedChecks* are respectively HashMap and > WeakHashMap which are not thread-safe. So we need to put them in synchronized > block whenever we access them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076935#comment-17076935 ] Toshihiro Suzuki commented on HDFS-15249: - Thank you for reviewing and committing this! [~aajisaka] [~weichiu] > ThrottledAsyncChecker is not thread-safe. > - > > Key: HDFS-15249 > URL: https://issues.apache.org/jira/browse/HDFS-15249 > Project: Hadoop HDFS > Issue Type: Bug > Components: federation >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.4.0 > > > ThrottledAsyncChecker should be thread-safe because it can be used by > multiple threads when we have multiple namespaces. > *checksInProgress* and *completedChecks* are respectively HashMap and > WeakHashMap which are not thread-safe. So we need to put them in synchronized > block whenever we access them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076924#comment-17076924 ] Toshihiro Suzuki commented on HDFS-15249: - [~weichiu] How do we proceed with this? > ThrottledAsyncChecker is not thread-safe. > - > > Key: HDFS-15249 > URL: https://issues.apache.org/jira/browse/HDFS-15249 > Project: Hadoop HDFS > Issue Type: Bug > Components: federation >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > > ThrottledAsyncChecker should be thread-safe because it can be used by > multiple threads when we have multiple namespaces. > *checksInProgress* and *completedChecks* are respectively HashMap and > WeakHashMap which are not thread-safe. So we need to put them in synchronized > block whenever we access them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071456#comment-17071456 ] Toshihiro Suzuki commented on HDFS-15249: - Currently, we already have a synchronized block for *checksInProgress* and *completedChecks* in addResultCachingCallback(), but don't have in schedule(): https://github.com/apache/hadoop/blob/80b877a72f52ef0f4acafe15db55b8ed61fbe6d2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/checker/ThrottledAsyncChecker.java#L170-L174 https://github.com/apache/hadoop/blob/80b877a72f52ef0f4acafe15db55b8ed61fbe6d2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/checker/ThrottledAsyncChecker.java#L179-L183 I thought we also needed a synchronized block in schedule() and filed this Jira. > ThrottledAsyncChecker is not thread-safe. > - > > Key: HDFS-15249 > URL: https://issues.apache.org/jira/browse/HDFS-15249 > Project: Hadoop HDFS > Issue Type: Bug > Components: federation >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > > ThrottledAsyncChecker should be thread-safe because it can be used by > multiple threads when we have multiple namespaces. > *checksInProgress* and *completedChecks* are respectively HashMap and > WeakHashMap which are not thread-safe. So we need to put them in synchronized > block whenever we access them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071455#comment-17071455 ] Toshihiro Suzuki commented on HDFS-15249: - [~weichiu] Thank you for taking look at this. Actually, I didn't see any Exceptions from this issue. When I checked the source code for another issue, I found this issue. Thanks. > ThrottledAsyncChecker is not thread-safe. > - > > Key: HDFS-15249 > URL: https://issues.apache.org/jira/browse/HDFS-15249 > Project: Hadoop HDFS > Issue Type: Bug > Components: federation >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > > ThrottledAsyncChecker should be thread-safe because it can be used by > multiple threads when we have multiple namespaces. > *checksInProgress* and *completedChecks* are respectively HashMap and > WeakHashMap which are not thread-safe. So we need to put them in synchronized > block whenever we access them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071448#comment-17071448 ] Wei-Chiu Chuang commented on HDFS-15249: [~brfrn169] I assume you saw ConcurrentModificationException thrown in the DataNode. Can you post the stacktrace for future reference? > ThrottledAsyncChecker is not thread-safe. > - > > Key: HDFS-15249 > URL: https://issues.apache.org/jira/browse/HDFS-15249 > Project: Hadoop HDFS > Issue Type: Bug > Components: federation >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > > ThrottledAsyncChecker should be thread-safe because it can be used by > multiple threads when we have multiple namespaces. > *checksInProgress* and *completedChecks* are respectively HashMap and > WeakHashMap which are not thread-safe. So we need to put them in synchronized > block whenever we access them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15249) ThrottledAsyncChecker is not thread-safe.
[ https://issues.apache.org/jira/browse/HDFS-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070784#comment-17070784 ] Toshihiro Suzuki commented on HDFS-15249: - CC: [~arp] [~weichiu] > ThrottledAsyncChecker is not thread-safe. > - > > Key: HDFS-15249 > URL: https://issues.apache.org/jira/browse/HDFS-15249 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > > ThrottledAsyncChecker should be thread-safe because it can be used by > multiple threads when we have multiple namespaces. > *checksInProgress* and *completedChecks* are respectively HashMap and > WeakHashMap which are not thread-safe. So we need to put them in synchronized > block whenever we access them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
