[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17273259#comment-17273259 ] Lisheng Sun commented on HDFS-15661: Commited to trunk. Thanks [~LiJinglun] for your report and contribution! Thanks [~weichiu] for your review! > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch, HDFS-15661.002.patch, > HDFS-15661.003.patch, HDFS-15661.004.patch, HDFS-15661.005.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271747#comment-17271747 ] Wei-Chiu Chuang commented on HDFS-15661: LGTM > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch, HDFS-15661.002.patch, > HDFS-15661.003.patch, HDFS-15661.004.patch, HDFS-15661.005.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271382#comment-17271382 ] Lisheng Sun commented on HDFS-15661: +1 on https://issues.apache.org/jira/secure/attachment/13019288/HDFS-15661.005.patch > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch, HDFS-15661.002.patch, > HDFS-15661.003.patch, HDFS-15661.004.patch, HDFS-15661.005.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[
https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271354#comment-17271354
]
Hadoop QA commented on HDFS-15661:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
54s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
53s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
25s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
4s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
14s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 26s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
0s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
13s{color} | {color:blue}{color} | {color:blue} Used deprecated FindBugs
config; considering switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
46s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
3s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
10s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
10s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
58s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
58s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
4s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 3s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271212#comment-17271212 ] Jinglun commented on HDFS-15661: Hi [~leosun08], thanks your comments ! Fix checkstyle and submit v05. The failed unit tests run well on my local computer so should not be related. > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch, HDFS-15661.002.patch, > HDFS-15661.003.patch, HDFS-15661.004.patch, HDFS-15661.005.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271048#comment-17271048 ] Lisheng Sun commented on HDFS-15661: Thank [~LiJinglun] for nice fixing，according to the way we commucate offline. Can you fix the checkstyle and confirm the failed ut is unrelated to the v04 firstly ? > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch, HDFS-15661.002.patch, > HDFS-15661.003.patch, HDFS-15661.004.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[
https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270916#comment-17270916
]
Hadoop QA commented on HDFS-15661:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
48s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
1s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
6s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
14s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
52s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
22s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 41s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
1s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
0s{color} | {color:blue}{color} | {color:blue} Used deprecated FindBugs config;
considering switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
20s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
3s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
3s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
43s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
43s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 52s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/425/artifact/out/diff-checkstyle-hadoop-hdfs-project.txt{color}
| {color:orange} hadoop-hdfs-project: The patch generated 1 new + 48 unchanged
- 0 fixed = 49 total (was 48) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
0s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
|
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270862#comment-17270862 ] Jinglun commented on HDFS-15661: Submit v04 removing the redundant temporary variable 'detector'. Hi [~leosun08], please take a review when you have time. > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch, HDFS-15661.002.patch, > HDFS-15661.003.patch, HDFS-15661.004.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266795#comment-17266795 ] Jinglun commented on HDFS-15661: The failed test is not related. > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch, HDFS-15661.002.patch, > HDFS-15661.003.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[
https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266705#comment-17266705
]
Hadoop QA commented on HDFS-15661:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
46s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
57s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
2s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
10s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
52s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
14s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 25s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
6s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
5s{color} | {color:blue}{color} | {color:blue} Used deprecated FindBugs config;
considering switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
24s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
0s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
2s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
2s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
47s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
47s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
0s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 55s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
|
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266693#comment-17266693 ] Jinglun commented on HDFS-15661: Submit v03 fixing checkstyle. Hi [~weichiu] [~leosun08] could you help review v03, thanks ! > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch, HDFS-15661.002.patch, > HDFS-15661.003.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[
https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1721#comment-1721
]
Hadoop QA commented on HDFS-15661:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
13s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
19s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
22s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
15s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
54s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
18s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 29s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
7s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
2s{color} | {color:blue}{color} | {color:blue} Used deprecated FindBugs config;
considering switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
23s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
2s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
5s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
5s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
43s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
43s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 51s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/413/artifact/out/diff-checkstyle-hadoop-hdfs-project.txt{color}
| {color:orange} hadoop-hdfs-project: The patch generated 1 new + 48 unchanged
- 0 fixed = 49 total (was 48) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
2s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
|
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[
https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266059#comment-17266059
]
Jinglun commented on HDFS-15661:
Hi [~weichiu], thanks your comments !
{quote}It seems like a good idea to keep a single dead node detector object. If
I understand it correctly, the original design works in federated cluster.
After this patch, the dead node information won't be shared between difference
block pools.
{quote}
This makes sense ! I'll try the CloseableReferenceCount and submit a new patch.
> The DeadNodeDetector shouldn't be shared by different DFSClients.
> -
>
> Key: HDFS-15661
> URL: https://issues.apache.org/jira/browse/HDFS-15661
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Jinglun
>Assignee: Jinglun
>Priority: Major
> Attachments: HDFS-15661.001.patch
>
>
> Currently the DeadNodeDetector is a member of ClientContext. That means it is
> shared by many different DFSClients. When one DFSClient.close() is invoked,
> the DeadNodeDetecotor thread would be interrupted and impact other DFSClients.
> From the original design of HDFS-13571 we could see the DeadNodeDetector is
> supposed to share dead nodes of many input streams from the same client.
> We should move the DeadNodeDetector as a member of DFSClient instead of
> ClientContext.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265745#comment-17265745 ] Wei-Chiu Chuang commented on HDFS-15661: I'm not familiar with the dead node detector -- it's a new feature. The patch itself looks good. If the goal is to ensure that closing one dfs client doesn't prematurely terminate the deadnode detector thread, maybe a simpler approach is to use the CloseableReferenceCount class (in hadoop-common) to keep the reference count. It seems like a good idea to keep a single dead node detector object. If I understand it correctly, the original design works in federated cluster. After this patch, the dead node information won't be shared between difference block pools. > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264829#comment-17264829 ] Wei-Chiu Chuang commented on HDFS-15661: [~leosun08] do you have cycles to review this patch for [~LiJinglun]? > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[
https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223026#comment-17223026
]
Hadoop QA commented on HDFS-15661:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
18s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | | {color:green} The patch appears to include 1 new or modified
test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
22s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
15s{color} | | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
17s{color} | | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
23s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 33s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
0s{color} | | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
28s{color} | | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
0s{color} | | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
0s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
15s{color} | | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
15s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
50s{color} | | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
50s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no blanks issues. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 53s{color} |
[/results-checkstyle-hadoop-hdfs-project.txt|https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/273/artifact/out/results-checkstyle-hadoop-hdfs-project.txt]
| {color:orange} hadoop-hdfs-project: The patch generated 3 new + 48 unchanged
- 0 fixed = 51 total (was 48) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
1s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
20m 7s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} |
[jira] [Commented] (HDFS-15661) The DeadNodeDetector shouldn't be shared by different DFSClients.
[ https://issues.apache.org/jira/browse/HDFS-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222887#comment-17222887 ] Jinglun commented on HDFS-15661: Hi [~leosun08] [~linyiqun] [~hexiaoqiao], do you have time to help reviewing this ? Thanks very much ! > The DeadNodeDetector shouldn't be shared by different DFSClients. > - > > Key: HDFS-15661 > URL: https://issues.apache.org/jira/browse/HDFS-15661 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jinglun >Assignee: Jinglun >Priority: Major > Attachments: HDFS-15661.001.patch > > > Currently the DeadNodeDetector is a member of ClientContext. That means it is > shared by many different DFSClients. When one DFSClient.close() is invoked, > the DeadNodeDetecotor thread would be interrupted and impact other DFSClients. > From the original design of HDFS-13571 we could see the DeadNodeDetector is > supposed to share dead nodes of many input streams from the same client. > We should move the DeadNodeDetector as a member of DFSClient instead of > ClientContext. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
