[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764813=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764813 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 02/May/22 00:03 Start Date: 02/May/22 00:03 Worklog Time Spent: 10m Work Description: tomscut merged PR #4251: URL: https://github.com/apache/hadoop/pull/4251 Issue Time Tracking --- Worklog Id: (was: 764813) Time Spent: 5h 40m (was: 5.5h) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 5h 40m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764812=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764812 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 02/May/22 00:02 Start Date: 02/May/22 00:02 Worklog Time Spent: 10m Work Description: tomscut commented on PR #4251: URL: https://github.com/apache/hadoop/pull/4251#issuecomment-1114372327 The failed unit test is unrelated to the change. Issue Time Tracking --- Worklog Id: (was: 764812) Time Spent: 5.5h (was: 5h 20m) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 5.5h > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764747=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764747
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 01/May/22 08:04
Start Date: 01/May/22 08:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4251:
URL: https://github.com/apache/hadoop/pull/4251#issuecomment-1114163750
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 25s | | branch-3.3 passed |
| +1 :green_heart: | compile | 1m 29s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 1m 8s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 1m 39s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 1m 49s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 3m 58s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 28m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 27s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 50s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed |
| +1 :green_heart: | spotbugs | 3m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 228m 32s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 344m 32s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestLargeDirectoryDelete |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4251 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux a429932b2c3f 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / fe1cdcc1fddbfdffbd6aef932e8e276705296ea1 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/4/testReport/ |
| Max. process+thread count | 1914 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/4/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
Issue Time Tracking
---
Worklog Id: (was: 764747)
Time Spent: 5h 20m (was: 5h 10m)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
>
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764746=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764746
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 01/May/22 08:01
Start Date: 01/May/22 08:01
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4251:
URL: https://github.com/apache/hadoop/pull/4251#issuecomment-1114162949
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 1s | | branch-3.3 passed |
| +1 :green_heart: | compile | 1m 29s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 1m 9s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 1m 39s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 1m 49s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 4m 3s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 28m 40s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed |
| +1 :green_heart: | javac | 1m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 51s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed |
| +1 :green_heart: | spotbugs | 3m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 227m 19s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 8s | | The patch does not
generate ASF License warnings. |
| | | 344m 9s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4251 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux bdc786fcc6c8 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / fe1cdcc1fddbfdffbd6aef932e8e276705296ea1 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/2/testReport/ |
| Max. process+thread count | 1889 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/2/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
Issue Time Tracking
---
Worklog Id: (was: 764746)
Time Spent: 5h 10m (was: 5h)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> HDFS-16396 provides
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764721=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764721 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 01/May/22 02:21 Start Date: 01/May/22 02:21 Worklog Time Spent: 10m Work Description: tomscut commented on PR #4251: URL: https://github.com/apache/hadoop/pull/4251#issuecomment-1114106611 Failed unit tests seem to be unrelated to the change. I retriggered Jenkins to make sure. Issue Time Tracking --- Worklog Id: (was: 764721) Time Spent: 5h (was: 4h 50m) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 5h > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764656=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764656
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 30/Apr/22 11:43
Start Date: 30/Apr/22 11:43
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4251:
URL: https://github.com/apache/hadoop/pull/4251#issuecomment-1113973591
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 10m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 57s | | branch-3.3 passed |
| +1 :green_heart: | compile | 1m 26s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 1m 4s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 1m 34s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 1m 43s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 3m 38s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 28m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 47s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed |
| +1 :green_heart: | spotbugs | 3m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 54s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 210m 20s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not
generate ASF License warnings. |
| | | 333m 14s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4251 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 7bb0255fa870 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / fe1cdcc1fddbfdffbd6aef932e8e276705296ea1 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/1/testReport/ |
| Max. process+thread count | 2170 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4251/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
Issue Time Tracking
---
Worklog Id: (was: 764656)
Time Spent: 4h 50m (was: 4h 40m)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764634=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764634 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 30/Apr/22 08:16 Start Date: 30/Apr/22 08:16 Worklog Time Spent: 10m Work Description: tomscut commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1113946874 > Thank you @tomscut, created branch-3.3 backport PR #4251 For branch-3.2 backport, some dependent Jiras ([HDFS-16396](https://issues.apache.org/jira/browse/HDFS-16396), [HDFS-16287](https://issues.apache.org/jira/browse/HDFS-16287), [HDFS-16327](https://issues.apache.org/jira/browse/HDFS-16327)) also need to be back-ported first, so we can keep 3.3 backport for now? Thanks @virajjasani for explaining this. Issue Time Tracking --- Worklog Id: (was: 764634) Time Spent: 4h 40m (was: 4.5h) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764621=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764621 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 30/Apr/22 06:09 Start Date: 30/Apr/22 06:09 Worklog Time Spent: 10m Work Description: virajjasani commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1113927074 Thank you @tomscut, created branch-3.3 backport PR #4251 Issue Time Tracking --- Worklog Id: (was: 764621) Time Spent: 4.5h (was: 4h 20m) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 4.5h > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764620=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764620 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 30/Apr/22 06:09 Start Date: 30/Apr/22 06:09 Worklog Time Spent: 10m Work Description: virajjasani opened a new pull request, #4251: URL: https://github.com/apache/hadoop/pull/4251 branch-3.3 backport of PR https://github.com/apache/hadoop/pull/4186 Issue Time Tracking --- Worklog Id: (was: 764620) Time Spent: 4h 20m (was: 4h 10m) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764100=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764100 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 29/Apr/22 09:18 Start Date: 29/Apr/22 09:18 Worklog Time Spent: 10m Work Description: tomscut commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1113095914 Hi @virajjasani could you please cherry-pick this to branch-3.3 and branch-3.2. Thank you very much. Issue Time Tracking --- Worklog Id: (was: 764100) Time Spent: 4h 10m (was: 4h) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764089=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764089 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 29/Apr/22 09:00 Start Date: 29/Apr/22 09:00 Worklog Time Spent: 10m Work Description: tomscut merged PR #4186: URL: https://github.com/apache/hadoop/pull/4186 Issue Time Tracking --- Worklog Id: (was: 764089) Time Spent: 3h 50m (was: 3h 40m) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=764090=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764090 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 29/Apr/22 09:00 Start Date: 29/Apr/22 09:00 Worklog Time Spent: 10m Work Description: tomscut commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1113075459 Thanks @virajjasani for your contribution. Issue Time Tracking --- Worklog Id: (was: 764090) Time Spent: 4h (was: 3h 50m) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 4h > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=762647=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762647 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 27/Apr/22 01:37 Start Date: 27/Apr/22 01:37 Worklog Time Spent: 10m Work Description: virajjasani commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1110437828 Thanks for the review @tomscut ! Issue Time Tracking --- Worklog Id: (was: 762647) Time Spent: 3h 40m (was: 3.5h) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=762352=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762352
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 26/Apr/22 14:35
Start Date: 26/Apr/22 14:35
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1109875631
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 2s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 39s | | trunk passed |
| +1 :green_heart: | compile | 1m 42s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 50s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 33s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 33s | | the patch passed |
| +1 :green_heart: | compile | 1m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 3s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 34s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 365m 6s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not
generate ASF License warnings. |
| | | 485m 45s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/11/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4186 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 1314cff2b79d 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cfd7d2df778b8eb0f46d790a6c3856209fb4a3f6 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/11/testReport/ |
| Max. process+thread count | 2012 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=761606=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-761606
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 25/Apr/22 06:53
Start Date: 25/Apr/22 06:53
Worklog Time Spent: 10m
Work Description: virajjasani commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r857307250
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java:
##
@@ -1856,15 +1875,16 @@ public DatanodeCommand[]
handleHeartbeat(DatanodeRegistration nodeReg,
nodeinfo.setBalancerBandwidth(0);
}
-if (slowPeerTracker != null) {
- final Map slowPeersMap = slowPeers.getSlowPeers();
- if (!slowPeersMap.isEmpty()) {
-if (LOG.isDebugEnabled()) {
- LOG.debug("DataNode " + nodeReg + " reported slow peers: " +
- slowPeersMap);
-}
-for (String slowNodeId : slowPeersMap.keySet()) {
- slowPeerTracker.addReport(slowNodeId, nodeReg.getIpcAddr(false));
+synchronized (slowPeerTrackerLock) {
Review Comment:
Will get back with adding new class `SlowPeerDisabledTracker` that extends
`SlowPeerTracker` such that all extended methods in SlowPeerDisabledTracker are
no-op, hence we never need to assign null to `slowPeerTracker` of
DatanodeManager in the first place and hence never need to deal with NPEs. This
will save us from using volatile or synchronization as well because both of
them would add to the performance of heartbeat handling.
Toggling b/ enabled and disabled mode of "slow peer tracking" would be much
simplified this way. Let me get back with the changes tomorrow.
Thanks @tomscut for the nice discussion!
Issue Time Tracking
---
Worklog Id: (was: 761606)
Time Spent: 3h 20m (was: 3h 10m)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=761542=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-761542 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 25/Apr/22 00:59 Start Date: 25/Apr/22 00:59 Worklog Time Spent: 10m Work Description: tomscut commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1107959154 Hi @tasanuma @ayushtkn , could you please also take a look at this? Thanks. Issue Time Tracking --- Worklog Id: (was: 761542) Time Spent: 3h 10m (was: 3h) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=760535=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-760535
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 22/Apr/22 01:16
Start Date: 22/Apr/22 01:16
Worklog Time Spent: 10m
Work Description: tomscut commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r855696796
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java:
##
@@ -1856,15 +1875,16 @@ public DatanodeCommand[]
handleHeartbeat(DatanodeRegistration nodeReg,
nodeinfo.setBalancerBandwidth(0);
}
-if (slowPeerTracker != null) {
- final Map slowPeersMap = slowPeers.getSlowPeers();
- if (!slowPeersMap.isEmpty()) {
-if (LOG.isDebugEnabled()) {
- LOG.debug("DataNode " + nodeReg + " reported slow peers: " +
- slowPeersMap);
-}
-for (String slowNodeId : slowPeersMap.keySet()) {
- slowPeerTracker.addReport(slowNodeId, nodeReg.getIpcAddr(false));
+synchronized (slowPeerTrackerLock) {
Review Comment:
Thanks @virajjasani for updating. Adding lock `slowPeerTrackerLock` looks
good. But I'm not sure if it affects heartbeat here.
I have one small suggestion:
1. Make dataNodePeerStatsEnabled volatile.
2. When dataNodePeerStatsEnabledVal = false, do not set slowPeerTracker to
null.
3. Change the condition from `if (slowPeerTracker != null)` to `if
(slowPeerTracker != null) && dataNodePeerStatsEnabled` where `slowPeerTracker`
is used.
Disadvantages:
if `dataNodePeerStatsEnabledVal = false`, `slowPeerTracker` will be
retained. But I guess the operation to `disable slowPeerTracker` is not common.
Advantages:
Does not cause lock contention and does not cause NPE.
What do you think of this?
Issue Time Tracking
---
Worklog Id: (was: 760535)
Time Spent: 3h (was: 2h 50m)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=760496=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-760496
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 21/Apr/22 22:52
Start Date: 21/Apr/22 22:52
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1105834672
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 30s | | trunk passed |
| +1 :green_heart: | compile | 1m 45s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 35s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 51s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 52s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 1m 29s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 7s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 3s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 48s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 457m 26s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/9/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not
generate ASF License warnings. |
| | | 577m 16s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4186 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 301af2123a1b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 37500630b9c764183f282e649fe8e4b3af7c93ea |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/9/testReport/ |
| Max. process+thread count | 2669 (vs. ulimit of 5500) |
| modules | C:
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=760446=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-760446
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 21/Apr/22 21:26
Start Date: 21/Apr/22 21:26
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1105777078
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 20s | | trunk passed |
| +1 :green_heart: | compile | 1m 59s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 15s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 16s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 52s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 27s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 27s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 371m 25s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not
generate ASF License warnings. |
| | | 492m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4186 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 51dcc738e763 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 37500630b9c764183f282e649fe8e4b3af7c93ea |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/8/testReport/ |
| Max. process+thread count | 2082 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=759599=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759599
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 21:31
Start Date: 20/Apr/22 21:31
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1104473455
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 3s | | trunk passed |
| +1 :green_heart: | compile | 1m 43s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 22s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 387m 36s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 15s | | The patch does not
generate ASF License warnings. |
| | | 498m 51s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestReplaceDatanodeFailureReplication |
| | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4186 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 53c4c3d935ff 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0ddf08692c9f5cfe83359a7aa675a7fed8a9855d |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=759468=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759468
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 18:21
Start Date: 20/Apr/22 18:21
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1104275766
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 17s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 54s | | trunk passed |
| +1 :green_heart: | compile | 2m 3s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 51s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 51s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 29m 16s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 33s | | the patch passed |
| +1 :green_heart: | compile | 1m 38s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 38s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 6s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 3s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 37s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 59s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 420m 35s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 35s | | The patch does not
generate ASF License warnings. |
| | | 549m 7s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4186 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 3219e2d68801 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0ddf08692c9f5cfe83359a7aa675a7fed8a9855d |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/6/testReport/ |
| Max. process+thread count | 1782 (vs. ulimit of 5500) |
| modules | C:
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=759062=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759062 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 20/Apr/22 09:16 Start Date: 20/Apr/22 09:16 Worklog Time Spent: 10m Work Description: virajjasani commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1103679775 A minor change required to fix the initialization order. Awaiting QA results. Issue Time Tracking --- Worklog Id: (was: 759062) Time Spent: 2h 10m (was: 2h) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=759048=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759048
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 08:40
Start Date: 20/Apr/22 08:40
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1103627481
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 2m 11s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 9s | | trunk passed |
| +1 :green_heart: | compile | 2m 2s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 28m 26s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 38s | | the patch passed |
| +1 :green_heart: | compile | 1m 46s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 46s | | the patch passed |
| +1 :green_heart: | compile | 1m 42s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 42s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 48s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 8s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 45s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 5s | | the patch passed |
| -1 :x: | shadedclient | 28m 27s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 46m 15s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/5/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not
generate ASF License warnings. |
| | | 177m 59s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsTokens |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.hdfs.TestLeaseRecovery2 |
| | hadoop.hdfs.server.namenode.TestFSNamesystem |
| | hadoop.hdfs.TestReadStripedFileWithDecodingDeletedData |
| | hadoop.hdfs.TestHdfsAdmin |
| | hadoop.fs.viewfs.TestViewFsAtHdfsRoot |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing |
| | hadoop.hdfs.server.datanode.TestCorruptMetadataFile |
| | hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd |
| | hadoop.hdfs.TestDatanodeDeath |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.TestAbandonBlock |
| | hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks |
| | hadoop.hdfs.server.blockmanagement.TestNodeCount |
| | hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold |
| | hadoop.hdfs.TestClientReportBadBlock |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.TestPersistBlocks |
| | hadoop.hdfs.server.blockmanagement.TestRedundancyMonitor |
| | hadoop.hdfs.server.namenode.snapshot.TestListSnapshot |
| |
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758971
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 05:41
Start Date: 20/Apr/22 05:41
Worklog Time Spent: 10m
Work Description: virajjasani commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r853744159
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java:
##
@@ -366,6 +365,21 @@ public class DatanodeManager {
DFSConfigKeys.DFS_NAMENODE_BLOCKS_PER_POSTPONEDBLOCKS_RESCAN_KEY_DEFAULT);
}
+ /**
+ * Determines whether slow peer tracker should be enabled. If
dataNodePeerStatsEnabledVal is
+ * true, slow peer tracker is initialized.
+ *
+ * @param conf The configuration to use while initializing slowPeerTracker.
+ * @param timer Timer object for slowPeerTracker.
+ * @param dataNodePeerStatsEnabledVal To determine whether slow peer
tracking should be enabled.
+ */
+ public void initSlowPeerTracker(Configuration conf, Timer timer,
+ boolean dataNodePeerStatsEnabledVal) {
+this.dataNodePeerStatsEnabled = dataNodePeerStatsEnabledVal;
+this.slowPeerTracker = dataNodePeerStatsEnabled ?
+new SlowPeerTracker(conf, timer) : null;
Review Comment:
Done, please take a look @tomscut
Issue Time Tracking
---
Worklog Id: (was: 758971)
Time Spent: 1h 50m (was: 1h 40m)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758968=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758968
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 05:27
Start Date: 20/Apr/22 05:27
Worklog Time Spent: 10m
Work Description: virajjasani commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r853737690
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java:
##
@@ -366,6 +365,21 @@ public class DatanodeManager {
DFSConfigKeys.DFS_NAMENODE_BLOCKS_PER_POSTPONEDBLOCKS_RESCAN_KEY_DEFAULT);
}
+ /**
+ * Determines whether slow peer tracker should be enabled. If
dataNodePeerStatsEnabledVal is
+ * true, slow peer tracker is initialized.
+ *
+ * @param conf The configuration to use while initializing slowPeerTracker.
+ * @param timer Timer object for slowPeerTracker.
+ * @param dataNodePeerStatsEnabledVal To determine whether slow peer
tracking should be enabled.
+ */
+ public void initSlowPeerTracker(Configuration conf, Timer timer,
+ boolean dataNodePeerStatsEnabledVal) {
+this.dataNodePeerStatsEnabled = dataNodePeerStatsEnabledVal;
+this.slowPeerTracker = dataNodePeerStatsEnabled ?
+new SlowPeerTracker(conf, timer) : null;
Review Comment:
Let me get back to this in a while.
Issue Time Tracking
---
Worklog Id: (was: 758968)
Time Spent: 1h 40m (was: 1.5h)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758964=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758964
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 05:18
Start Date: 20/Apr/22 05:18
Worklog Time Spent: 10m
Work Description: virajjasani commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r853733962
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java:
##
@@ -2406,27 +2412,49 @@ String reconfigureSlowNodesParameters(final
DatanodeManager datanodeManager,
namesystem.writeLock();
String result;
try {
- if (property.equals(DFS_NAMENODE_AVOID_SLOW_DATANODE_FOR_READ_KEY)) {
-boolean enable = (newVal == null ?
DFS_NAMENODE_AVOID_SLOW_DATANODE_FOR_READ_DEFAULT :
+ switch (property) {
+ case DFS_NAMENODE_AVOID_SLOW_DATANODE_FOR_READ_KEY: {
+boolean enable = (newVal == null ?
+DFS_NAMENODE_AVOID_SLOW_DATANODE_FOR_READ_DEFAULT :
Boolean.parseBoolean(newVal));
result = Boolean.toString(enable);
datanodeManager.setAvoidSlowDataNodesForReadEnabled(enable);
- } else if (property.equals(
-DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_KEY))
{
+break;
+ }
+ case DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_KEY: {
boolean enable = (newVal == null ?
DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_DEFAULT :
Boolean.parseBoolean(newVal));
result = Boolean.toString(enable);
bm.setExcludeSlowNodesEnabled(enable);
- } else if (property.equals(DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_KEY))
{
+break;
+ }
+ case DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_KEY: {
int maxSlowpeerCollectNodes = (newVal == null ?
DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_DEFAULT :
Integer.parseInt(newVal));
result = Integer.toString(maxSlowpeerCollectNodes);
datanodeManager.setMaxSlowpeerCollectNodes(maxSlowpeerCollectNodes);
- } else {
-throw new IllegalArgumentException("Unexpected property " +
-property + " in reconfigureSlowNodesParameters");
+break;
+ }
+ case DFS_DATANODE_PEER_STATS_ENABLED_KEY: {
+Timer timer = new Timer();
+if (newVal != null && !newVal.equalsIgnoreCase("true") &&
!newVal.equalsIgnoreCase(
+"false")) {
+ throw new ReconfigurationException(property, newVal,
getConf().get(property),
+ new NumberFormatException(newVal + " is not boolean value"));
Review Comment:
Sounds good, let me change this. Thanks @tomscut
Issue Time Tracking
---
Worklog Id: (was: 758964)
Time Spent: 1.5h (was: 1h 20m)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758914=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758914
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 01:38
Start Date: 20/Apr/22 01:38
Worklog Time Spent: 10m
Work Description: tomscut commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r853655682
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java:
##
@@ -366,6 +365,21 @@ public class DatanodeManager {
DFSConfigKeys.DFS_NAMENODE_BLOCKS_PER_POSTPONEDBLOCKS_RESCAN_KEY_DEFAULT);
}
+ /**
+ * Determines whether slow peer tracker should be enabled. If
dataNodePeerStatsEnabledVal is
+ * true, slow peer tracker is initialized.
+ *
+ * @param conf The configuration to use while initializing slowPeerTracker.
+ * @param timer Timer object for slowPeerTracker.
+ * @param dataNodePeerStatsEnabledVal To determine whether slow peer
tracking should be enabled.
+ */
+ public void initSlowPeerTracker(Configuration conf, Timer timer,
+ boolean dataNodePeerStatsEnabledVal) {
+this.dataNodePeerStatsEnabled = dataNodePeerStatsEnabledVal;
+this.slowPeerTracker = dataNodePeerStatsEnabled ?
+new SlowPeerTracker(conf, timer) : null;
Review Comment:
If `this.slowPeerTracker` is set to null directly, may cause NPE.
Issue Time Tracking
---
Worklog Id: (was: 758914)
Time Spent: 1h 20m (was: 1h 10m)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758913=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758913
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 01:37
Start Date: 20/Apr/22 01:37
Worklog Time Spent: 10m
Work Description: tomscut commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r853655682
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java:
##
@@ -366,6 +365,21 @@ public class DatanodeManager {
DFSConfigKeys.DFS_NAMENODE_BLOCKS_PER_POSTPONEDBLOCKS_RESCAN_KEY_DEFAULT);
}
+ /**
+ * Determines whether slow peer tracker should be enabled. If
dataNodePeerStatsEnabledVal is
+ * true, slow peer tracker is initialized.
+ *
+ * @param conf The configuration to use while initializing slowPeerTracker.
+ * @param timer Timer object for slowPeerTracker.
+ * @param dataNodePeerStatsEnabledVal To determine whether slow peer
tracking should be enabled.
+ */
+ public void initSlowPeerTracker(Configuration conf, Timer timer,
+ boolean dataNodePeerStatsEnabledVal) {
+this.dataNodePeerStatsEnabled = dataNodePeerStatsEnabledVal;
+this.slowPeerTracker = dataNodePeerStatsEnabled ?
+new SlowPeerTracker(conf, timer) : null;
Review Comment:
If this.slowPeerTracker is set to null, may cause NPE.
Issue Time Tracking
---
Worklog Id: (was: 758913)
Time Spent: 1h 10m (was: 1h)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758912=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758912
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 20/Apr/22 01:30
Start Date: 20/Apr/22 01:30
Worklog Time Spent: 10m
Work Description: tomscut commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r853653286
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java:
##
@@ -2406,27 +2412,49 @@ String reconfigureSlowNodesParameters(final
DatanodeManager datanodeManager,
namesystem.writeLock();
String result;
try {
- if (property.equals(DFS_NAMENODE_AVOID_SLOW_DATANODE_FOR_READ_KEY)) {
-boolean enable = (newVal == null ?
DFS_NAMENODE_AVOID_SLOW_DATANODE_FOR_READ_DEFAULT :
+ switch (property) {
+ case DFS_NAMENODE_AVOID_SLOW_DATANODE_FOR_READ_KEY: {
+boolean enable = (newVal == null ?
+DFS_NAMENODE_AVOID_SLOW_DATANODE_FOR_READ_DEFAULT :
Boolean.parseBoolean(newVal));
result = Boolean.toString(enable);
datanodeManager.setAvoidSlowDataNodesForReadEnabled(enable);
- } else if (property.equals(
-DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_KEY))
{
+break;
+ }
+ case DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_KEY: {
boolean enable = (newVal == null ?
DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_DEFAULT :
Boolean.parseBoolean(newVal));
result = Boolean.toString(enable);
bm.setExcludeSlowNodesEnabled(enable);
- } else if (property.equals(DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_KEY))
{
+break;
+ }
+ case DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_KEY: {
int maxSlowpeerCollectNodes = (newVal == null ?
DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_DEFAULT :
Integer.parseInt(newVal));
result = Integer.toString(maxSlowpeerCollectNodes);
datanodeManager.setMaxSlowpeerCollectNodes(maxSlowpeerCollectNodes);
- } else {
-throw new IllegalArgumentException("Unexpected property " +
-property + " in reconfigureSlowNodesParameters");
+break;
+ }
+ case DFS_DATANODE_PEER_STATS_ENABLED_KEY: {
+Timer timer = new Timer();
+if (newVal != null && !newVal.equalsIgnoreCase("true") &&
!newVal.equalsIgnoreCase(
+"false")) {
+ throw new ReconfigurationException(property, newVal,
getConf().get(property),
+ new NumberFormatException(newVal + " is not boolean value"));
Review Comment:
Hi @virajjasani , should here throw an IllegalArgumentException?
Issue Time Tracking
---
Worklog Id: (was: 758912)
Time Spent: 1h (was: 50m)
> Reconfigure slow peer enable for Namenode
> -
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758241=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758241
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 19/Apr/22 03:32
Start Date: 19/Apr/22 03:32
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1101958299
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 52s | | trunk passed |
| +1 :green_heart: | compile | 1m 43s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 48s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 388m 32s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 499m 27s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestReplaceDatanodeFailureReplication |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4186 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux fdba476b4d70 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e6c940c30d0f61629d9041c49e297a3f3648fce0 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/4/testReport/ |
| Max. process+thread count
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758236=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758236
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 19/Apr/22 03:23
Start Date: 19/Apr/22 03:23
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1101953931
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 35s | | trunk passed |
| +1 :green_heart: | compile | 1m 40s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 16s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 43s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 29s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 20s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 38s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 371m 8s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not
generate ASF License warnings. |
| | | 489m 37s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4186 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 70ac105b10e3 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e6c940c30d0f61629d9041c49e297a3f3648fce0 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/3/testReport/ |
| Max. process+thread count | 2018 (vs. ulimit of 5500) |
| modules | C:
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758076=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758076 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 18/Apr/22 19:15 Start Date: 18/Apr/22 19:15 Worklog Time Spent: 10m Work Description: virajjasani commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1101678149 @tasanuma could you please take a look? Issue Time Tracking --- Worklog Id: (was: 758076) Time Spent: 0.5h (was: 20m) > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=758067=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758067
]
ASF GitHub Bot logged work on HDFS-16528:
-
Author: ASF GitHub Bot
Created on: 18/Apr/22 19:03
Start Date: 18/Apr/22 19:03
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1101665780
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 32s | | trunk passed |
| +1 :green_heart: | compile | 1m 31s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 52s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 31 new + 97 unchanged
- 0 fixed = 128 total (was 97) |
| +1 :green_heart: | mvnsite | 1m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 374m 21s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not
generate ASF License warnings. |
| | | 489m 13s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.tools.TestDFSAdmin |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4186/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4186 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux ac56a5acd8c1 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7e9268260ffc918051b435ca3a85fe5f2de028f6 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=757823=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757823 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot Created on: 18/Apr/22 10:53 Start Date: 18/Apr/22 10:53 Worklog Time Spent: 10m Work Description: virajjasani opened a new pull request, #4186: URL: https://github.com/apache/hadoop/pull/4186 ### Description of PR HDFS-16396 provides reconfig options for several configs associated with slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some slownodes related configs as the reconfig options in Namenode. The purpose of this PR is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as reconfigurable option for Namenode. ### How was this patch tested? Local env, and using UT. ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? Issue Time Tracking --- Worklog Id: (was: 757823) Remaining Estimate: 0h Time Spent: 10m > Reconfigure slow peer enable for Namenode > - > > Key: HDFS-16528 > URL: https://issues.apache.org/jira/browse/HDFS-16528 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > HDFS-16396 provides reconfig options for several configs associated with > slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some > slownodes related configs as the reconfig options in Namenode. > The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as > reconfigurable option for Namenode (similar to how HDFS-16396 has included it > for Datanode). -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
