[jira] [Commented] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[
https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832059#comment-17832059
]
ASF GitHub Bot commented on HDFS-17446:
---
hadoop-yetus commented on PR #6685:
URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2026773524
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 15s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 4s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 1m 16s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 0s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 3s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 3m 41s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/3/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html)
| hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) |
| +1 :green_heart: | shadedclient | 36m 56s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 301m 35s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 444m 55s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-hdfs-project/hadoop-hdfs |
| | Write to static field
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.scanBlockCnt
from instance method
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addToReplicasMap(ReplicaMap,
File, RamDiskReplicaTracker, boolean, List, Queue) At
BlockPoolSlice.java:from instance method
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addToReplicasMap(ReplicaMap,
File, RamDiskReplicaTracker, boolean, List, Queue) At
BlockPoolSlice.java:[line 752] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6685 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 9dde4c70fd5f 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk
[jira] [Commented] (HDFS-17447) Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs.
[
https://issues.apache.org/jira/browse/HDFS-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832043#comment-17832043
]
ASF GitHub Bot commented on HDFS-17447:
---
hadoop-yetus commented on PR #6687:
URL: https://github.com/apache/hadoop/pull/6687#issuecomment-2026691032
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 19s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 45s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 26s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6687/1/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 29s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6687/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 125 unchanged
- 1 fixed = 127 total (was 126) |
| +1 :green_heart: | mvnsite | 0m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 3s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 49s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 202m 52s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 293m 15s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6687/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6687 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5b4bf2eefbc2 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 042d612773c79d8c0916c6d67dab825fc6151d10 |
| Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6687/1/te
[jira] [Commented] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[
https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832029#comment-17832029
]
ASF GitHub Bot commented on HDFS-17446:
---
hadoop-yetus commented on PR #6685:
URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2026565855
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 35s | | trunk passed |
| +1 :green_heart: | compile | 1m 27s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 1m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 23s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 44s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 40m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 4s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 43m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 62m 40s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/5/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 221m 19s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestErasureCodingMultipleRacks |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.TestDecommissionWithBackoffMonitor |
| | hadoop.hdfs.TestSetrepDecreasing |
| | hadoop.hdfs.TestDFSShell |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6685 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 1a6afc21e874 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cdd464c29a3ec6cd8f13104b5a9c5b38b4c277e5 |
| Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/5/testRep
[jira] [Commented] (HDFS-17447) Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs.
[ https://issues.apache.org/jira/browse/HDFS-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832018#comment-17832018 ] ASF GitHub Bot commented on HDFS-17447: --- xiaojunxiang2023 closed pull request #6687: HDFS-17447. Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs. URL: https://github.com/apache/hadoop/pull/6687 > Add logging to print the implementation class of AccessControlEnforcer, print > ACEnforcer when AccessControlException occurs. > > > Key: HDFS-17447 > URL: https://issues.apache.org/jira/browse/HDFS-17447 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: xiaojunxiang >Priority: Major > Labels: pull-request-available > > As we know, when Ranger authentication is enabled in the HDFS, if Ranger > authentication fails to match, it is degraded to the FSPermissionChecker > authentication of the HDFS ACL. > If an ACE occurs, it is not clear whether it comes from Ranger authentication > or FSPermissionChecker authentication. So I think we should add a description > of AccessControlEnforcer to the ACE exception description to help locate the > problem. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17447) Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs.
[ https://issues.apache.org/jira/browse/HDFS-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaojunxiang updated HDFS-17447: Summary: Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs. (was: Add an implementation class that prints AccessControlEnforcer when an ACE exception occurs.) > Add logging to print the implementation class of AccessControlEnforcer, print > ACEnforcer when AccessControlException occurs. > > > Key: HDFS-17447 > URL: https://issues.apache.org/jira/browse/HDFS-17447 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: xiaojunxiang >Priority: Major > Labels: pull-request-available > > As we know, when Ranger authentication is enabled in the HDFS, if Ranger > authentication fails to match, it is degraded to the FSPermissionChecker > authentication of the HDFS ACL. > If an ACE occurs, it is not clear whether it comes from Ranger authentication > or FSPermissionChecker authentication. So I think we should add a description > of AccessControlEnforcer to the ACE exception description to help locate the > problem. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17447) Add an implementation class that prints AccessControlEnforcer when an ACE exception occurs.
[ https://issues.apache.org/jira/browse/HDFS-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832004#comment-17832004 ] ASF GitHub Bot commented on HDFS-17447: --- xiaojunxiang2023 opened a new pull request, #6687: URL: https://github.com/apache/hadoop/pull/6687 As we know, when Ranger authentication is enabled in the HDFS, if Ranger authentication fails to match, it is degraded to the FSPermissionChecker authentication of the HDFS ACL. If an ACE occurs, it is not clear whether it comes from Ranger authentication or FSPermissionChecker authentication. So I think we should add a description of AccessControlEnforcer to the ACE exception description to help locate the problem. > Add an implementation class that prints AccessControlEnforcer when an ACE > exception occurs. > --- > > Key: HDFS-17447 > URL: https://issues.apache.org/jira/browse/HDFS-17447 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: xiaojunxiang >Priority: Major > > As we know, when Ranger authentication is enabled in the HDFS, if Ranger > authentication fails to match, it is degraded to the FSPermissionChecker > authentication of the HDFS ACL. > If an ACE occurs, it is not clear whether it comes from Ranger authentication > or FSPermissionChecker authentication. So I think we should add a description > of AccessControlEnforcer to the ACE exception description to help locate the > problem. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17447) Add an implementation class that prints AccessControlEnforcer when an ACE exception occurs.
[ https://issues.apache.org/jira/browse/HDFS-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-17447: -- Labels: pull-request-available (was: ) > Add an implementation class that prints AccessControlEnforcer when an ACE > exception occurs. > --- > > Key: HDFS-17447 > URL: https://issues.apache.org/jira/browse/HDFS-17447 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: xiaojunxiang >Priority: Major > Labels: pull-request-available > > As we know, when Ranger authentication is enabled in the HDFS, if Ranger > authentication fails to match, it is degraded to the FSPermissionChecker > authentication of the HDFS ACL. > If an ACE occurs, it is not clear whether it comes from Ranger authentication > or FSPermissionChecker authentication. So I think we should add a description > of AccessControlEnforcer to the ACE exception description to help locate the > problem. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-17447) Add an implementation class that prints AccessControlEnforcer when an ACE exception occurs.
xiaojunxiang created HDFS-17447: --- Summary: Add an implementation class that prints AccessControlEnforcer when an ACE exception occurs. Key: HDFS-17447 URL: https://issues.apache.org/jira/browse/HDFS-17447 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Reporter: xiaojunxiang As we know, when Ranger authentication is enabled in the HDFS, if Ranger authentication fails to match, it is degraded to the FSPermissionChecker authentication of the HDFS ACL. If an ACE occurs, it is not clear whether it comes from Ranger authentication or FSPermissionChecker authentication. So I think we should add a description of AccessControlEnforcer to the ACE exception description to help locate the problem. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[
https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831994#comment-17831994
]
ASF GitHub Bot commented on HDFS-17446:
---
hadoop-yetus commented on PR #6685:
URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2026311691
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 51m 35s | | trunk passed |
| +1 :green_heart: | compile | 1m 30s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 1m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 35s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 14s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/1/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 57s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 42 unchanged -
0 fixed = 43 total (was 42) |
| +1 :green_heart: | mvnsite | 1m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 3m 18s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/1/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html)
| hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) |
| +1 :green_heart: | shadedclient | 35m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 279m 48s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 428m 53s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-hdfs-project/hadoop-hdfs |
| | Write to static field
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.scanBlockCnt
from instance method
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addToReplicasMap(ReplicaMap,
File, RamDiskReplicaTracker, boolean, List, Queue) At
BlockPoolSlice.java:from instance method
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addToReplicasMap(ReplicaMap,
File, RamDiskReplicaTracker, boolean, List, Queue) At
BlockPoolSlice.java:[line 752] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/
[jira] [Commented] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[
https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831993#comment-17831993
]
ASF GitHub Bot commented on HDFS-17446:
---
hadoop-yetus commented on PR #6685:
URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2026305411
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 51m 10s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 1m 13s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 1m 12s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 22s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 43s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 40m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 1m 6s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/2/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 1s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/2/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 42 unchanged -
0 fixed = 43 total (was 42) |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 3m 20s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/2/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html)
| hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) |
| +1 :green_heart: | shadedclient | 40m 6s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 261m 36s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not
generate ASF License warnings. |
| | | 418m 41s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-hdfs-project/hadoop-hdfs |
| | Write to static field
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.scanBlockCnt
from instance method
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addToReplicasMap(ReplicaMap,
File, RamDiskReplicaTracker, boolean, List, Queue) At
BlockPoolSlice.java:from instance method
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addToReplicasMap(ReplicaMap,
File, RamDiskReplicaTracker, boolean, List, Queue) At
BlockPoolSlice.java:[line 752] |
| Failed
[jira] [Commented] (HDFS-17389) [FGL] Client RPCs involving read process supports fine-grained lock
[
https://issues.apache.org/jira/browse/HDFS-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831913#comment-17831913
]
ASF GitHub Bot commented on HDFS-17389:
---
hadoop-yetus commented on PR #6590:
URL: https://github.com/apache/hadoop/pull/6590#issuecomment-2025749407
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 16s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ HDFS-17384 Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 0s | | HDFS-17384 passed |
| +1 :green_heart: | compile | 1m 23s | | HDFS-17384 passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 1m 17s | | HDFS-17384 passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 1m 11s | | HDFS-17384 passed |
| +1 :green_heart: | mvnsite | 1m 25s | | HDFS-17384 passed |
| +1 :green_heart: | javadoc | 1m 9s | | HDFS-17384 passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 44s | | HDFS-17384 passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 17s | | HDFS-17384 passed |
| +1 :green_heart: | shadedclient | 35m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 9s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 10s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 275m 24s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6590/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 419m 33s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestLeaseManager |
| | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6590/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6590 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4ae13f6fb718 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | HDFS-17384 / 2824464f36c56f9fb0ba1b470a08468c6f738594 |
| Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/j
[jira] [Commented] (HDFS-17388) [FGL] Client RPCs involving write process supports fine-grained lock
[
https://issues.apache.org/jira/browse/HDFS-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831908#comment-17831908
]
ASF GitHub Bot commented on HDFS-17388:
---
hadoop-yetus commented on PR #6589:
URL: https://github.com/apache/hadoop/pull/6589#issuecomment-2025736207
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ HDFS-17384 Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 0s | | HDFS-17384 passed |
| +1 :green_heart: | compile | 1m 22s | | HDFS-17384 passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 1m 17s | | HDFS-17384 passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 1m 12s | | HDFS-17384 passed |
| +1 :green_heart: | mvnsite | 1m 23s | | HDFS-17384 passed |
| +1 :green_heart: | javadoc | 1m 7s | | HDFS-17384 passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 42s | | HDFS-17384 passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 18s | | HDFS-17384 passed |
| +1 :green_heart: | shadedclient | 35m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 3m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 226m 46s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 367m 25s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6589/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6589 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 1da46dc5030c 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | HDFS-17384 / 1b095eee20311175240ea4165f41d40b92a95b47 |
| Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6589/5/testReport/ |
| Max. process+thread count | 3485 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6589/5/console |
| versions | git=2.25.1 maven=3.6
[jira] [Updated] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[ https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-17446: -- Labels: pull-request-available (was: ) > The DataNode adds a log to report its progress during addToReplicasMap > execution. > - > > Key: HDFS-17446 > URL: https://issues.apache.org/jira/browse/HDFS-17446 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: xiaojunxiang >Priority: Critical > Labels: pull-request-available > Attachments: image-2024-03-28-23-53-29-057.png > > > Datanodes do not print logs when addToReplicasMap is just started, so we will > not be aware of its progress, or even wait up to 4 hours for a large cluster. > This is very confusing. > A cluster is as follows: > !image-2024-03-28-23-53-29-057.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[ https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831882#comment-17831882 ] ASF GitHub Bot commented on HDFS-17446: --- xiaojunxiang2023 opened a new pull request, #6685: URL: https://github.com/apache/hadoop/pull/6685 Datanodes do not print logs when addToReplicasMap is just started, so we will not be aware of its progress, or even wait up to 4 hours for a large cluster. This is very confusing. A cluster is as follows:  > The DataNode adds a log to report its progress during addToReplicasMap > execution. > - > > Key: HDFS-17446 > URL: https://issues.apache.org/jira/browse/HDFS-17446 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: xiaojunxiang >Priority: Critical > Attachments: image-2024-03-28-23-53-29-057.png > > > Datanodes do not print logs when addToReplicasMap is just started, so we will > not be aware of its progress, or even wait up to 4 hours for a large cluster. > This is very confusing. > A cluster is as follows: > !image-2024-03-28-23-53-29-057.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution
xiaojunxiang created HDFS-17446: --- Summary: The DataNode adds a log to report its progress during addToReplicasMap execution Key: HDFS-17446 URL: https://issues.apache.org/jira/browse/HDFS-17446 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Reporter: xiaojunxiang Attachments: image-2024-03-28-23-53-29-057.png Datanodes do not print logs when addToReplicasMap is just started, so we will not be aware of its progress, or even wait up to 4 hours for a large cluster. This is very confusing. A cluster is as follows: !image-2024-03-28-23-53-29-057.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[ https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaojunxiang updated HDFS-17446: Summary: The DataNode adds a log to report its progress during addToReplicasMap execution. (was: The DataNode adds a log to report its progress during addToReplicasMap execution) > The DataNode adds a log to report its progress during addToReplicasMap > execution. > - > > Key: HDFS-17446 > URL: https://issues.apache.org/jira/browse/HDFS-17446 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: xiaojunxiang >Priority: Critical > Attachments: image-2024-03-28-23-53-29-057.png > > > Datanodes do not print logs when addToReplicasMap is just started, so we will > not be aware of its progress, or even wait up to 4 hours for a large cluster. > This is very confusing. > A cluster is as follows: > !image-2024-03-28-23-53-29-057.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17389) [FGL] Client RPCs involving read process supports fine-grained lock
[
https://issues.apache.org/jira/browse/HDFS-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831867#comment-17831867
]
ASF GitHub Bot commented on HDFS-17389:
---
hadoop-yetus commented on PR #6590:
URL: https://github.com/apache/hadoop/pull/6590#issuecomment-2025507325
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 22s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ HDFS-17384 Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 54s | | HDFS-17384 passed |
| +1 :green_heart: | compile | 0m 41s | | HDFS-17384 passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 0m 37s | | HDFS-17384 passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 37s | | HDFS-17384 passed |
| +1 :green_heart: | mvnsite | 0m 42s | | HDFS-17384 passed |
| +1 :green_heart: | javadoc | 0m 46s | | HDFS-17384 passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 2s | | HDFS-17384 passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 49s | | HDFS-17384 passed |
| +1 :green_heart: | shadedclient | 25m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 42s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 10s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 2m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 200m 16s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6590/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 27s | | The patch does not
generate ASF License warnings. |
| | | 301m 6s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestLeaseManager |
| | hadoop.hdfs.TestDistributedFileSystemWithECFileWithRandomECPolicy |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6590/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6590 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e9f2238f72df 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | HDFS-17384 / 2824464f36c56f9fb0ba1b470a08468c6f738594 |
| Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Test Result
[jira] [Resolved] (HDFS-17443) TestNameEditsConfigs does not check null before closing fileSys and cluster
[
https://issues.apache.org/jira/browse/HDFS-17443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Chitlangia resolved HDFS-17443.
--
Fix Version/s: 3.5.0
Assignee: ConfX
Resolution: Fixed
Thanks [~FuzzingTeam] for the contribution.
> TestNameEditsConfigs does not check null before closing fileSys and cluster
> ---
>
> Key: HDFS-17443
> URL: https://issues.apache.org/jira/browse/HDFS-17443
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ConfX
>Assignee: ConfX
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
>
> h2. What happened:
> Got NullPointerException when trying to run TestNameEditsConfigs with
> misconfiguration rather than the actual exception.
> h2. Buggy code:
> {code:java}
> @Test
> public void testNameEditsConfigsFailure() throws IOException {
> ...
> } finally {
> fileSys.close(); // here fileSys might be null
> cluster.shutdown();
> }{code}
> h2. StackTrace:
> {code:java}
> java.lang.NullPointerException: Cannot invoke
> "org.apache.hadoop.fs.FileSystem.close()" because "fileSys" is null
>at
> org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs.testNameEditsConfigsFailure(TestNameEditsConfigs.java:450){code}
> h2. How to reproduce:
> (1) Set {{dfs.namenode.edits.dir.minimum}} to {{251625215}}
> (2) Run test:
> {{org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs#testNameEditsConfigsFailure}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17443) TestNameEditsConfigs does not check null before closing fileSys and cluster
[
https://issues.apache.org/jira/browse/HDFS-17443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831861#comment-17831861
]
ASF GitHub Bot commented on HDFS-17443:
---
dineshchitlangia merged PR #6683:
URL: https://github.com/apache/hadoop/pull/6683
> TestNameEditsConfigs does not check null before closing fileSys and cluster
> ---
>
> Key: HDFS-17443
> URL: https://issues.apache.org/jira/browse/HDFS-17443
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ConfX
>Priority: Major
> Labels: pull-request-available
>
> h2. What happened:
> Got NullPointerException when trying to run TestNameEditsConfigs with
> misconfiguration rather than the actual exception.
> h2. Buggy code:
> {code:java}
> @Test
> public void testNameEditsConfigsFailure() throws IOException {
> ...
> } finally {
> fileSys.close(); // here fileSys might be null
> cluster.shutdown();
> }{code}
> h2. StackTrace:
> {code:java}
> java.lang.NullPointerException: Cannot invoke
> "org.apache.hadoop.fs.FileSystem.close()" because "fileSys" is null
>at
> org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs.testNameEditsConfigsFailure(TestNameEditsConfigs.java:450){code}
> h2. How to reproduce:
> (1) Set {{dfs.namenode.edits.dir.minimum}} to {{251625215}}
> (2) Run test:
> {{org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs#testNameEditsConfigsFailure}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-17216) When distcp handle the small files, the bandwidth parameter will be invalid, resulting in serious overspeed behavior
[ https://issues.apache.org/jira/browse/HDFS-17216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Chitlangia resolved HDFS-17216. -- Fix Version/s: 3.5.0 Assignee: xiaojunxiang Resolution: Fixed Thanks [~bigdata_zoodev] for the contribution and [~hiwangzhihui] for the review. > When distcp handle the small files, the bandwidth parameter will be invalid, > resulting in serious overspeed behavior > > > Key: HDFS-17216 > URL: https://issues.apache.org/jira/browse/HDFS-17216 > Project: Hadoop HDFS > Issue Type: Bug > Components: distcp >Affects Versions: 3.3.4 >Reporter: xiaojunxiang >Assignee: xiaojunxiang >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0 > > Attachments: DiscpAnalyze.jpg > > > When distcp copies small files (file size slightly smaller than the > bandwidth), the throbber only starts to throb after 1 second, and the > throttled is specific to a single file. so the throbber becomes invalid, > causing distcp to fill the cluster bandwidth and crush production traffic, > which is a terrible thing. > Also, it takes time for files to set up the IO pipeline, so you shouldn't > test with very small files, which will slow the transfer, especially as > bandwidth kicks in, which will amplify the impact of small files on the rate -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17216) When distcp handle the small files, the bandwidth parameter will be invalid, resulting in serious overspeed behavior
[ https://issues.apache.org/jira/browse/HDFS-17216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831843#comment-17831843 ] ASF GitHub Bot commented on HDFS-17216: --- xiaojunxiang2023 commented on PR #6138: URL: https://github.com/apache/hadoop/pull/6138#issuecomment-2025369238 > +1 LGTM, thanks @xiaojunxiang2023 for the contribution and @hiwangzhihui for review. @dineshchitlangia @hiwangzhihui thank you very much. > When distcp handle the small files, the bandwidth parameter will be invalid, > resulting in serious overspeed behavior > > > Key: HDFS-17216 > URL: https://issues.apache.org/jira/browse/HDFS-17216 > Project: Hadoop HDFS > Issue Type: Bug > Components: distcp >Affects Versions: 3.3.4 >Reporter: xiaojunxiang >Priority: Major > Labels: pull-request-available > Attachments: DiscpAnalyze.jpg > > > When distcp copies small files (file size slightly smaller than the > bandwidth), the throbber only starts to throb after 1 second, and the > throttled is specific to a single file. so the throbber becomes invalid, > causing distcp to fill the cluster bandwidth and crush production traffic, > which is a terrible thing. > Also, it takes time for files to set up the IO pipeline, so you shouldn't > test with very small files, which will slow the transfer, especially as > bandwidth kicks in, which will amplify the impact of small files on the rate -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17216) When distcp handle the small files, the bandwidth parameter will be invalid, resulting in serious overspeed behavior
[ https://issues.apache.org/jira/browse/HDFS-17216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831838#comment-17831838 ] ASF GitHub Bot commented on HDFS-17216: --- dineshchitlangia merged PR #6138: URL: https://github.com/apache/hadoop/pull/6138 > When distcp handle the small files, the bandwidth parameter will be invalid, > resulting in serious overspeed behavior > > > Key: HDFS-17216 > URL: https://issues.apache.org/jira/browse/HDFS-17216 > Project: Hadoop HDFS > Issue Type: Bug > Components: distcp >Affects Versions: 3.3.4 >Reporter: xiaojunxiang >Priority: Major > Labels: pull-request-available > Attachments: DiscpAnalyze.jpg > > > When distcp copies small files (file size slightly smaller than the > bandwidth), the throbber only starts to throb after 1 second, and the > throttled is specific to a single file. so the throbber becomes invalid, > causing distcp to fill the cluster bandwidth and crush production traffic, > which is a terrible thing. > Also, it takes time for files to set up the IO pipeline, so you shouldn't > test with very small files, which will slow the transfer, especially as > bandwidth kicks in, which will amplify the impact of small files on the rate -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17216) When distcp handle the small files, the bandwidth parameter will be invalid, resulting in serious overspeed behavior
[ https://issues.apache.org/jira/browse/HDFS-17216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831832#comment-17831832 ] ASF GitHub Bot commented on HDFS-17216: --- hiwangzhihui commented on PR #6138: URL: https://github.com/apache/hadoop/pull/6138#issuecomment-2025324133 > Overall, LGTM. Pls address the issues from CI run - blank line, checkstyle and review test failures. hi，@dineshchitlangia . Now it has passed the Jenkins Check, please help to merge it. > When distcp handle the small files, the bandwidth parameter will be invalid, > resulting in serious overspeed behavior > > > Key: HDFS-17216 > URL: https://issues.apache.org/jira/browse/HDFS-17216 > Project: Hadoop HDFS > Issue Type: Bug > Components: distcp >Affects Versions: 3.3.4 >Reporter: xiaojunxiang >Priority: Major > Labels: pull-request-available > Attachments: DiscpAnalyze.jpg > > > When distcp copies small files (file size slightly smaller than the > bandwidth), the throbber only starts to throb after 1 second, and the > throttled is specific to a single file. so the throbber becomes invalid, > causing distcp to fill the cluster bandwidth and crush production traffic, > which is a terrible thing. > Also, it takes time for files to set up the IO pipeline, so you shouldn't > test with very small files, which will slow the transfer, especially as > bandwidth kicks in, which will amplify the impact of small files on the rate -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17388) [FGL] Client RPCs involving write process supports fine-grained lock
[
https://issues.apache.org/jira/browse/HDFS-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831725#comment-17831725
]
ASF GitHub Bot commented on HDFS-17388:
---
ferhui commented on code in PR #6589:
URL: https://github.com/apache/hadoop/pull/6589#discussion_r1542703171
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java:
##
@@ -982,6 +982,11 @@ public final long computeFileSizeNotIncludingLastUcBlock()
{
* if includesLastUcBlock == false.
* @return file size
*/
+ // ComputeFileSize only needs the FSLock even through it involves block.
Review Comment:
can add new comments into the above comment scope?
e.g. below 'Compute file size of the current file.'
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java:
##
@@ -1998,6 +1998,7 @@ FileStatus getAuditFileInfo(INodesInPath iip)
if (inode.isFile()) {
final INodeFile fileNode = inode.asFile();
+ // Depends on status of the last block
Review Comment:
do you mean it needs to hold lock here?
Can add more comments here.
> [FGL] Client RPCs involving write process supports fine-grained lock
>
>
> Key: HDFS-17388
> URL: https://issues.apache.org/jira/browse/HDFS-17388
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
>
> The client write process involves many client RPCs.
>
> This ticket is used to make these RPCs support fine-grained lock.
> * mkdir
> * create
> * addBlock
> * abandonBlock
> * getAdditionalDatanode
> * updateBlockForPipeline
> * updatePipeline
> * fsync
> * complete
> * rename
> * rename2
> * append
> * renewLease
> * recoverLease
> * delete
> * createSymlink
> * renewDelegationToken
> * cancelDelegationToken
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-17445) [FGL] All remaining operations support fine-grained locking
ZanderXu created HDFS-17445: --- Summary: [FGL] All remaining operations support fine-grained locking Key: HDFS-17445 URL: https://issues.apache.org/jira/browse/HDFS-17445 Project: Hadoop HDFS Issue Type: Sub-task Reporter: ZanderXu Assignee: ZanderXu -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17438) RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.
[
https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831697#comment-17831697
]
ASF GitHub Bot commented on HDFS-17438:
---
hadoop-yetus commented on PR #6655:
URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2024788874
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 53s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 1m 22s |
[/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/4/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html)
| hadoop-hdfs-project/hadoop-hdfs-rbf in trunk has 1 extant spotbugs
warnings. |
| +1 :green_heart: | shadedclient | 33m 57s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 34m 17s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 32m 31s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 160m 19s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination |
| | hadoop.hdfs.server.federation.resolver.TestNamenodeResolver |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6655 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 33bedc335b1f 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linu
[jira] [Commented] (HDFS-17438) RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.
[
https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831698#comment-17831698
]
ASF GitHub Bot commented on HDFS-17438:
---
hadoop-yetus commented on PR #6655:
URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2024789343
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 51m 44s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 1m 21s |
[/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/2/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html)
| hadoop-hdfs-project/hadoop-hdfs-rbf in trunk has 1 extant spotbugs
warnings. |
| +1 :green_heart: | shadedclient | 39m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 18s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 24s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 32m 21s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 180m 41s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.federation.resolver.TestNamenodeResolver |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6655 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 233f8dbfcbcc 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f6811333e47f3a0b4c6e3a838a95d64fdbda0270 |
| Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/ja
[jira] [Commented] (HDFS-17438) RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.
[
https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831687#comment-17831687
]
ASF GitHub Bot commented on HDFS-17438:
---
hadoop-yetus commented on PR #6655:
URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2024744856
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 52s | | trunk passed |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 1m 21s |
[/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/3/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html)
| hadoop-hdfs-project/hadoop-hdfs-rbf in trunk has 1 extant spotbugs
warnings. |
| +1 :green_heart: | shadedclient | 32m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 29m 19s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 155m 33s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.federation.security.TestRouterSecurityManager |
| | hadoop.hdfs.server.federation.resolver.TestNamenodeResolver |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6655 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5ef6b4c2b4f6 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14
13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f6811333e47f3a0b4c6e3a838a95d64fdbda0270 |
| Default Java | Private Build-1.8.0
[jira] [Commented] (HDFS-17397) Choose another DN as soon as possible, when encountering network issues
[
https://issues.apache.org/jira/browse/HDFS-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831684#comment-17831684
]
ASF GitHub Bot commented on HDFS-17397:
---
hadoop-yetus commented on PR #6591:
URL: https://github.com/apache/hadoop/pull/6591#issuecomment-2024733961
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 30s | | trunk passed |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 0m 57s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 2m 38s |
[/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html)
| hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs
warnings. |
| +1 :green_heart: | shadedclient | 34m 49s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 49s | | the patch passed |
| +1 :green_heart: | compile | 0m 53s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 0m 45s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 47s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 2m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 38s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 25s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 135m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6591 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 8e980caff1e4 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 73d6c12734975d4adbd52f39e810478322b55a9f |
| Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/testReport/ |
| Max. process+thread count | 552 (vs. ul
[jira] [Commented] (HDFS-17397) Choose another DN as soon as possible, when encountering network issues
[ https://issues.apache.org/jira/browse/HDFS-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831656#comment-17831656 ] ASF GitHub Bot commented on HDFS-17397: --- xleoken commented on PR #6591: URL: https://github.com/apache/hadoop/pull/6591#issuecomment-2024587940 @Hexiaoqiao I tried to learn more about the `DataStreamer`. 1、Write  2、Default block size: 128MB, packet size: 64KB, Add debug log, DataStreamer -> ResponseProcessor  3、Simulate error exception  4、`dfs.datanode.socket.write.timeout` and `dfs.client.slow.io.warning.threshold.ms`  > Choose another DN as soon as possible, when encountering network issues > --- > > Key: HDFS-17397 > URL: https://issues.apache.org/jira/browse/HDFS-17397 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: xleoken >Priority: Minor > Labels: pull-request-available > Attachments: hadoop.png > > > Choose another DN as soon as possible, when encountering network issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17444) Add getJournalSyncerStatus jmx metrics, to Indicates whether the JournalSyncer thread for the every namespace has enter working state.
[ https://issues.apache.org/jira/browse/HDFS-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaojunxiang updated HDFS-17444: Description: The JornalNode JVM process is not immediately in a normal state until the JournalSyncer thread is started. For some management platforms such as Ambari rolling restart JournalNode, we need a jmx metric to determine whether the JournalSyncer thread is started and enter working state for current namespace before restarting the next JournalNode. Otherwise, restart too quickly and more than half of JournalNodes will be out of order, causing the NameNode to die. When i add it , the effect is as follows: !example-1.png! was: The JornalNode JVM process is not immediately in a normal state until the JournalSyncer thread is started. For some management platforms such as Ambari rolling restart JournalNode, we need a jmx metric to determine whether the JournalSyncer thread is started and enter working state for current namespace before restarting the next JournalNode. Otherwise, restart too quickly and more than half of JournalNodes will be out of order, causing the NameNode to die. Extra info: The JournalSyncer thread is started when a NameNode requests EditLog information. > Add getJournalSyncerStatus jmx metrics, to Indicates whether the > JournalSyncer thread for the every namespace has enter working state. > -- > > Key: HDFS-17444 > URL: https://issues.apache.org/jira/browse/HDFS-17444 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.3.4, 3.3.6 >Reporter: xiaojunxiang >Assignee: xiaojunxiang >Priority: Major > Attachments: example-1.png > > > The JornalNode JVM process is not immediately in a normal state until the > JournalSyncer thread is started. > For some management platforms such as Ambari rolling restart JournalNode, > we need a jmx metric to determine whether the JournalSyncer thread is started > and enter working state for current namespace before restarting the next > JournalNode. Otherwise, restart too quickly and more than half of > JournalNodes will be out of order, causing the NameNode to die. > When i add it , the effect is as follows: >!example-1.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17444) Add getJournalSyncerStatus jmx metrics, to Indicates whether the JournalSyncer thread for the every namespace has enter working state.
[ https://issues.apache.org/jira/browse/HDFS-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaojunxiang updated HDFS-17444: Attachment: example-1.png > Add getJournalSyncerStatus jmx metrics, to Indicates whether the > JournalSyncer thread for the every namespace has enter working state. > -- > > Key: HDFS-17444 > URL: https://issues.apache.org/jira/browse/HDFS-17444 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.3.4, 3.3.6 >Reporter: xiaojunxiang >Assignee: xiaojunxiang >Priority: Major > Attachments: example-1.png > > > The JornalNode JVM process is not immediately in a normal state until the > JournalSyncer thread is started. > For some management platforms such as Ambari rolling restart JournalNode, > we need a jmx metric to determine whether the JournalSyncer thread is started > and enter working state for current namespace before restarting the next > JournalNode. Otherwise, restart too quickly and more than half of > JournalNodes will be out of order, causing the NameNode to die. > Extra info: The JournalSyncer thread is started when a NameNode requests > EditLog information. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17397) Choose another DN as soon as possible, when encountering network issues
[ https://issues.apache.org/jira/browse/HDFS-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831650#comment-17831650 ] ASF GitHub Bot commented on HDFS-17397: --- xleoken opened a new pull request, #6591: URL: https://github.com/apache/hadoop/pull/6591 ### Description of PR When there is a network issue between the client and DN, the write process will enter hang state. We hope to choose another DN as soon as possible when encountering network problems.   ### How was this patch tested? Local tested. ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Choose another DN as soon as possible, when encountering network issues > --- > > Key: HDFS-17397 > URL: https://issues.apache.org/jira/browse/HDFS-17397 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: xleoken >Priority: Minor > Labels: pull-request-available > Attachments: hadoop.png > > > Choose another DN as soon as possible, when encountering network issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
