[jira] [Commented] (HDFS-16931) Observer nn delete blocks asynchronously when tail OP_DELETE editlog
[
https://issues.apache.org/jira/browse/HDFS-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698032#comment-17698032
]
ASF GitHub Bot commented on HDFS-16931:
---
hadoop-yetus commented on PR #5424:
URL: https://github.com/apache/hadoop/pull/5424#issuecomment-1460617729
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 15s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 59s | | trunk passed |
| +1 :green_heart: | compile | 1m 28s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 55s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 26m 13s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 22s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 53s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5424/2/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 118 unchanged
- 0 fixed = 125 total (was 118) |
| +1 :green_heart: | mvnsite | 1m 23s | | the patch passed |
| -1 :x: | javadoc | 0m 53s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5424/2/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt)
| hadoop-hdfs in the patch failed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1. |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 283m 54s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5424/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 48s | | The patch does not
generate ASF License warnings. |
| | | 401m 19s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestNameEditsConfigs |
| | hadoop.fs.TestHDFSFileContextMainOperations |
| | hadoop.hdfs.TestAclsEndToEnd |
| | hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshot |
| | hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock |
| Subsystem | Report/Notes |
[jira] [Commented] (HDFS-16931) Observer nn delete blocks asynchronously when tail OP_DELETE editlog
[
https://issues.apache.org/jira/browse/HDFS-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692354#comment-17692354
]
ASF GitHub Bot commented on HDFS-16931:
---
hadoop-yetus commented on PR #5424:
URL: https://github.com/apache/hadoop/pull/5424#issuecomment-1440749446
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 21s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 2s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 2s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 52m 12s | | trunk passed |
| +1 :green_heart: | compile | 1m 38s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 10s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 1m 34s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 29m 47s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 36s | | the patch passed |
| +1 :green_heart: | compile | 1m 30s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 1m 30s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 55s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5424/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 118 unchanged
- 0 fixed = 125 total (was 118) |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| -1 :x: | javadoc | 0m 58s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5424/1/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-hdfs in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 30m 17s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 269m 3s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5424/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 406m 21s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.namenode.TestNameEditsConfigs |
| | hadoop.fs.TestHDFSFileContextMainOperations |
| | hadoop.hdfs.TestAclsEndToEnd |
| | hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshot |
| | hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5424/1/artifact/out/Dockerfile
|
| GITHUB PR |
[jira] [Commented] (HDFS-16931) Observer nn delete blocks asynchronously when tail OP_DELETE editlog
[ https://issues.apache.org/jira/browse/HDFS-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692196#comment-17692196 ] ASF GitHub Bot commented on HDFS-16931: --- liuaer opened a new pull request, #5424: URL: https://github.com/apache/hadoop/pull/5424 In our HDFS cluster that holds hundreds of millions of metadata, all of which is EC data. When Observer NN replays the OP_DELETE operation which deletes a large directory, involving tens of millions of metadata. Because the operation is synchronous, the Observer NN is stuck for nearly 50 minutes. [HDFS-16043](https://issues.apache.org/jira/browse/HDFS-16043) has solved the asynchronous deletion on the active NN. This is to solve the asynchronous deletion on the Observer NN. > Observer nn delete blocks asynchronously when tail OP_DELETE editlog > > > Key: HDFS-16931 > URL: https://issues.apache.org/jira/browse/HDFS-16931 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: kinit >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
