[jira] [Commented] (HDFS-17431) Fix log format for BlockRecoveryWorker#recoverBlocks
[ https://issues.apache.org/jira/browse/HDFS-17431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828583#comment-17828583 ] ASF GitHub Bot commented on HDFS-17431: --- haiyang1987 commented on PR #6643: URL: https://github.com/apache/hadoop/pull/6643#issuecomment-2008608210 Thanks @dineshchitlangia @ayushtkn @wzk784533 for your review and merge~ > Fix log format for BlockRecoveryWorker#recoverBlocks > > > Key: HDFS-17431 > URL: https://issues.apache.org/jira/browse/HDFS-17431 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > Labels: pull-request-available > > Fix log format for BlockRecoveryWorker#recoverBlocks -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17431) Fix log format for BlockRecoveryWorker#recoverBlocks
[ https://issues.apache.org/jira/browse/HDFS-17431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828581#comment-17828581 ] ASF GitHub Bot commented on HDFS-17431: --- dineshchitlangia merged PR #6643: URL: https://github.com/apache/hadoop/pull/6643 > Fix log format for BlockRecoveryWorker#recoverBlocks > > > Key: HDFS-17431 > URL: https://issues.apache.org/jira/browse/HDFS-17431 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > Labels: pull-request-available > > Fix log format for BlockRecoveryWorker#recoverBlocks -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17431) Fix log format for BlockRecoveryWorker#recoverBlocks
[
https://issues.apache.org/jira/browse/HDFS-17431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828580#comment-17828580
]
ASF GitHub Bot commented on HDFS-17431:
---
haiyang1987 commented on PR #6643:
URL: https://github.com/apache/hadoop/pull/6643#issuecomment-2008603273
Thanks @wzk784533 @ayushtkn @dineshchitlangia for your review.
I found that the log format showed some problems, such as the mentioned in
this issue https://github.com/apache/hadoop/pull/6635.
```
2024-03-13 23:07:05,401 WARN datanode.DataNode
(BlockRecoveryWorker.java:run(623))
[org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$1@54e291ac] -
recover Block: RecoveringBlock{BP-xxx:blk_xxx_xxx; getBlockSize()=0;
corrupt=false; offset=-1; locs=[DatanodeInfoWithStorage[dn1:50010,null,null],
DatanodeInfoWithStorage[dn2:50010,null,null]]; cachedLocs=[]}
FAILED: {} org.apache.hadoop.ipc.RemoteException(java.io.IOException): The
recovery id 28577373754 does not match current recovery id 28578772548 for
block BP-xxx:blk_xxx_xxx
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.commitBlockSynchronization(FSNamesystem.java:4129)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.commitBlockSynchronization(NameNodeRpcServer.java:1184)
at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.commitBlockSynchronization(DatanodeProtocolServerSideTranslatorPB.java:310)
at
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:34391)
at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:635)
at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:603)
at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:587)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1137)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1236)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1134)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:2005)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3360)
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1579)
at org.apache.hadoop.ipc.Client.call(Client.java:1511)
at org.apache.hadoop.ipc.Client.call(Client.java:1402)
at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:268)
at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:142)
at com.sun.proxy.$Proxy17.commitBlockSynchronization(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.commitBlockSynchronization(DatanodeProtocolClientSideTranslatorPB.java:342)
at
org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$RecoveryTaskContiguous.syncBlock(BlockRecoveryWorker.java:334)
at
org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$RecoveryTaskContiguous.recover(BlockRecoveryWorker.java:189)
at
org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$1.run(BlockRecoveryWorker.java:620)
at java.lang.Thread.run(Thread.java:748)
```
`LOG.warn("recover Block: {} FAILED: {}", b, e);`
it invoke e will print the entire trace. so for the second placeholders is
meaningless, i think choose to remove for the second placeholders or change e
to e.toString()
Hi sir @ayushtkn @dineshchitlangia @wzk784533 what dou you think?
Thanks~
> Fix log format for BlockRecoveryWorker#recoverBlocks
>
>
> Key: HDFS-17431
> URL: https://issues.apache.org/jira/browse/HDFS-17431
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
>
> Fix log format for BlockRecoveryWorker#recoverBlocks
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17431) Fix log format for BlockRecoveryWorker#recoverBlocks
[
https://issues.apache.org/jira/browse/HDFS-17431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828490#comment-17828490
]
ASF GitHub Bot commented on HDFS-17431:
---
ayushtkn commented on code in PR #6643:
URL: https://github.com/apache/hadoop/pull/6643#discussion_r1530927655
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockRecoveryWorker.java:
##
@@ -628,7 +628,7 @@ public void run() {
new RecoveryTaskContiguous(b).recover();
}
} catch (IOException e) {
- LOG.warn("recover Block: {} FAILED: {}", b, e);
+ LOG.warn("recover Block: {} FAILED: ", b, e);
Review Comment:
Whats wrong here? the number of placeholders are correct only, for the
second one it will invoke e.toString(), now you are changing it to print the
entire trace.
I don't think it is broken, it looks like it was intentional
> Fix log format for BlockRecoveryWorker#recoverBlocks
>
>
> Key: HDFS-17431
> URL: https://issues.apache.org/jira/browse/HDFS-17431
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
>
> Fix log format for BlockRecoveryWorker#recoverBlocks
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17431) Fix log format for BlockRecoveryWorker#recoverBlocks
[
https://issues.apache.org/jira/browse/HDFS-17431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828479#comment-17828479
]
ASF GitHub Bot commented on HDFS-17431:
---
hadoop-yetus commented on PR #6643:
URL: https://github.com/apache/hadoop/pull/6643#issuecomment-2007841730
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 17s | | trunk passed |
| +1 :green_heart: | compile | 1m 29s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 29s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 4s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 18s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 41s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 228m 41s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 379m 37s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6643/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6643 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 28104aeadaba 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0ce2a9e09116ee8807a24c37e87595b52f3713da |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6643/1/testReport/ |
| Max. process+thread count | 4053 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6643/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message
[jira] [Commented] (HDFS-17431) Fix log format for BlockRecoveryWorker#recoverBlocks
[ https://issues.apache.org/jira/browse/HDFS-17431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828325#comment-17828325 ] ASF GitHub Bot commented on HDFS-17431: --- wzk784533 commented on PR #6643: URL: https://github.com/apache/hadoop/pull/6643#issuecomment-2007159204 LGTM > Fix log format for BlockRecoveryWorker#recoverBlocks > > > Key: HDFS-17431 > URL: https://issues.apache.org/jira/browse/HDFS-17431 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > Labels: pull-request-available > > Fix log format for BlockRecoveryWorker#recoverBlocks -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17431) Fix log format for BlockRecoveryWorker#recoverBlocks
[ https://issues.apache.org/jira/browse/HDFS-17431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828293#comment-17828293 ] ASF GitHub Bot commented on HDFS-17431: --- haiyang1987 opened a new pull request, #6643: URL: https://github.com/apache/hadoop/pull/6643 ### Description of PR https://issues.apache.org/jira/browse/HDFS-17431 Fix log format for BlockRecoveryWorker#recoverBlocks > Fix log format for BlockRecoveryWorker#recoverBlocks > > > Key: HDFS-17431 > URL: https://issues.apache.org/jira/browse/HDFS-17431 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > > Fix log format for BlockRecoveryWorker#recoverBlocks -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
