[ https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832796#comment-17832796 ]
ASF GitHub Bot commented on HDFS-17446: --------------------------------------- xiaojunxiang2023 commented on code in PR #6685: URL: https://github.com/apache/hadoop/pull/6685#discussion_r1546157919 ########## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java: ########## @@ -749,6 +749,8 @@ void addToReplicasMap(ReplicaMap volumeMap, File dir, lazyWriteReplicaMap, isFinalized, exceptions, subTaskQueue); subTask.fork(); subTaskQueue.add(subTask); + } else if (isFinalized && ++scanBlockCnt % 10000 == 0) { + LOG.info((scanBlockCnt - 1) + " blocks (finalize state) have been scanned."); Review Comment: > The log style is not easy to view. I have optimize it: ![Uploading image.png…]() ########## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java: ########## @@ -749,6 +749,8 @@ void addToReplicasMap(ReplicaMap volumeMap, File dir, lazyWriteReplicaMap, isFinalized, exceptions, subTaskQueue); subTask.fork(); subTaskQueue.add(subTask); + } else if (isFinalized && ++scanBlockCnt % 10000 == 0) { + LOG.info((scanBlockCnt - 1) + " blocks (finalize state) have been scanned."); Review Comment: > The log style is not easy to view. I have optimize it: ![image](https://github.com/apache/hadoop/assets/65019264/eb50cd49-2c0f-439d-8989-3252477fea71) > The DataNode adds a log to report its progress during addToReplicasMap > execution. > --------------------------------------------------------------------------------- > > Key: HDFS-17446 > URL: https://issues.apache.org/jira/browse/HDFS-17446 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs > Reporter: xiaojunxiang > Priority: Critical > Labels: pull-request-available > Attachments: image-2024-03-28-23-53-29-057.png, > image-2024-04-01-17-26-05-433.png > > > Datanodes do not print logs when addToReplicasMap is just started, so we will > not be aware of its progress, or even wait up to 4 hours for a large cluster. > This is very confusing. > A cluster is as follows: > !image-2024-03-28-23-53-29-057.png! > > The optimized result is as follows: > (Because Scan thread is ForkJoinPool, so the printing may be repeated, but it > will affect the overall progress of the presentation > ) > !image-2024-04-01-17-26-05-433.png! > > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org