[
https://issues.apache.org/jira/browse/HDFS-16180?focusedWorklogId=640099&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-640099
]
ASF GitHub Bot logged work on HDFS-16180:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Aug/21 03:07
Start Date: 20/Aug/21 03:07
Worklog Time Spent: 10m
Work Description: Neilxzn commented on pull request #3315:
URL: https://github.com/apache/hadoop/pull/3315#issuecomment-902395807
cc @jojochuang. Thank you for the review. I think we should catch
FileNotFoundException about the meta file in FsVolumeImpl.nextBlock.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 640099)
Time Spent: 0.5h (was: 20m)
> FsVolumeImpl.nextBlock should consider that the block meta file has been
> deleted.
> ---------------------------------------------------------------------------------
>
> Key: HDFS-16180
> URL: https://issues.apache.org/jira/browse/HDFS-16180
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.3.0, 3.4.0
> Reporter: Max Xie
> Assignee: Max Xie
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In my cluster, we found that when VolumeScanner run, sometime dn will throw
> some error log below
> ```
>
> 2021-08-19 08:00:11,549 INFO
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService:
> Deleted BP-1020175758-nnip-1597745872895 blk_1142977964_69237147 URI
> file:/disk1/dfs/data/current/BP-1020175758-
> nnip-1597745872895/current/finalized/subdir0/subdir21/blk_1142977964
> 2021-08-19 08:00:48,368 ERROR
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl:
> nextBlock(DS-060c8e4c-1ef6-49f5-91ef-91957356891a, BP-1020175758-
> nnip-1597745872895): I/O error
> java.io.IOException: Meta file not found,
> blockFile=/disk1/dfs/data/current/BP-1020175758-
> nnip-1597745872895/current/finalized/subdir0/subdir21/blk_1142977964
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetUtil.findMetaFile(FsDatasetUtil.java:101)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl$BlockIteratorImpl.nextBlock(FsVolumeImpl.java:809)
> at
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner.runLoop(VolumeScanner.java:528)
> at
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:628)
> 2021-08-19 08:00:48,368 WARN
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner:
> VolumeScanner(/disk1/dfs/data, DS-060c8e4c-1ef6-49f5-91ef-91957356891a):
> nextBlock error on
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl$BlockIteratorImpl@7febc6b4
> ```
> When VolumeScanner scan block blk_1142977964, it has been deleted by
> datanode, scanner can not find the meta file of blk_1142977964, so it throw
> these error log.
>
> Maybe we should handle FileNotFoundException during nextblock to reduce error
> log and nextblock retry times.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
