[ https://issues.apache.org/jira/browse/HDFS-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wei-Chiu Chuang resolved HDFS-16180. ------------------------------------ Fix Version/s: 3.4.0 Resolution: Fixed > FsVolumeImpl.nextBlock should consider that the block meta file has been > deleted. > --------------------------------------------------------------------------------- > > Key: HDFS-16180 > URL: https://issues.apache.org/jira/browse/HDFS-16180 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode > Affects Versions: 3.3.0, 3.4.0 > Reporter: Max Xie > Assignee: Max Xie > Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > In my cluster, we found that when VolumeScanner run, sometime dn will throw > some error log below > ``` > > 2021-08-19 08:00:11,549 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: > Deleted BP-1020175758-nnip-1597745872895 blk_1142977964_69237147 URI > file:/disk1/dfs/data/current/BP-1020175758- > nnip-1597745872895/current/finalized/subdir0/subdir21/blk_1142977964 > 2021-08-19 08:00:48,368 ERROR > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl: > nextBlock(DS-060c8e4c-1ef6-49f5-91ef-91957356891a, BP-1020175758- > nnip-1597745872895): I/O error > java.io.IOException: Meta file not found, > blockFile=/disk1/dfs/data/current/BP-1020175758- > nnip-1597745872895/current/finalized/subdir0/subdir21/blk_1142977964 > at > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetUtil.findMetaFile(FsDatasetUtil.java:101) > at > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl$BlockIteratorImpl.nextBlock(FsVolumeImpl.java:809) > at > org.apache.hadoop.hdfs.server.datanode.VolumeScanner.runLoop(VolumeScanner.java:528) > at > org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:628) > 2021-08-19 08:00:48,368 WARN > org.apache.hadoop.hdfs.server.datanode.VolumeScanner: > VolumeScanner(/disk1/dfs/data, DS-060c8e4c-1ef6-49f5-91ef-91957356891a): > nextBlock error on > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl$BlockIteratorImpl@7febc6b4 > ``` > When VolumeScanner scan block blk_1142977964, it has been deleted by > datanode, scanner can not find the meta file of blk_1142977964, so it throw > these error log. > > Maybe we should handle FileNotFoundException during nextblock to reduce error > log and nextblock retry times. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org