Ashwin Ramesh created HDFS-12092:
------------------------------------
Summary: VolumeScanner exits when block metadata file is corrupted
on datanode.
Key: HDFS-12092
URL: https://issues.apache.org/jira/browse/HDFS-12092
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode, hdfs
Affects Versions: 2.8.0
Reporter: Ashwin Ramesh
Restarted a datanode, corrupted the metafile for blk_1073741825 with something
like echo 'zzzzzzzz' > blk_1073741825_1001.meta, and datanode logs reveal that
the VolumeScanner exits due to an illegal argument exception. Here is the
relevant trace
2017-07-05 22:03:41,878
[VolumeScannerThread(/grid/0/tmp/hadoop-hdfsqa/dfs/data)] DEBUG
datanode.VolumeScanner: start scanning block
BP-955735389-10.215.76.172-1494002319684:blk_1073741825_1001
2017-07-05 22:03:41,879
[VolumeScannerThread(/grid/0/tmp/hadoop-hdfsqa/dfs/data)] ERROR
datanode.VolumeScanner: VolumeScanner(/grid/0/tmp/hadoop-hdfsqa/dfs/data,
DS-7817e9a3-c179-4901-8757-af965b27b689) exiting because of exception
java.lang.IllegalArgumentException: id=122 out of range [0, 5)
at
org.apache.hadoop.util.DataChecksum$Type.valueOf(DataChecksum.java:67)
at
org.apache.hadoop.util.DataChecksum.newDataChecksum(DataChecksum.java:123)
at
org.apache.hadoop.hdfs.server.datanode.BlockMetadataHeader.readHeader(BlockMetadataHeader.java:178)
at
org.apache.hadoop.hdfs.server.datanode.BlockMetadataHeader.readHeader(BlockMetadataHeader.java:142)
at
org.apache.hadoop.hdfs.server.datanode.BlockMetadataHeader.readHeader(BlockMetadataHeader.java:156)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.loadLastPartialChunkChecksum(FsVolumeImpl.java:1022)
at
org.apache.hadoop.hdfs.server.datanode.FinalizedReplica.getLastChecksumAndDataLen(FinalizedReplica.java:104)
at
org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:259)
at
org.apache.hadoop.hdfs.server.datanode.VolumeScanner.scanBlock(VolumeScanner.java:484)
at
org.apache.hadoop.hdfs.server.datanode.VolumeScanner.runLoop(VolumeScanner.java:614)
at
org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:699)
2017-07-05 22:03:41,879
[VolumeScannerThread(/grid/0/tmp/hadoop-hdfsqa/dfs/data)] INFO
datanode.VolumeScanner: VolumeScanner(/grid/0/tmp/hadoop-hdfsqa/dfs/data,
DS-7817e9a3-c179-4901-8757-af965b27b689) exiting.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
