[
https://issues.apache.org/jira/browse/HDFS-5012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310206#comment-15310206
]
Christian Bartolomaeus commented on HDFS-5012:
----------------------------------------------
Some additional information to the above stack trace: The error happened after
a machine running a DataNode crashed and rebooted unexpectedly. The warning
message was logged by that machine after the reboot when the DataNode process
started.
On three other DataNodes (those holding replicas of the block in question) the
following error was logged:
{noformat}
PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException:
THIS IS NOT SUPPOSED TO HAPPEN: replica.getGenerationStamp() >= recoveryId =
1527175689, block=blk_2570851709037266390_1527175689, replica=FinalizedReplica,
blk_2570851709037266390_1527175689, FINALIZED
getNumBytes() = 48360562
getBytesOnDisk() = 48360562
getVisibleLength()= 48360562
getVolume() = /var/lib/hdfs5/data/current
getBlockFile() =
/var/lib/hdfs5/data/current/BP-655596758-10.10.34.1-1341996058045/current/finalized/subdir38/subdir48/blk_2570851709037266390
unlinked =false
{noformat}
> replica.getGenerationStamp() may be >= recoveryId
> -------------------------------------------------
>
> Key: HDFS-5012
> URL: https://issues.apache.org/jira/browse/HDFS-5012
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.0.5-alpha
> Reporter: Ted Yu
> Attachments: testReplicationQueueFailover.txt
>
>
> The following was first observed by [~jdcryans] in
> TestReplicationQueueFailover running against 2.0.5-alpha:
> {code}
> 2013-07-16 17:14:33,340 ERROR [IPC Server handler 7 on 35081]
> security.UserGroupInformation(1481): PriviledgedActionException as:ec2-user
> (auth:SIMPLE) cause:java.io.IOException: THIS IS NOT SUPPOSED TO HAPPEN:
> replica.getGenerationStamp() >= recoveryId = 1041,
> block=blk_4297992342878601848_1041, replica=FinalizedReplica,
> blk_4297992342878601848_1041, FINALIZED
> getNumBytes() = 794
> getBytesOnDisk() = 794
> getVisibleLength()= 794
> getVolume() =
> /home/ec2-user/jenkins/workspace/HBase-0.95-Hadoop-2/hbase-server/target/test-data/f2763e32-fe49-4988-ac94-eeca82431821/dfscluster_643a635e-4e39-4aa5-974c-25e01db16ff7/dfs/data/data3/current
> getBlockFile() =
> /home/ec2-user/jenkins/workspace/HBase-0.95-Hadoop-2/hbase-server/target/test-data/f2763e32-fe49-4988-ac94-eeca82431821/dfscluster_643a635e-4e39-4aa5-974c-25e01db16ff7/dfs/data/data3/current/BP-1477359609-10.197.55.49-1373994849464/current/finalized/blk_4297992342878601848
> unlinked =false
> 2013-07-16 17:14:33,341 WARN
> [org.apache.hadoop.hdfs.server.datanode.DataNode$2@64a1fcba]
> datanode.DataNode(1894): Failed to obtain replica info for block
> (=BP-1477359609-10.197.55.49-1373994849464:blk_4297992342878601848_1041) from
> datanode (=127.0.0.1:47006)
> java.io.IOException: THIS IS NOT SUPPOSED TO HAPPEN:
> replica.getGenerationStamp() >= recoveryId = 1041,
> block=blk_4297992342878601848_1041, replica=FinalizedReplica,
> blk_4297992342878601848_1041, FINALIZED
> getNumBytes() = 794
> getBytesOnDisk() = 794
> getVisibleLength()= 794
> getVolume() =
> /home/ec2-user/jenkins/workspace/HBase-0.95-Hadoop-2/hbase-server/target/test-data/f2763e32-fe49-4988-ac94-eeca82431821/dfscluster_643a635e-4e39-4aa5-974c-25e01db16ff7/dfs/data/data3/current
> getBlockFile() =
> /home/ec2-user/jenkins/workspace/HBase-0.95-Hadoop-2/hbase-server/target/test-data/f2763e32-fe49-4988-ac94-eeca82431821/dfscluster_643a635e-4e39-4aa5-974c-25e01db16ff7/dfs/data/data3/current/BP-1477359609-10.197.55.49-1373994849464/current/finalized/blk_4297992342878601848
> unlinked =false
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
