[
https://issues.apache.org/jira/browse/HADOOP-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676063#action_12676063
]
Hairong Kuang commented on HADOOP-5311:
---------------------------------------
Here is more information regarding this failure. The log came from the same
primary datanode:
org.apache.hadoop.hdfs.server.datanode.DataNode:
oldblock=blk_1415000632081498137_954380(length=31016448),
newblock=blk_1415000632081498137_989001(length=31016448), datanode=XX
org.apache.hadoop.hdfs.server.datanode.DataNode: Received block
blk_1415000632081498137_989001 of size 31016448 as part of lease recovery.
It looks that pipeline recovery succeeded at the primary datanode. The new
generation stamp is 989001. But the client saw this recovery as a failure and
used the old generation stamp 954380 to recover the error:
WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
blk_1415000632081498137_954380 failed because recovery from primary datanode
XX failed 4 times. Will retry...
This retry went on forever.
> Write pipeline recovery fails
> -----------------------------
>
> Key: HADOOP-5311
> URL: https://issues.apache.org/jira/browse/HADOOP-5311
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Hairong Kuang
> Fix For: 0.18.4
>
>
> A write pipeline recovery fails on the error below:
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 53006, call
> recoverBlock(blk_1415000632081498137_954380, false,
> [Lorg.apache.hadoop.hdfs.protocol.DatanodeInfo;
> @4ec82dc6) from XX: error: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: blk_1415000632081498137_954380 is already commited,
> storedBlock == null.
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:4487)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:473)
> at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
