[
https://issues.apache.org/jira/browse/HADOOP-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676078#action_12676078
]
Hairong Kuang commented on HADOOP-5311:
---------------------------------------
After I read the NameNode log, I realized that my conclusion in the previous
comment was wrong. It turned out both NameNode and DFSClient initiated the
pipeline recovery. NameNode started earlier and the pipeline recovery
succeeded. The log shown above was from the NN initiated pipeline recovery. As
a result, the block's generation stamp were bumped up in datanodes and in NN's
memory. When DFSClient initiated the pipeline recovery, it still used the old
generation stamp so the recovery failed with the stack trace shown in the
description. But DFSClient kept on trying forever.
> Write pipeline recovery fails
> -----------------------------
>
> Key: HADOOP-5311
> URL: https://issues.apache.org/jira/browse/HADOOP-5311
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Hairong Kuang
> Fix For: 0.18.4
>
>
> A write pipeline recovery fails on the error below:
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 53006, call
> recoverBlock(blk_1415000632081498137_954380, false,
> [Lorg.apache.hadoop.hdfs.protocol.DatanodeInfo;
> @4ec82dc6) from XX: error: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: blk_1415000632081498137_954380 is already commited,
> storedBlock == null.
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:4487)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:473)
> at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
