[
https://issues.apache.org/jira/browse/HADOOP-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12570808#action_12570808
]
Raghu Angadi commented on HADOOP-2669:
--------------------------------------
Ashish, are you consistently hitting this? Runping mentioned that the client
node had no cpu left. Even if the client is very busy, I don't think it is
expected to loose the lease.
When this happens again, could you grep for file name in Namenode log and
attach here? Another useful thing would be to grep for all the blocks assigned
to this file in Namenode log.
> DFS client lost lease during writing into DFS files
> ---------------------------------------------------
>
> Key: HADOOP-2669
> URL: https://issues.apache.org/jira/browse/HADOOP-2669
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Runping Qi
>
> I have a program that reads a block compressed sequence file, does some
> processing on the records and writes the
> processed records into another block compressed sequence file.
> During execution of the program, I got the following exception:
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.dfs.LeaseExpiredException: No lease on xxxxx/part-00000
> at
> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:976)
> at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:293)
> at sun.reflect.GeneratedMethodAccessor47.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:379)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:596)
> at org.apache.hadoop.ipc.Client.call(Client.java:482)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:184)
> at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
> at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:1554)
> at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:1500)
> at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1626)
> at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.writeChunk(DFSClient.java:1602)
> at
> org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:140)
> at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:100)
> at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:39)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at
> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.writeBuffer(SequenceFile.java:1181)
> at
> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.sync(SequenceFile.java:1198)
> at
> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.append(SequenceFile.java:1248)
> at
> org.apache.hadoop.mapred.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:69)
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
