[
https://issues.apache.org/jira/browse/HADOOP-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HADOOP-1765:
--------------------------
Attachment: namenode-excerpt.log.gz
This is namenode log excerpt at DEBUG level. To find the problematic file,
search for the string 8020883803428152097 (or for the IOException).
The creation scenario that seems to be bringing on the failure is as follows:
{code}
Create file named X
Fill it
Close it.
Rename it as file named Y
Create new file named X
Fill it
Try to close it <-- IOException
{code}
> IOException on close: Unknown file
> ----------------------------------
>
> Key: HADOOP-1765
> URL: https://issues.apache.org/jira/browse/HADOOP-1765
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: stack
> Priority: Minor
> Attachments: namenode-excerpt.log.gz
>
>
> On occasion I get an IOE trying to close a MapFile I've just opened and
> filled. Below is the content of the namenode log. The first thing in it
> after startup messages is the IOE followed by other messages about
> '...current leaseholder is trying to recreate file'. It repeats for ever.
> This is TRUNK r567876.
> I filed this issue at Raghu's suggestion. Here's what he said on the list:
> .bq Yes, the earler message was also from Namenode log. Before that line
> there should have been a message that adds a block to this file in the same
> log. I think you should file a Jira for this. It probably related to
> HADOOP-999. Raghu.
> {code}
> 2007-08-22 21:38:49,071 INFO org.apache.hadoop.net.NetworkTopology: Adding a
> new node: /default-rack/208.76.44.139:50010
> 2007-08-22 21:43:50,216 INFO org.apache.hadoop.fs.FSNamesystem: Roll Edit Log
> from 208.76.44.139
> 2007-08-22 21:45:21,459 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 1 on 9000, call
> complete(/bfd/hadoop-stack-data/tmp/hbase/compaction.tmp/hregion_hbaserepository,,8918388410463499185/repo/mapfiles/-1/data,
> DFSClient_1857293290) from 208.76.44.139:52301: error: java.io.IOException:
> Unknown file:
> /bfd/hadoop-stack-data/tmp/hbase/compaction.tmp/hregion_hbaserepository,,8918388410463499185/repo/mapfiles/-1/data
> java.io.IOException: Unknown file:
> /bfd/hadoop-stack-data/tmp/hbase/compaction.tmp/hregion_hbaserepository,,8918388410463499185/repo/mapfiles/-1/data
> at org.apache.hadoop.dfs.FSDirectory.addBlocks(FSDirectory.java:561)
> at
> org.apache.hadoop.dfs.FSNamesystem.completeFileInternal(FSNamesystem.java:1002)
> at org.apache.hadoop.dfs.FSNamesystem.completeFile(FSNamesystem.java:952)
> at org.apache.hadoop.dfs.NameNode.complete(NameNode.java:348)
> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566)
> 2007-08-22 21:45:24,394 WARN org.apache.hadoop.dfs.StateChange: DIR*
> NameSystem.startFile: failed to create file
> /bfd/hadoop-stack-data/tmp/hbase/compaction.tmp/hregion_hbaserepository,,8918388410463499185/repo/mapfiles/-1/data
> for DFSClient_1857293290 on client 208.76.44.139 because current leaseholder
> is trying to recreate file.
> 2007-08-22 21:45:24,394 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 1 on 9000, call
> create(/bfd/hadoop-stack-data/tmp/hbase/compaction.tmp/hregion_hbaserepository,,8918388410463499185/repo/mapfiles/-1/data,
> DFSClient_1857293290, true, 3, 67108864) from 208.76.44.139:52312: error:
> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file
> /bfd/hadoop-stack-data/tmp/hbase/compaction.tmp/hregion_hbaserepository,,8918388410463499185/repo/mapfiles/-1/data
> for DFSClient_1857293290 on client 208.76.44.139 because current leaseholder
> is trying to recreate file.
> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file
> /bfd/hadoop-stack-data/tmp/hbase/compaction.tmp/hregion_hbaserepository,,8918388410463499185/repo/mapfiles/-1/data
> for DFSClient_1857293290 on client 208.76.44.139 because current leaseholder
> is trying to recreate file.
> at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:740)
> at org.apache.hadoop.dfs.NameNode.create(NameNode.java:307)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566)
> {code}
> Let me try and replicate with DEBUG level enabled. Also, I think I know how
> to easily replicate. Need to do some tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
