[ http://issues.apache.org/jira/browse/HADOOP-118?page=all ]
Owen O'Malley updated HADOOP-118:
---------------------------------
Attachment: release-create-locks.patch
This patch
1. registers DFS Clients onto a list to be close when the jvm exits.
2. makes closing a DFSClient call abandonFileCreate on each file that was not
finished.
3. it throws an exception if the client is used after it is closed.
4. removes the close done by the JobClient
5. adds the lease holder to the abandonFileInProgress message. This allows
the host's lease to be cleaned up as well as detecting attempts to abandon
other clients files.
> Namenode does not always clean up pendingCreates
> ------------------------------------------------
>
> Key: HADOOP-118
> URL: http://issues.apache.org/jira/browse/HADOOP-118
> Project: Hadoop
> Type: Bug
> Components: dfs
> Versions: 0.1.0
> Environment: 195 node cluster running redhat linux (2.6 kernel)
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Priority: Critical
> Fix For: 0.2
> Attachments: release-create-locks.patch
>
> In some failure modes, the pending creates list is not cleaned up and
> prevents that file from ever being created.
> When I try to create the file after the first job was killed (hours
> previously), I get:
> 060404 084619 Cannot start file because pendingCreates is non-null.
> src=/user/oom/rand/part000118
> 060404 084619 Server handler 0 on 8020 call error: java.io.IOException:
> Cannot create file /user/oom/rand/part000118 on client DFSClient_-1656137458
> java.io.IOException: Cannot create file /user/oom/rand/part000118 on client
> DFSClient_-1656137458
> at org.apache.hadoop.dfs.NameNode.create(NameNode.java:147)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:237)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:216)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
