[ http://issues.apache.org/jira/browse/HADOOP-157?page=all ]
Owen O'Malley updated HADOOP-157:
---------------------------------
Attachment: pending-creates-wait.patch
This patch improves the failures reporting.
1. I created org.apache.hadoop.ipc.RemoteException class that includes the
class name of the exception that was the cause.
2. The ipc client throws this RemoteException rather than
java.rmi.RemoteException.
3. The DFSClient.create waits and retries if the file is already being created.
4. Killed tasks do not complain when they have non-zero exit codes from their
process.
5. Improved the error message when tasks are killed for not updating their
progress.
6. Dfs' ClientProtocol.addBlock now takes the client name rather than the
client machine.
7. Problems renewing dfs leases are now logged.
8. More details in the exception messages when DfsClient.create fails.
9. addBlock now checks to make sure it is the same client that owns the lease
who is adding to the file.
10. FileUnderConstruction now records who is creating the file.
11. Some new exception classes defined for problems that DFSClient wants to
catch
> job fails because pendingCreates is not cleaned up after a task fails
> ---------------------------------------------------------------------
>
> Key: HADOOP-157
> URL: http://issues.apache.org/jira/browse/HADOOP-157
> Project: Hadoop
> Type: Bug
> Components: dfs
> Versions: 0.1.1
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.2
> Attachments: pending-creates-wait.patch
>
> When a task fails under map/reduce, if the client doesn't abandon the files
> in progress (usually because it was killed), the lease on the name node lasts
> 1 minute. During that minute, I see 3 backup copies of the task fail because
> pendingCreates is non-null.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
