map output http client does not timeout
---------------------------------------
Key: HADOOP-259
URL: http://issues.apache.org/jira/browse/HADOOP-259
Project: Hadoop
Type: Bug
Components: mapred
Versions: 0.3
Reporter: Owen O'Malley
Assigned to: Owen O'Malley
Fix For: 0.3
The new map output http client uses java.net.URLConnection to fetch the data
file. However under Java 1.4 there is no way to specify a timeout and it is set
to infinite (or if not infinite at least 12 hours). This causes reduce tasks
to get "stuck" in the "reduce > copy" phase even after the "Task failed to
report status for 600 seconds. Killing." message.
I will add the code in the ReduceTaskRunner to make sure that copies in-flight
don't get stuck, but this is another point where a switch to java 1.5 would be
helpful. Under 1.5 I could set the timeout on the connection and the read would
timeout after the given interval and my entire change would be local to
MapOutputLocation.copyFile.
For now, I'll assume that we need to maintain support for 1.4 and make the
corresponding fix, but I'm grumbling...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
