Progress in writing a DFS file does not count towards Job progress and can make
the task timeout
------------------------------------------------------------------------------------------------
Key: HADOOP-318
URL: http://issues.apache.org/jira/browse/HADOOP-318
Project: Hadoop
Type: Bug
Components: mapred
Versions: 0.3.2
Environment: all, but especially on big busy clusters
Reporter: Milind Bhandarkar
Assigned to: Milind Bhandarkar
Fix For: 0.4.0
When a task writes to DFS file, depending on how busy the cluster is, it can
timeout after 10 minutes by default, because the progress towards writing a DFS
file does not count as progress of the task. The solution (patch is
forthcoming) is to provide a way to callback reporter to report task progress
from DFSOutputStream.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
