[
http://issues.apache.org/jira/browse/HADOOP-318?page=comments#action_12417527 ]
Owen O'Malley commented on HADOOP-318:
--------------------------------------
The only way around it that I can see is if we had:
FSDataOutputStream:
setProgressable(Progressable prog)
RecordWriter:
setProgressable(Progressable prog)
which just pushes the problem down a level.
> Progress in writing a DFS file does not count towards Job progress and can
> make the task timeout
> ------------------------------------------------------------------------------------------------
>
> Key: HADOOP-318
> URL: http://issues.apache.org/jira/browse/HADOOP-318
> Project: Hadoop
> Type: Bug
> Components: mapred
> Versions: 0.3.2
> Environment: all, but especially on big busy clusters
> Reporter: Milind Bhandarkar
> Assignee: Milind Bhandarkar
> Fix For: 0.4.0
> Attachments: hadoop-latency-new.patch, hadoop-latency.patch
>
> When a task writes to DFS file, depending on how busy the cluster is, it can
> timeout after 10 minutes by default, because the progress towards writing a
> DFS file does not count as progress of the task. The solution (patch is
> forthcoming) is to provide a way to callback reporter to report task progress
> from DFSOutputStream.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
