[
https://issues.apache.org/jira/browse/HADOOP-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12588098#action_12588098
]
Chris Douglas commented on HADOOP-3229:
---------------------------------------
The flag is set by TrackedRecordReader for each record given to the map. Do we
need to call it for each record emitted? It's also set after each call to
reduce in the combiner (not each record; my mistake). I thought the problem was
that the spill doesn't report progress after the sort, without a combiner ("on
writing"). As demonstrated in HADOOP-2284, the overhead of setting this flag-
as you assert- is slight, but not free.
If we wanted to set this flag after each record written, then we might as well
add a SequenceFile::createWriter method that takes a Progressable object and a
FSDataOutputStream, and cover both cases.
> Map OutputCollector does not report progress on writes
> ------------------------------------------------------
>
> Key: HADOOP-3229
> URL: https://issues.apache.org/jira/browse/HADOOP-3229
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Environment: all
> Reporter: Alejandro Abdelnur
> Fix For: 0.17.0
>
> Attachments: 3229-0.patch, HADOOP-3229.patch
>
>
> It seem that the collector implementation used during the map phase does not
> report progress on writing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
