[
https://issues.apache.org/jira/browse/HADOOP-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622142#action_12622142
]
Arun C Murthy commented on HADOOP-2168:
---------------------------------------
This is turning out to be harder than I expected - the problem being that only
MapTask.TrackerRecordReader updates progress and in this case the Pipes
application isn't looping through the Java RecordReader...
Possible solutions:
1. Add a 'setProgress' api to org.apache.hadoop.mapred.Reporter - this is quite
dangerous since applications can potentially use this to throw the Scheduler
off.
2. Use a dummy InputFormat/RecordReader along with a specialized Reporter which
can communicate the progress to the RecordReader and hence the rest of the
framework.
Given the dangers of the first approach, Owen/me agree we should go with the
second for now. Thoughts?
> Pipes with a C++ record reader does not update progress in the map until it
> is 100%
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2168
> URL: https://issues.apache.org/jira/browse/HADOOP-2168
> Project: Hadoop Core
> Issue Type: Bug
> Components: pipes
> Reporter: Owen O'Malley
> Assignee: Arun C Murthy
>
> Currently C++ maps with C++ record readers do not update the progress, even
> if the application has updated it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
