[
https://issues.apache.org/jira/browse/MAPREDUCE-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508384#comment-13508384
]
Mostafa Elhemali commented on MAPREDUCE-4840:
---------------------------------------------
Ah OK looks like I was the one confused. I don't believe it works, though as
mentioned above I can't really test because of unrelated Windows problems.
There's this code in TaskAttemptListenerImpl though:
{code}
@Override
public void reportNextRecordRange(TaskAttemptID taskAttemptID, Range range)
throws IOException {
// This is used when the feature of skipping records is enabled.
// This call exists as a hadoop mapreduce legacy wherein all changes in
// counters/progress/phase/output-size are reported through statusUpdate()
// call but not the next record range information.
throw new IOException("Not yet implemented.");
}
{code}
So I guess the right thing to do is fix the implementation? Not sure if there's
a JIRA tracking that.
> Delete dead code and deprecate public API related to skipping bad records
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-4840
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4840
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Mostafa Elhemali
> Priority: Minor
> Attachments: MAPREDUCE-4840.patch
>
>
> It looks like the decision was made in MAPREDUCE-1932 to remove support for
> skipping bad records rather than fix it (it doesn't work right now in trunk).
> If that's the case then we should probably delete all the dead code related
> to it and deprecate the public API's for it right?
> Dead code I'm talking about:
> 1. Task class: skipping, skipRanges, writeSkipRecs
> 2. MapTask class: SkippingRecordReader inner class
> 3. ReduceTask class: SkippingReduceValuesIterator inner class
> 4. Tests: TestBadRecords
> Public API:
> 1. SkipBadRecords class
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
