[
https://issues.apache.org/jira/browse/MAPREDUCE-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880521#comment-13880521
]
Jason Lowe commented on MAPREDUCE-5670:
---------------------------------------
Thanks for updating the patch, looking better. Couple of things on the latest
patch:
* We should not loop for the record reader next() to return false. Since we
explicitly set up a bunch of empty files it should return false with only one
invocation (we should assert for this), and progress should still be called a
total of 3 times. If it somehow doesn't return false in the first invocation
then something is wrong, and continuing to loop is inviting a hung test if for
some reason it never does return false. ;-)
* outDir should be unique between the two tests or we risk having them collide
if the tests are run in parallel. I suggest something like
TestCombineFileRecordReader.class.getName() which will be different between the
two tests.
* Rather than loop to explicitly delete the individual files, we should just
call FileUtil.fullyDelete on outDir.
> CombineFileRecordReader should report progress when moving to the next file
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-5670
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5670
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.9
> Reporter: Jason Lowe
> Assignee: Chen He
> Priority: Minor
> Attachments: MR-5670.patch, MR-5670v2.patch
>
>
> If a combine split consists of many "empty" files (i.e.: no record found by
> the underlying record reader) then theoretically a task can timeout due to
> lack of reported progress.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
