[
https://issues.apache.org/jira/browse/HCATALOG-373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266770#comment-13266770
]
[email protected] commented on HCATALOG-373:
--------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4971/
-----------------------------------------------------------
Review request for hcatalog and Francis Liu.
Summary
-------
Update ProgressReporter to work with both old and new mapreduce API. Delay
creating the base record reader so we have a StatusReporter and can use
counters.
This addresses bug HCATALOG-373.
https://issues.apache.org/jira/browse/HCATALOG-373
Diffs
-----
src/java/org/apache/hcatalog/common/HCatUtil.java cb6404a
src/java/org/apache/hcatalog/mapreduce/HCatBaseInputFormat.java 268167e
src/java/org/apache/hcatalog/mapreduce/HCatRecordReader.java 65f96f4
src/java/org/apache/hcatalog/mapreduce/ProgressReporter.java fb379cd
Diff: https://reviews.apache.org/r/4971/diff
Testing
-------
"ant clean test" passes
I can run pig+hcatalog queries using Elephant-Bird deprecated API wrappers,
which is why this issue originally came up.
Thanks,
Travis
> ProgressReporter should work with both old and new MR API
> ---------------------------------------------------------
>
> Key: HCATALOG-373
> URL: https://issues.apache.org/jira/browse/HCATALOG-373
> Project: HCatalog
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: HCATALOG-373_progress_reporter.diff,
> HCATALOG-373_progress_reporter_2.diff, HCATALOG-373_progress_reporter_3.diff,
> HCATALOG-373_progress_reporter_4.diff
>
>
> {{org.apache.hcatalog.mapreduce.ProgressReporter}} currently implements
> {{org.apache.hadoop.mapred.Reporter}}. It should also extend
> {{org.apache.hadoop.mapreduce.StatusReporter}} so it works with code
> expecting either an old or new API reporter.
> The use case is using a wrapper so a serde works with a new-API input format.
> https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/mapred/input/DeprecatedInputFormatWrapper.java#L163
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
