[
https://issues.apache.org/jira/browse/HCATALOG-373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268767#comment-13268767
]
[email protected] commented on HCATALOG-373:
--------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4971/#review7586
-----------------------------------------------------------
I'll take a look at adding a test for this, address the issues below and update
the patch. Thanks for the feedback!
src/java/org/apache/hcatalog/common/HCatUtil.java
<https://reviews.apache.org/r/4971/#comment16773>
Good suggestion, I'll move to internal util since its not intended for
public use.
src/java/org/apache/hcatalog/mapreduce/HCatRecordReader.java
<https://reviews.apache.org/r/4971/#comment16771>
Basically to minimize the diff - will update.
src/java/org/apache/hcatalog/mapreduce/HCatRecordReader.java
<https://reviews.apache.org/r/4971/#comment16772>
I don't actually understand this whole properties loop, since its not used.
Any clue why this is here? If not, mind if I just remove it entirely?
Originally I left this alone since it was not strictly necessary for the patch.
- Travis
On 2012-05-02 17:55:29, Travis Crawford wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4971/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-02 17:55:29)
bq.
bq.
bq. Review request for hcatalog and Francis Liu.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Update ProgressReporter to work with both old and new mapreduce API. Delay
creating the base record reader so we have a StatusReporter and can use
counters.
bq.
bq.
bq. This addresses bug HCATALOG-373.
bq. https://issues.apache.org/jira/browse/HCATALOG-373
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/org/apache/hcatalog/common/HCatUtil.java cb6404a
bq. src/java/org/apache/hcatalog/mapreduce/HCatBaseInputFormat.java 268167e
bq. src/java/org/apache/hcatalog/mapreduce/HCatRecordReader.java 65f96f4
bq. src/java/org/apache/hcatalog/mapreduce/ProgressReporter.java fb379cd
bq.
bq. Diff: https://reviews.apache.org/r/4971/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. "ant clean test" passes
bq.
bq. I can run pig+hcatalog queries using Elephant-Bird deprecated API
wrappers, which is why this issue originally came up.
bq.
bq.
bq. Thanks,
bq.
bq. Travis
bq.
bq.
> ProgressReporter should work with both old and new MR API
> ---------------------------------------------------------
>
> Key: HCATALOG-373
> URL: https://issues.apache.org/jira/browse/HCATALOG-373
> Project: HCatalog
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: HCATALOG-373_progress_reporter.diff,
> HCATALOG-373_progress_reporter_2.diff, HCATALOG-373_progress_reporter_3.diff,
> HCATALOG-373_progress_reporter_4.diff
>
>
> {{org.apache.hcatalog.mapreduce.ProgressReporter}} currently implements
> {{org.apache.hadoop.mapred.Reporter}}. It should also extend
> {{org.apache.hadoop.mapreduce.StatusReporter}} so it works with code
> expecting either an old or new API reporter.
> The use case is using a wrapper so a serde works with a new-API input format.
> https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/mapred/input/DeprecatedInputFormatWrapper.java#L163
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
