[
https://issues.apache.org/jira/browse/HCATALOG-373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273727#comment-13273727
]
[email protected] commented on HCATALOG-373:
--------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4971/#review7816
-----------------------------------------------------------
Looks good to me. Just a concern with the change in the test output directory.
If you agree with any of my proposals, I can change that on commit. Will run
e2e tests.
src/test/org/apache/hcatalog/mapreduce/HCatMapReduceTest.java
<https://reviews.apache.org/r/4971/#comment17147>
The warehouse directory is set in the ant test target. If you're doing this
for runs in your IDE then it'd be better to have this check if the sysproperty
hive.metastore.warehouse.dir is set to support both cases.
It'll probably better if you put the dir in /tmp similar to ant the config
"/tmp/hcat_junit_warehouse". Or add ant into the name {user.home}/hive_build/...
- Francis
On 2012-05-05 01:23:50, Travis Crawford wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4971/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-05 01:23:50)
bq.
bq.
bq. Review request for hcatalog and Francis Liu.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Update ProgressReporter to work with both old and new mapreduce API. Delay
creating the base record reader so we have a StatusReporter and can use
counters.
bq.
bq.
bq. This addresses bug HCATALOG-373.
bq. https://issues.apache.org/jira/browse/HCATALOG-373
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/org/apache/hcatalog/mapreduce/HCatBaseInputFormat.java 268167e
bq. src/java/org/apache/hcatalog/mapreduce/HCatRecordReader.java 65f96f4
bq. src/java/org/apache/hcatalog/mapreduce/InternalUtil.java 1837081
bq. src/java/org/apache/hcatalog/mapreduce/ProgressReporter.java fb379cd
bq. src/test/org/apache/hcatalog/mapreduce/HCatMapReduceTest.java f3d07a0
bq.
bq. Diff: https://reviews.apache.org/r/4971/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. "ant clean test" passes
bq.
bq. I can run pig+hcatalog queries using Elephant-Bird deprecated API
wrappers, which is why this issue originally came up.
bq.
bq.
bq. Thanks,
bq.
bq. Travis
bq.
bq.
> ProgressReporter should work with both old and new MR API
> ---------------------------------------------------------
>
> Key: HCATALOG-373
> URL: https://issues.apache.org/jira/browse/HCATALOG-373
> Project: HCatalog
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: HCATALOG-373_progress_reporter.diff,
> HCATALOG-373_progress_reporter_2.diff, HCATALOG-373_progress_reporter_3.diff,
> HCATALOG-373_progress_reporter_4.diff
>
>
> {{org.apache.hcatalog.mapreduce.ProgressReporter}} currently implements
> {{org.apache.hadoop.mapred.Reporter}}. It should also extend
> {{org.apache.hadoop.mapreduce.StatusReporter}} so it works with code
> expecting either an old or new API reporter.
> The use case is using a wrapper so a serde works with a new-API input format.
> https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/mapred/input/DeprecatedInputFormatWrapper.java#L163
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
