-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4971/
-----------------------------------------------------------
(Updated 2012-05-05 01:23:50.756124)
Review request for hcatalog and Francis Liu.
Changes
-------
Patch updated to include a test that counters exist when input has been read.
Initially I went to add a test specific to this functionality but there was
significant overlap with what already exists, so I updated an existing test to
check that the bytes read counter is set when data has actually been read. This
seems like the best approach as the ongoing maintenance will be low. The check
includes a comment about why the check exists to help future developers.
Suggestions from the previous review have also been incorporated. Specifically,
we reuse hcatSplit instead of recasting. As the whole setting properties loop
is not used I removed that section.
Summary
-------
Update ProgressReporter to work with both old and new mapreduce API. Delay
creating the base record reader so we have a StatusReporter and can use
counters.
This addresses bug HCATALOG-373.
https://issues.apache.org/jira/browse/HCATALOG-373
Diffs (updated)
-----
src/java/org/apache/hcatalog/mapreduce/HCatBaseInputFormat.java 268167e
src/java/org/apache/hcatalog/mapreduce/HCatRecordReader.java 65f96f4
src/java/org/apache/hcatalog/mapreduce/InternalUtil.java 1837081
src/java/org/apache/hcatalog/mapreduce/ProgressReporter.java fb379cd
src/test/org/apache/hcatalog/mapreduce/HCatMapReduceTest.java f3d07a0
Diff: https://reviews.apache.org/r/4971/diff
Testing
-------
"ant clean test" passes
I can run pig+hcatalog queries using Elephant-Bird deprecated API wrappers,
which is why this issue originally came up.
Thanks,
Travis
