[
https://issues.apache.org/jira/browse/HIVE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-3234:
------------------------------
Attachment: HIVE-3234.D6699.1.patch
omalley requested code review of "HIVE-3234 [jira] getting the reporter in the
recordwriter".
Reviewers: JIRA
hive-3234. pass reporter down to outputfileformat
We would like to generate some custom statistics and report back to
map/reduce later wen implement the
FileSinkOperator.RecordWriter interface. However, the current interface
design doesn't allow us to get the map reduce reporter object. Please extend
the current FileSinkOperator.RecordWriter interface so that it's close() method
passes in a map reduce reporter object.
For the same reason, please also extend the RecordReader interface too to
include a reporter object so that users can passes in custom map reduce
counters.
TEST PLAN
Added a unit test that uses a custom input output format, which
ensures the Progressable is a non-null Reporter.
REVISION DETAIL
https://reviews.facebook.net/D6699
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/RowContainer.java
ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java
ql/src/test/org/apache/hadoop/hive/ql/io/udf/Rot13InputFormat.java
ql/src/test/org/apache/hadoop/hive/ql/io/udf/Rot13OutputFormat.java
ql/src/test/queries/clientpositive/custom_input_output_format.q
ql/src/test/results/clientpositive/custom_input_output_format.q.out
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/15939/
To: JIRA, omalley
> getting the reporter in the recordwriter
> ----------------------------------------
>
> Key: HIVE-3234
> URL: https://issues.apache.org/jira/browse/HIVE-3234
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
> Affects Versions: 0.9.1
> Environment: any
> Reporter: Jimmy Hu
> Assignee: Owen O'Malley
> Labels: newbie
> Fix For: 0.9.1
>
> Attachments: HIVE-3234.D6699.1.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> We would like to generate some custom statistics and report back to
> map/reduce later wen implement the
> FileSinkOperator.RecordWriter interface. However, the current interface
> design doesn't allow us to get the map reduce reporter object. Please extend
> the current FileSinkOperator.RecordWriter interface so that it's close()
> method passes in a map reduce reporter object.
> For the same reason, please also extend the RecordReader interface too to
> include a reporter object so that users can passes in custom map reduce
> counters.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
