[
https://issues.apache.org/jira/browse/SQOOP-428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192591#comment-13192591
]
[email protected] commented on SQOOP-428:
-----------------------------------------------------
bq. On 2012-01-24 21:50:05, Tom White wrote:
bq. > Looks good. Have you run manual tests with it too?
Yes, I've used it and manually checked the result (only with Snappy though) and
the result is correct (that's when we stumbled upon SQOOP-429)
bq. On 2012-01-24 21:50:05, Tom White wrote:
bq. > src/test/com/cloudera/sqoop/TestAvroImport.java, line 89
bq. > <https://reviews.apache.org/r/3600/diff/1/?file=70555#file70555line89>
bq. >
bq. > You should check that the files that are written are compressed (by
looking at DataFileReader's metadata).
bq. >
bq. > We also need a test for --compress.
Thanks for the hint. I'll look into it and update the review.
- Lars
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3600/#review4566
-----------------------------------------------------------
On 2012-01-24 14:07:58, Lars Francke wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3600/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-01-24 14:07:58)
bq.
bq.
bq. Review request for Sqoop.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This basically only ports all the code from Avro's (1.5.4)
AvroOutputFormat to the new MR API.
bq.
bq. I've changed the test to extract the common functionality into a helper
method because they are the same apart from the two command line arguments.
bq.
bq. I could have deleted AvroJob completely but as I was told last time that
binary compatibility needs to be maintained I left it in. It's not needed
anymore as all necessary functionality can be gotten from Avro's own version of
that file as far as I can tell. So if it's okay to delete that redundant file
(two actually, cloudera and apache package) let me know and I'll provide a new
patch.
bq.
bq.
bq. This addresses bug SQOOP-428.
bq. https://issues.apache.org/jira/browse/SQOOP-428
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/org/apache/sqoop/mapreduce/AvroJob.java a57aaf1
bq. src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java 96befd7
bq. src/test/com/cloudera/sqoop/TestAvroImport.java 1b8b046
bq.
bq. Diff: https://reviews.apache.org/r/3600/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. All tests pass for hadoopversion=20 but TestColumnTypes fails for me on
23. I can't see how that's related though.
bq.
bq.
bq. Thanks,
bq.
bq. Lars
bq.
bq.
> AvroOutputFormat doesn't support compression even though documentation claims
> it does
> -------------------------------------------------------------------------------------
>
> Key: SQOOP-428
> URL: https://issues.apache.org/jira/browse/SQOOP-428
> Project: Sqoop
> Issue Type: Bug
> Components: docs
> Affects Versions: 1.4.0-incubating
> Reporter: Lars Francke
> Priority: Minor
> Labels: avro, document
> Attachments: SQOOP-428.1.patch
>
>
> The documentation claims that Avro files can be compressed as well:
> {quote}
> By default, data is not compressed. You can compress your data by using the
> deflate (gzip) algorithm with the -z or --compress argument, or specify any
> Hadoop compression codec using the --compression-codec argument. This applies
> to SequenceFile, text, and Avro files.
> {quote}
> This is not true as the AvroOutputFormat currently doesn't support
> compression.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
