[
https://issues.apache.org/jira/browse/HBASE-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833678#action_12833678
]
stack commented on HBASE-2225:
------------------------------
.bq We detect compression mode of the table first. If the table is compressed,
we don't apply GzipCodec. Otherwise we apply GzipCodec unless
no_compression_export is specified.
Isn't the fact that the table is compressed orthogonal to whether or not the
export should be compressed?
I'd say, no compression should be the default. Thats how its been working up
to this.
I'm good with the compression being gzip since as Lars and Ted say, its native
to sequencefiles.
> Enable compression in HBase Export
> ----------------------------------
>
> Key: HBASE-2225
> URL: https://issues.apache.org/jira/browse/HBASE-2225
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.20.1
> Environment: OS agnostic
> Reporter: Ted Yu
> Priority: Minor
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> org.apache.hadoop.hbase.mapreduce.Export should set compression codec
> In createSubmittableJob(), the following should be added:
> FileOutputFormat.setCompressOutput(job, true);
> FileOutputFormat.setOutputCompressorClass(job,
> org.apache.hadoop.io.compress.GzipCodec.class);
> From my experiment, 10% to 50% reduction in Export output has been observed.
> SequenceFileInputFormat used by the Import tool is able to detect GzipCodec -
> there is no change for Import class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
