Oh your question is regarding compression codec for temporary files?.
I am not sure about that.
On Tue, Nov 18, 2014 at 4:13 PM, Krishna Kalyan <krishnakaly...@gmail.com>
wrote:
> Hi lulynn,
> If you are using biginsights follow the steps to enable compression.
>
> set output.compression.enabled true;
> set output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> data = LOAD
> '/bigdata/sample_data/NOAA_Weather_csv/2011/999999-53019-2011.csv.gz'
> using PigStorage(',') as (projectname:chararray);
> STORE data INTO '/comCodecGzip';
>
> Cheers,
> Krishna
>
> On Tue, Nov 18, 2014 at 2:04 PM, lulynn_2008 <lulynn_2...@163.com> wrote:
>
>> BTW, cmx is "com.ibm.biginsights.compress.CmxCodec", the related jar is
>> ibm-compression.jar.
>>
>>
>>
>> At 2014-11-18 15:49:53, "lulynn_2008" <lulynn_2...@163.com> wrote:
>>
>> Hi All,
>> I am trying to use CMX as temp file compression
>>
>> codec, i.e
>> SET pig.tmpfilecompression true;
>> SET pig.tmpfilecompression.codec cmx;
>>
>> but following errors happened:
>> Caused by: java.io.IOException: Invalid temporary file compression codec
>> []. Expected compression codecs are gz and lzo
>>
>> from pig cookbook I found following line
>>
>> "pig.tmpfilecompression.codec - Specifies which compression codec to use.
>> Currently, Pig accepts "gz" and "lzo" as possible values. However, because
>> LZO is under GPL license (and disabled by default) you will need to
>> configure your cluster to use the LZO codec to take advantage of this
>> feature. For details, see
>> http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ. "
>>
>>
>> Is there any workaround? or there are roadmaps for adding cmx as a
>>
>> supported codec ? I was using pig 0.12.0.
>>
>> Thanks
>>
>>
>>
>>
>
