[
https://issues.apache.org/jira/browse/PIG-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296938#comment-14296938
]
Remi Catherinot commented on PIG-4341:
--------------------------------------
There was another case about snappy being a valid tmp file compression codec.
It is PIG-2319. It is still unresolved because it finally depends on Hadoop
supporting Snappy in its TFile implementation. CMX have the same dependency and
TFile supports CMX. In the described mod-op, "lzo" is instanciated as being the
CMX codec class in core-site.xml. I may be mistaken but I think the pig patch
is functionnal because of this configuration trick, but since lzo is not cmx,
doing such a thing may not be such a good idea. Here is the link to the current
TFile implementation
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/file/tfile/TFile.html.
> Add CMX support to pig.tmpfilecompression.codec
> -----------------------------------------------
>
> Key: PIG-4341
> URL: https://issues.apache.org/jira/browse/PIG-4341
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Affects Versions: 0.13.0
> Reporter: fang fang chen
> Assignee: fang fang chen
> Fix For: 0.15.0
>
> Attachments: PIG-4341.patch
>
>
> Pig has supported compression(GZ, GZIP, LZO). But latest pig has not
> supported CMX codec yet. cmx is "com.ibm.biginsights.compress.CmxCodec". This
> information also could be found from latest release pig-0.13.0 documentation:
> http://pig.apache.org/docs/r0.13.0/perf.html.
> Besides, I once tested CMX codec with pig-0.13.0 before. Following was the
> current settings:
> SET pig.tmpfilecompression true;
> SET pig.tmpfilecompression.codec cmx;
> Error:
> Caused by: java.io.IOException: Invalid temporary file compression codec
> [cmx]. Expected compression codecs for org.apache.pig.impl.io.TFileStorage
> are GZ,GZIP,LZO.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
