[
https://issues.apache.org/jira/browse/PIG-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635619#action_12635619
]
Arun C Murthy commented on PIG-467:
-----------------------------------
It definitely is better to use lzo rather than zlib for compressing
intermediate map-outputs. However, lzo might not be available on the cluster,
and the right 32/64 bit lzo libraries need to installed. Hence, it would be
pertinent to default to zlib but have an easy way for the user or admin to
change it to lzo.
> compressing intermediate results
> --------------------------------
>
> Key: PIG-467
> URL: https://issues.apache.org/jira/browse/PIG-467
> Project: Pig
> Issue Type: Improvement
> Affects Versions: types_branch
> Reporter: Olga Natkovich
> Fix For: types_branch
>
>
> It is recommended with Hadoop 18 and later versions to compress data passed
> between Map and Reduce. We need to test to make sure that it gives
> performance gain
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
