Samarth Jain created PARQUET-1643:
-------------------------------------
Summary: Use airlift non-native implementations for GZIP, LZ0 and
LZ4 codecs
Key: PARQUET-1643
URL: https://issues.apache.org/jira/browse/PARQUET-1643
Project: Parquet
Issue Type: Improvement
Reporter: Samarth Jain
[~rdblue] pointed me to [https://github.com/airlift/aircompressor] which
provides non-native implementations of compression codecs. It claims to be much
faster than native wrappers that parquet uses. This Jira is to track the work
needed for exploring using these codecs, getting benchmark results and making
changes including not needing to pool compressors and decompressors anymore.
Note that this doesn't include SNAPPY since Parquet already has its own
non-hadoopy implementation for it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
