[
https://issues.apache.org/jira/browse/PARQUET-678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860338#comment-15860338
]
Cotton Seed commented on PARQUET-678:
-------------------------------------
We find lz4 gives similar compression and is about 20% faster for our
application. In addition to zstd, I'm sure there is interest in other new
compression algorithms, like brotli. It would seem natural for Parquet to work
with any Hadoop compression codec. I can work up a patch if there would be
interest in accepting it.
> Allow for custom compression codecs
> -----------------------------------
>
> Key: PARQUET-678
> URL: https://issues.apache.org/jira/browse/PARQUET-678
> Project: Parquet
> Issue Type: Wish
> Reporter: Steven Anton
> Priority: Minor
>
> I understand that the list of accepted compression codecs is explicity
> limited to uncompressed, snappy, gzip, and lzo. (See
> parquet.hadoop.metadata.CompressionCodecName.java) Is there a reason for
> this? Or is there an easy workaround? On the surface it seems like an
> unnecessary restriction.
> I ask because I have written a custom codec to implement encryption and I'm
> unable to use it with Parquet, which is a real shame because it is the main
> storage format I was hoping to use.
> Other thoughts on how to implement encryption in Parquet with this limitation?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
