[
https://issues.apache.org/jira/browse/LUCENE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472298#comment-17472298
]
Praveen Nishchal commented on LUCENE-8739:
------------------------------------------
Hi [~rcmuir]
That is exactly what I am doing :)
CustomCompressionCodec is inside lucene/codecs (same location as
SimpleTextCodec) and reuses Lucene90CompressingStoredFieldsFormat to improvise
storedfield compression using zstd. The idea is to power users to choose
compression algorithm and also bring their own compression algorithm via
CustomCompressionCodec. Currently it has zstd only
[https://github.com/apache/lucene/pull/439]
Zstd has overwhelmed me by being *37%* faster than Cloudflare zlib and *54%*
faster than vanilla zlib in terms of retrieved time while slightly
outperforming both in terms of compression ratio at compression level 6.
!image-2022-01-11-02-18-11-402.png|width=441,height=118!
!image-2022-01-11-02-18-57-752.png|width=448,height=44!
> ZSTD Compressor support in Lucene
> ---------------------------------
>
> Key: LUCENE-8739
> URL: https://issues.apache.org/jira/browse/LUCENE-8739
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/codecs
> Reporter: Sean Torres
> Priority: Minor
> Labels: features
> Attachments: image-2022-01-11-02-18-11-402.png,
> image-2022-01-11-02-18-57-752.png
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> ZStandard has a great speed and compression ratio tradeoff.
> ZStandard is open source compression from Facebook.
> More about ZSTD
> [https://github.com/facebook/zstd]
> [https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
