[
https://issues.apache.org/jira/browse/LUCENE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470506#comment-17470506
]
Praveen Nishchal commented on LUCENE-8739:
------------------------------------------
WOW! That's a lot of wonderful feedback here :)
I started working on this to provide Lucene users an option to use Zstandard
for compression/decompression but this seems to be turning out really well! I
am encouraged by the data Adrien has put here and Zstandard with dictionary,
and at level 6 it seems to outperform zlib in terms of compression ratio.
I have updated PR to reflect 48KB block size with suggested code change.
Custom Codec is so designed that we can introduce any compression level and any
block size. Different use cases may involve changing compression level for
either better compression ratio or compression speed. It is extensible as well
to provide a new compression algorithm or a different zstd flavor.
> ZSTD Compressor support in Lucene
> ---------------------------------
>
> Key: LUCENE-8739
> URL: https://issues.apache.org/jira/browse/LUCENE-8739
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/codecs
> Reporter: Sean Torres
> Priority: Minor
> Labels: features
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> ZStandard has a great speed and compression ratio tradeoff.
> ZStandard is open source compression from Facebook.
> More about ZSTD
> [https://github.com/facebook/zstd]
> [https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
