[
https://issues.apache.org/jira/browse/LUCENE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486614#comment-17486614
]
Praveen Nishchal commented on LUCENE-8739:
------------------------------------------
As we observed earlier, Zstd is at par with vanilla/Cloudflare zlib in terms of
compression ratio but at the same time, there is a significant gain in
retrieval time. I have made the default compression level as 6 (though it is a
configurable parameter), with 48KB block size and 8KB dictionary. Any
additional comments?
This solution is part of custom codec and will allow users to use ZSTD on their
data. However, we can revisit the idea of adding it to Lucene core in the
future when Project Panama lands.
> ZSTD Compressor support in Lucene
> ---------------------------------
>
> Key: LUCENE-8739
> URL: https://issues.apache.org/jira/browse/LUCENE-8739
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/codecs
> Reporter: Sean Torres
> Priority: Minor
> Labels: features
> Attachments: image-2022-01-11-02-18-11-402.png,
> image-2022-01-11-02-18-57-752.png
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> ZStandard has a great speed and compression ratio tradeoff.
> ZStandard is open source compression from Facebook.
> More about ZSTD
> [https://github.com/facebook/zstd]
> [https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
