[ https://issues.apache.org/jira/browse/LUCENE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17443964#comment-17443964 ]
Adrien Grand commented on LUCENE-8739: -------------------------------------- I ran your PR with the new stored fields benchmark to see how codecs compare: ||Codec ||Indexing time (ms) ||Disk usage (MB) || Retrieval time per 10k docs (ms) || | BEST_SPEED | 35383 | 90.175 | 190.17524 | | BEST_COMPRESSION (vanilla zlib) | 76671 | 58.682 | 1910.42106 | | BEST_COMPRESSION (Cloudflare zlib) | 54791 | 58.601 | 1395.53593 | | ZSTD (level=1) | 42433 | 70.527 | 240.04036 | | ZSTD (level=3) | 53426 | 68.737 | 259.61897 | | ZSTD (level=6) | 100697 | 66.283 | 251.91177 | >From a quick look at your PR, it looks like you are not using dictionaries, >which would explain why we're seeing a worse compression ratio? > ZSTD Compressor support in Lucene > --------------------------------- > > Key: LUCENE-8739 > URL: https://issues.apache.org/jira/browse/LUCENE-8739 > Project: Lucene - Core > Issue Type: New Feature > Components: core/codecs > Reporter: Sean Torres > Priority: Minor > Labels: features > Time Spent: 1h 10m > Remaining Estimate: 0h > > ZStandard has a great speed and compression ratio tradeoff. > ZStandard is open source compression from Facebook. > More about ZSTD > [https://github.com/facebook/zstd] > [https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/] -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org