siying opened a new pull request, #43338: URL: https://github.com/apache/spark/pull/43338
### What changes were proposed in this pull request? Turn to use LZ4 in RocksDB state store. ### Why are the changes needed? LZ4 is generally faster than Snappy. That's probably we use LZ4 in changelogs and other places by default. However, we don't change RocksDB's default of Snappy compression style. The RocksDB Team recommend LZ4 or ZSTD and the default is kept to Snappy only for backward compatible reason. We should use LZ4 instead. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests should fix it. Some benchmarks are run and we see positive improvements. ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
