RexXiong commented on code in PR #3555:
URL: https://github.com/apache/celeborn/pull/3555#discussion_r2872371683
##########
client/benchmarks/LZ4TPCDSDataBenchmark-jdk17-results.txt:
##########
@@ -6,48 +6,48 @@ OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux
6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Compression: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Compression with chunk size 65536 4 times 2131 2134
5 0.0 532707902.0 1.0X
+Compression with chunk size 65536 4 times 2379 2380
2 0.0 594723682.5 1.0X
OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Decompression: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
-Decompression with chunk size 65536 4 times 536 541
9 0.0 133951799.0 1.0X
+Decompression with chunk size 65536 4 times 724 728
3 0.0 181120209.3 1.0X
Review Comment:
Seems LZ4 has significantly decreased,As a shuffle service, IMO It's
unacceptable, Currently, we are also using ZSTD in production with good
results. I believe it is now time to change the default from LZ4 to ZSTD. This
can also help prevent performance degradation issues for community users during
future upgrades. WDYT @pan3793 @SteNicholas ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
