dferstay opened a new pull request #690: URL: https://github.com/apache/pulsar-client-go/pull/690
### Motivation This updates zstd to 1.5.0 which supports some major performance improvements. For details see: https://github.com/facebook/zstd/releases/tag/v1.5.0 Results from `pulsar/internal/compression/compression_bench_test.go` show the speedup. ``` name old time/op new time/op delta CompressionParallel/zstd-cgo-level-fastest 1.30ms ±13% 0.95ms ± 0% -27.09% (p=0.000 n=9+8) CompressionParallel/zstd-cgo-level-fastest-2 672µs ± 2% 517µs ± 4% -23.12% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-fastest-4 421µs ± 6% 323µs ± 3% -23.10% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-fastest-8 241µs ± 1% 187µs ± 1% -22.25% (p=0.000 n=8+8) CompressionParallel/zstd-cgo-level-default 1.34ms ±15% 1.00ms ± 2% -25.02% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-default-2 676µs ± 1% 525µs ± 1% -22.30% (p=0.000 n=10+9) CompressionParallel/zstd-cgo-level-default-4 389µs ± 1% 317µs ± 5% -18.39% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-default-8 245µs ±11% 210µs ± 2% -14.36% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-best 1.29ms ± 2% 0.97ms ± 0% -24.63% (p=0.000 n=8+8) CompressionParallel/zstd-cgo-level-best-2 684µs ± 1% 528µs ± 2% -22.75% (p=0.000 n=8+10) CompressionParallel/zstd-cgo-level-best-4 434µs ± 4% 323µs ± 2% -25.64% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-best-8 260µs ± 7% 211µs ± 1% -19.05% (p=0.000 n=10+9) name old speed new speed delta CompressionParallel/zstd-cgo-level-fastest 77.4MB/s ±12% 105.9MB/s ± 0% +36.85% (p=0.000 n=9+8) CompressionParallel/zstd-cgo-level-fastest-2 150MB/s ± 2% 195MB/s ± 4% +30.11% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-fastest-4 240MB/s ± 6% 311MB/s ± 3% +29.93% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-fastest-8 418MB/s ± 1% 537MB/s ± 1% +28.62% (p=0.000 n=8+8) CompressionParallel/zstd-cgo-level-default 75.6MB/s ±14% 100.4MB/s ± 2% +32.82% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-default-2 149MB/s ± 1% 191MB/s ± 1% +28.71% (p=0.000 n=10+9) CompressionParallel/zstd-cgo-level-default-4 259MB/s ± 1% 318MB/s ± 5% +22.66% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-default-8 411MB/s ±10% 479MB/s ± 2% +16.39% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-best 77.8MB/s ± 2% 103.2MB/s ± 0% +32.66% (p=0.000 n=8+8) CompressionParallel/zstd-cgo-level-best-2 147MB/s ± 1% 190MB/s ± 2% +29.47% (p=0.000 n=8+10) CompressionParallel/zstd-cgo-level-best-4 232MB/s ± 4% 312MB/s ± 2% +34.44% (p=0.000 n=10+10) CompressionParallel/zstd-cgo-level-best-8 388MB/s ± 8% 478MB/s ± 1% +23.27% (p=0.000 n=10+9) ``` ``` name old time/op new time/op delta Compression/zstd-cgo-level-fastest 369µs ± 0% 371µs ± 0% +0.65% (p=0.000 n=8+9) Compression/zstd-cgo-level-fastest-2 369µs ± 0% 370µs ± 1% ~ (p=0.796 n=9+9) Compression/zstd-cgo-level-fastest-4 369µs ± 0% 373µs ± 3% ~ (p=0.101 n=8+10) Compression/zstd-cgo-level-fastest-8 371µs ± 2% 374µs ± 2% ~ (p=0.053 n=9+10) Compression/zstd-cgo-level-default 1.28ms ± 0% 0.96ms ± 2% -25.24% (p=0.000 n=9+10) Compression/zstd-cgo-level-default-2 1.29ms ± 2% 0.96ms ± 2% -26.05% (p=0.000 n=10+10) Compression/zstd-cgo-level-default-4 1.30ms ± 2% 0.96ms ± 1% -25.88% (p=0.000 n=10+10) Compression/zstd-cgo-level-default-8 1.28ms ± 3% 0.96ms ± 1% -24.63% (p=0.000 n=10+10) Compression/zstd-cgo-level-best 3.90ms ± 0% 2.32ms ± 1% -40.39% (p=0.000 n=10+9) Compression/zstd-cgo-level-best-2 3.90ms ± 0% 2.36ms ± 0% -39.63% (p=0.000 n=9+8) Compression/zstd-cgo-level-best-4 3.94ms ± 2% 2.33ms ± 1% -40.88% (p=0.000 n=10+10) Compression/zstd-cgo-level-best-8 3.97ms ± 2% 2.36ms ± 0% -40.59% (p=0.000 n=10+9) name old speed new speed delta Compression/zstd-cgo-level-fastest 273MB/s ± 0% 271MB/s ± 0% -0.64% (p=0.000 n=8+9) Compression/zstd-cgo-level-fastest-2 272MB/s ± 0% 272MB/s ± 1% ~ (p=0.779 n=9+9) Compression/zstd-cgo-level-fastest-4 272MB/s ± 0% 270MB/s ± 3% ~ (p=0.101 n=8+10) Compression/zstd-cgo-level-fastest-8 271MB/s ± 2% 269MB/s ± 2% ~ (p=0.053 n=9+10) Compression/zstd-cgo-level-default 78.7MB/s ± 0% 105.2MB/s ± 2% +33.77% (p=0.000 n=9+10) Compression/zstd-cgo-level-default-2 77.8MB/s ± 2% 105.1MB/s ± 2% +35.21% (p=0.000 n=10+10) Compression/zstd-cgo-level-default-4 77.7MB/s ± 2% 104.8MB/s ± 1% +34.91% (p=0.000 n=10+10) Compression/zstd-cgo-level-default-8 78.9MB/s ± 3% 104.7MB/s ± 1% +32.66% (p=0.000 n=10+10) Compression/zstd-cgo-level-best 25.8MB/s ± 0% 43.3MB/s ± 1% +67.77% (p=0.000 n=10+9) Compression/zstd-cgo-level-best-2 25.8MB/s ± 0% 42.7MB/s ± 0% +65.63% (p=0.000 n=9+8) Compression/zstd-cgo-level-best-4 25.5MB/s ± 2% 43.1MB/s ± 1% +69.12% (p=0.000 n=10+10) Compression/zstd-cgo-level-best-8 25.4MB/s ± 2% 42.7MB/s ± 0% +68.30% (p=0.000 n=10+9) ``` ``` name old time/op new time/op delta Decompression/zstd-cgo-level-fastest 121µs ± 1% 111µs ±20% ~ (p=0.133 n=9+10) Decompression/zstd-cgo-level-fastest-2 119µs ±10% 98µs ± 6% -17.77% (p=0.000 n=9+10) Decompression/zstd-cgo-level-fastest-4 108µs ± 2% 96µs ± 6% -10.93% (p=0.000 n=10+10) Decompression/zstd-cgo-level-fastest-8 113µs ± 5% 103µs ± 4% -9.03% (p=0.000 n=10+10) Decompression/zstd-cgo-level-default 155µs ± 7% 135µs ± 4% -12.99% (p=0.000 n=10+10) Decompression/zstd-cgo-level-default-2 203µs ±51% 119µs ± 0% -41.63% (p=0.000 n=10+9) Decompression/zstd-cgo-level-default-4 138µs ± 2% 122µs ± 2% -11.63% (p=0.000 n=10+9) Decompression/zstd-cgo-level-default-8 143µs ± 1% 127µs ± 2% -11.16% (p=0.000 n=10+9) Decompression/zstd-cgo-level-best 137µs ± 1% 125µs ± 0% -8.98% (p=0.000 n=9+8) Decompression/zstd-cgo-level-best-2 139µs ±15% 122µs ±13% -12.42% (p=0.034 n=8+10) Decompression/zstd-cgo-level-best-4 124µs ± 1% 110µs ± 2% -11.14% (p=0.000 n=9+10) Decompression/zstd-cgo-level-best-8 129µs ± 3% 114µs ± 1% -11.17% (p=0.000 n=10+8) name old speed new speed delta Decompression/zstd-cgo-level-fastest 832MB/s ± 1% 914MB/s ±17% ~ (p=0.133 n=9+10) Decompression/zstd-cgo-level-fastest-2 848MB/s ±11% 1029MB/s ± 6% +21.29% (p=0.000 n=9+10) Decompression/zstd-cgo-level-fastest-4 930MB/s ± 2% 1044MB/s ± 6% +12.36% (p=0.000 n=10+10) Decompression/zstd-cgo-level-fastest-8 889MB/s ± 4% 977MB/s ± 4% +9.91% (p=0.000 n=10+10) Decompression/zstd-cgo-level-default 652MB/s ± 7% 748MB/s ± 5% +14.83% (p=0.000 n=10+10) Decompression/zstd-cgo-level-default-2 557MB/s ±41% 849MB/s ± 0% +52.42% (p=0.000 n=10+9) Decompression/zstd-cgo-level-default-4 730MB/s ± 2% 826MB/s ± 2% +13.16% (p=0.000 n=10+9) Decompression/zstd-cgo-level-default-8 705MB/s ± 1% 794MB/s ± 2% +12.56% (p=0.000 n=10+9) Decompression/zstd-cgo-level-best 734MB/s ± 1% 806MB/s ± 0% +9.87% (p=0.000 n=9+8) Decompression/zstd-cgo-level-best-2 660MB/s ±45% 815MB/s ±12% +23.45% (p=0.017 n=10+9) Decompression/zstd-cgo-level-best-4 811MB/s ± 1% 912MB/s ± 2% +12.54% (p=0.000 n=9+10) Decompression/zstd-cgo-level-best-8 782MB/s ± 3% 880MB/s ± 1% +12.56% (p=0.000 n=10+8) ``` ### Modifications The version of DataDog/zstd has been updated to tag v1.5.0 ### Verifying this change This change is already covered by existing tests, such as:. - pulsar/internal/compression/compression_test.go - pulsar/internal/compression/compression_bench_test.go ### Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): Yes, it moves to a newer version of `github.com/DataDog/zstd` - The public API: No - The schema: No - The default values of configurations: No - The wire protocol: No ### Documentation - Does this pull request introduce a new feature? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
