tustvold commented on PR #4884:
URL: https://github.com/apache/arrow-rs/pull/4884#issuecomment-1742186124
Running the benchmarks shows this does appear to regress performance
```
compress LZ4 - alphanumeric
time: [116.18 µs 116.44 µs 116.74 µs]
change: [+0.7922% +1.1707% +1.5607%] (p = 0.00 <
0.05)
Change within noise threshold.
LZ4 compressed 1048576 bytes of alphanumeric to 1052698 bytes
decompress LZ4 - alphanumeric
time: [34.815 µs 34.839 µs 34.865 µs]
change: [+20.848% +21.196% +21.530%] (p = 0.00 <
0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
7 (7.00%) high mild
2 (2.00%) high severe
compress LZ4_RAW - alphanumeric
time: [117.29 µs 117.50 µs 117.73 µs]
change: [+4.0342% +4.3202% +4.6091%] (p = 0.00 <
0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
7 (7.00%) high mild
LZ4_RAW compressed 1048576 bytes of alphanumeric to 1052690 bytes
decompress LZ4_RAW - alphanumeric
time: [33.121 µs 33.139 µs 33.159 µs]
change: [+9.0540% +9.5426% +10.041%] (p = 0.00 <
0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
5 (5.00%) high mild
5 (5.00%) high severe
Benchmarking compress LZ4 - words: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 8.0s, enable flat sampling, or reduce sample count to 50.
compress LZ4 - words time: [1.5822 ms 1.5831 ms 1.5840 ms]
change: [+14.406% +14.490% +14.573%] (p = 0.00 <
0.05)
Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
10 (10.00%) high mild
2 (2.00%) high severe
LZ4 compressed 1048576 bytes of words to 408369 bytes
decompress LZ4 - words time: [253.21 µs 253.31 µs 253.42 µs]
change: [+3.8904% +3.9696% +4.0484%] (p = 0.00 <
0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking compress LZ4_RAW - words: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 7.9s, enable flat sampling, or reduce sample count to 50.
compress LZ4_RAW - words
time: [1.5648 ms 1.5653 ms 1.5659 ms]
change: [+13.811% +13.877% +13.940%] (p = 0.00 <
0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
7 (7.00%) high mild
1 (1.00%) high severe
LZ4_RAW compressed 1048576 bytes of words to 408361 bytes
decompress LZ4_RAW - words
time: [253.63 µs 253.73 µs 253.84 µs]
change: [+3.8637% +3.9363% +4.0044%] (p = 0.00 <
0.05)
Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low severe
5 (5.00%) high mild
2 (2.00%) high severe
```
In particular we see regressions
* ~10-20% when decompressing non-compressible input
* ~15% when compressing compressible input
These benchmarks represent two pretty extreme cases, and it is likely that
most realistic workloads sit somewhere inbetween and would see fairly minor
regressions to both decompression and compression.
This is consistent with lz4_flex's [own
benchmarking](https://github.com/PSeitz/lz4_flex#results-v0110-02-06-2023-safe-decode-and-safe-encode-off)
which shows lz4_flex tending to perform better than lz4 at one of either
compression or decompression for a given corpus.
I personally am happy enough with the performance to not feel the need to
deal with the complexity of maintaining two possible implementations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
