Dandandan opened a new issue, #9530:
URL: https://github.com/apache/arrow-rs/issues/9530
**Is your feature request related to a problem or challenge? Please describe
what you are trying to do.**
It seems much faster than `snap` in my testing on compressing, even on e2e
benchmarks (clickbench) it is up to 2x as fast:
```
arrow_reader_clickbench/async_object_store/Q1
time: [906.71 µs 910.64 µs 914.99 µs]
change: [−3.1264% −1.9584% −0.9019%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
7 (7.00%) high mild
3 (3.00%) high severe
arrow_reader_clickbench/async_object_store/Q10
time: [5.4424 ms 5.4692 ms 5.4999 ms]
change: [−2.5400% −1.0993% +0.0171%] (p = 0.09 >
0.05)
No change in performance detected.
Found 16 outliers among 100 measurements (16.00%)
3 (3.00%) high mild
13 (13.00%) high severe
arrow_reader_clickbench/async_object_store/Q11
time: [6.1418 ms 6.1514 ms 6.1619 ms]
change: [−0.4752% +0.1639% +0.7037%] (p = 0.60 >
0.05)
No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
7 (7.00%) high mild
3 (3.00%) high severe
arrow_reader_clickbench/async_object_store/Q12
time: [11.065 ms 11.089 ms 11.125 ms]
change: [−17.577% −17.329% −17.043%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
arrow_reader_clickbench/async_object_store/Q13
time: [13.540 ms 13.578 ms 13.620 ms]
change: [−13.550% −13.282% −12.955%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
arrow_reader_clickbench/async_object_store/Q14
time: [12.368 ms 12.481 ms 12.643 ms]
change: [−16.844% −16.059% −14.979%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
arrow_reader_clickbench/async_object_store/Q19
time: [2.3651 ms 2.3716 ms 2.3784 ms]
change: [−0.2989% +0.4660% +1.1652%] (p = 0.23 >
0.05)
No change in performance detected.
arrow_reader_clickbench/async_object_store/Q20
time: [46.271 ms 46.409 ms 46.592 ms]
change: [−37.543% −37.315% −37.054%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
Benchmarking arrow_reader_clickbench/async_object_store/Q21: Warming up for
3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.3s, or reduce sample count to 90.
arrow_reader_clickbench/async_object_store/Q21
time: [53.187 ms 53.314 ms 53.444 ms]
change: [−35.829% −35.618% −35.418%] (p = 0.00 <
0.05)
Performance has improved.
Benchmarking arrow_reader_clickbench/async_object_store/Q22: Warming up for
3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 6.7s, or reduce sample count to 70.
arrow_reader_clickbench/async_object_store/Q22
time: [67.068 ms 67.386 ms 67.793 ms]
change: [−32.873% −32.418% −31.971%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
Benchmarking arrow_reader_clickbench/async_object_store/Q23: Warming up for
3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 15.4s, or reduce sample count to 30.
arrow_reader_clickbench/async_object_store/Q23
time: [147.95 ms 148.29 ms 148.67 ms]
change: [−33.220% −33.030% −32.818%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
arrow_reader_clickbench/async_object_store/Q24
time: [15.013 ms 15.030 ms 15.048 ms]
change: [−16.445% −16.294% −16.150%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
5 (5.00%) high mild
2 (2.00%) high severe
arrow_reader_clickbench/async_object_store/Q27
time: [32.775 ms 32.842 ms 32.922 ms]
change: [−46.691% −46.261% −45.930%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
arrow_reader_clickbench/async_object_store/Q28
time: [35.060 ms 35.184 ms 35.342 ms]
change: [−42.107% −41.869% −41.569%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe
arrow_reader_clickbench/async_object_store/Q30
time: [14.191 ms 14.216 ms 14.245 ms]
change: [−15.213% −15.022% −14.818%] (p = 0.00 <
0.05)
Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
```
**Describe the solution you'd like**
Add it
**Describe alternatives you've considered**
<!--
A clear and concise description of any alternative solutions or features
you've considered.
-->
**Additional context**
<!--
Add any other context or screenshots about the feature request here.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
