tustvold commented on PR #2278:
URL: https://github.com/apache/arrow-rs/pull/2278#issuecomment-1202996489
Running the benchmarks with `-C target-cpu=native` and the block size of
DeltaBitPackEncoder bumped up to 1024 from 128 and we get a 50% performance
improvement for 64-bit integers
```
arrow_array_reader/Int32Array/binary packed, mandatory, no NULLs
time: [23.948 us 23.964 us 23.981 us]
change: [-5.5829% -5.2823% -4.9443%] (p = 0.00 <
0.05)
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
8 (8.00%) high mild
4 (4.00%) high severe
arrow_array_reader/Int32Array/binary packed, optional, no NULLs
time: [40.763 us 40.771 us 40.781 us]
change: [-3.8441% -3.5619% -3.2927%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
arrow_array_reader/Int32Array/binary packed, optional, half NULLs
time: [48.342 us 48.366 us 48.391 us]
change: [+9.3416% +9.6807% +10.033%] (p = 0.00 <
0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) low mild
4 (4.00%) high mild
1 (1.00%) high severe
arrow_array_reader/Int64Array/binary packed, mandatory, no NULLs
time: [23.154 us 23.162 us 23.172 us]
change: [-56.875% -56.752% -56.626%] (p = 0.00 <
0.05)
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
7 (7.00%) high mild
5 (5.00%) high severe
arrow_array_reader/Int64Array/binary packed, optional, no NULLs
time: [39.882 us 39.890 us 39.898 us]
change: [-43.358% -43.288% -43.164%] (p = 0.00 <
0.05)
Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
1 (1.00%) low severe
1 (1.00%) low mild
9 (9.00%) high mild
5 (5.00%) high severe
arrow_array_reader/Int64Array/binary packed, optional, half NULLs
time: [48.638 us 48.670 us 48.704 us]
change: [-16.193% -15.929% -15.647%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
```
I wonder if we should just bump the default block size to 256 for 64-bit
integers :thinking:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
