tustvold commented on pull request #1228:
URL: https://github.com/apache/arrow-rs/pull/1228#issuecomment-1019939399
In order to test how much of the performance uplift was the changes to
SlicesIterator and how much UnalignedChunk I created a branch with the changes
to SlicesIterator but using BitChunks instead of UnalignedBitChunks.
#1225
```
filter u8
time: [289.51 us 289.72 us 289.93 us]
filter u8 high selectivity
time: [5.2759 us 5.2786 us 5.2819 us]
filter u8 low selectivity
time: [3.8342 us 3.8385 us 3.8453 us]
```
Improved SlicesIterator only
```
filter u8
time: [175.13 us 175.16 us 175.20 us]
filter u8 high selectivity
time: [2.5869 us 2.5880 us 2.5892 us]
filter u8 low selectivity
time: [1.9370 us 1.9388 us 1.9407 us]
```
This PR
```
filter u8
time: [140.52 us 140.55 us 140.58 us]
filter u8 high selectivity
time: [2.5015 us 2.5023 us 2.5033 us]
filter u8 low selectivity
time: [1.7586 us 1.7591 us 1.7597 us]
```
So the UnalignedBitChunks does yield a non-negligible performance benefit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
