tustvold commented on pull request #1082:
URL: https://github.com/apache/arrow-rs/pull/1082#issuecomment-999707518
> would you be able to share some performance benchmark results?
They're very preliminary at this stage, I'm not totally confident this code
is correct nor have I spent any time trying to optimise it, but here you go. My
primary focus has been proving out the interface from #1041, not polishing up
the specific optimisations yet.
"Old" is the new ByteArrayReader, "new" is the StringArrayReader
```
arrow_array_reader/read StringArray, plain encoded, mandatory, no NULLs -
old
time: [110.98 us 111.00 us 111.03 us]
arrow_array_reader/read StringArray, plain encoded, mandatory, no NULLs -
new
time: [124.77 us 124.99 us 125.32 us]
arrow_array_reader/read StringArray, plain encoded, optional, no NULLs - old
time: [122.15 us 122.17 us 122.20 us]
arrow_array_reader/read StringArray, plain encoded, optional, no NULLs - new
time: [136.72 us 136.76 us 136.81 us]
arrow_array_reader/read StringArray, plain encoded, optional, half NULLs -
old
time: [117.26 us 117.35 us 117.43 us]
arrow_array_reader/read StringArray, plain encoded, optional, half NULLs -
new
time: [258.05 us 258.17 us 258.30 us]
arrow_array_reader/read StringArray, dictionary encoded, mandatory, no NULLs
- old
time: [145.30 us 145.35 us 145.41 us]
arrow_array_reader/read StringArray, dictionary encoded, mandatory, no NULLs
- new
time: [117.14 us 117.18 us 117.22 us]
arrow_array_reader/read StringArray, dictionary encoded, optional, no NULLs
- old
time: [159.07 us 159.11 us 159.15 us]
arrow_array_reader/read StringArray, dictionary encoded, optional, no NULLs
- new
time: [134.39 us 134.41 us 134.43 us]
arrow_array_reader/read StringArray, dictionary encoded, optional, half
NULLs - old
time: [108.28 us 108.30 us 108.33 us]
arrow_array_reader/read StringArray, dictionary encoded, optional, half
NULLs - new
time: [230.15 us 230.23 us 230.32 us]
```
Aside from dictionary encoded columns with no nulls, it performs better.
This is probably just something suboptimal in the way I'm decoding RLE data,
and should be rectifiable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
