jorgecarleitao commented on pull request #8997:
URL: https://github.com/apache/arrow/pull/8997#issuecomment-751466829
@alamb , thanks a lot for taking the time to review this. This one is
challenging to review.
wrt to the `vec<u8>`, my current hypothesis is that the alignment does
improve performance when SIMD is on (as the benches show in that PR). As such,
my current direction is to get the best of both worlds: migrate the relevant
parts of `std::rawbytes::RawBytes` and `std::alloc` to `memory.rs` (as that
code is currently `unstable`) to still allocate aligned, but use the well
studied code from rust's std. This is also related to #9016 .
Basically, we have a performance problem in the reallocation code. The
following is the result of 4 runs:
```
mutable time: [929.26 us 931.88 us 935.42 us]
mutable prepared time: [1.0682 ms 1.0693 ms 1.0709 ms]
from_slice time: [4.4857 ms 4.5043 ms 4.5247 ms]
from_slice prepared time: [1.4358 ms 1.4406 ms 1.4467 ms]
```
1. start with an empty mutable and grow it.
2. start with a mutable with the correct capacity and grows (i.e. extend
without re-allocation).
3. do the same with a `vec<u8>` and at the end of all use `Buffer::from`
4. same as 2 and at the end of all use `Buffer::from`
The fact that there is no difference between 1 and 2 but a 3.5x difference
between 3 and 4 shows that we are doing something wrong.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
