neilconway opened a new pull request, #25479:
URL: https://github.com/apache/datafusion/pull/25479
## Which issue does this PR close?
- N/A
## Rationale for this change
`array_resize` prepares fill/padding values for the situations in which it
needs to grow the input arrays. When (a) no fill value was provided (b) no row
needs to grow, the code allocated an all-NULL array equal to the size of the
underlying backing array of the to-be-resized input. This is redundant, because
no fill is needed; it's also potentially expensive, because the backing array
might be much larger than the actual visible portion of the input.
Benchmarks:
- Shrink 500 → 10, omitted fill: 33.79 → 10.61 µs — 3.2× faster.
- Shrink 10 → 5, omitted fill, 10,000 backing elements: 13.63 → 11.25 µs —
about 17% less time.
- Shrink 10 → 5, omitted fill, 1,000,000 backing elements: 43.36 → 11.24
µs — 3.9× faster.
Other benchmark cases were within noise.
## What changes are included in this PR?
* Refactor `build_resized_list` to make the fill data `Optional`
* When no fill data is needed, don't supply any
* Add an assert that fill data is supplied if it is needed
* Add benchmark
* Add unit tests
## What is the testing strategy for this PR?
Benchmarked to confirm performance improvement. Existing tests pass. Added
new unit tests to improve code coverage; they are not intended to catch the
allocation change though.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
