r3stl355 opened a new pull request, #9112:
URL: https://github.com/apache/arrow-datafusion/pull/9112
## Which issue does this PR close?
#9089. It may not close it but could be a stepping stone towards more
optimal version.
## Rationale for this change
The change shows significant bench performance improvement
- Baseline/before
```
make_date_col_col_col_1000
time: [12.829 µs 12.866 µs 12.907 µs]
Found 12 outliers among 100 measurements (12.00%)
4 (4.00%) low mild
3 (3.00%) high mild
5 (5.00%) high severe
make_date_scalar_col_col_1000
time: [11.800 µs 11.830 µs 11.866 µs]
Found 13 outliers among 100 measurements (13.00%)
1 (1.00%) low severe
2 (2.00%) low mild
6 (6.00%) high mild
4 (4.00%) high severe
make_date_scalar_scalar_col_1000
time: [10.730 µs 10.755 µs 10.781 µs]
Found 10 outliers among 100 measurements (10.00%)
9 (9.00%) high mild
1 (1.00%) high severe
make_date_scalar_scalar_scalar
time: [1.0802 µs 1.0936 µs 1.1060 µs]
```
- After
```
make_date_col_col_col_1000
time: [5.9822 µs 6.0034 µs 6.0267 µs]
change: [-53.494% -53.287% -53.100%] (p = 0.00 <
0.05)
Performance has improved.
make_date_scalar_col_col_1000
time: [6.2536 µs 6.2686 µs 6.2840 µs]
change: [-47.052% -46.878% -46.700%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) low mild
3 (3.00%) high mild
make_date_scalar_scalar_col_1000
time: [6.5579 µs 6.5728 µs 6.5881 µs]
change: [-39.400% -39.096% -38.810%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
make_date_scalar_scalar_scalar
time: [1.1023 µs 1.1104 µs 1.1181 µs]
change: [-0.3478% +0.9142% +2.2206%] (p = 0.16 >
0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
```
## What changes are included in this PR?
Changed the way the value is looked up in the array - instead of looking up
each value by converting to PrimitiveArray, pre-created the instance of
PrimitiveArray and used that in the loop
## Are these changes tested?
Yes, all tests and example runs are passing
## Are there any user-facing changes?
No user and API changes, tried to match the behavior to what it was
(including returning errors)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
