viirya commented on PR #3198:
URL: https://github.com/apache/arrow-rs/pull/3198#issuecomment-1328138638
> Perhaps this could be simplified by using try_for_each_valid_idx?
Tried with `try_for_each_valid_idx`, but the benchmarks look not good:
```
equal_nulls_512 time: [1.8361 µs 1.8471 µs 1.8602 µs]
change: [+171.23% +172.28% +173.48%] (p = 0.00 <
0.05)
Performance has regressed.
Found 23 outliers among 100 measurements (23.00%)
1 (1.00%) low severe
1 (1.00%) low mild
3 (3.00%) high mild
18 (18.00%) high severe
equal_51200 time: [303.39 µs 303.51 µs 303.67 µs]
change: [+191.52% +191.71% +191.91%] (p = 0.00 <
0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
```
`try_for_each_valid_idx` comes to compare each element so it doesn't help on
the long array with few nulls (i.e., `equal_51200`), it also brings quite
regression on many null case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
