tschwarzinger opened a new pull request, #27492:
URL: https://github.com/apache/datafusion/pull/27492
## Which issue does this PR close?
- Closes #.
## Rationale for this change
`BinaryExpr::evaluate` called `self.data_type(schema)` for every operator
that is not handled by the early returns (e.g., `AND` and `OR`). This
recomputes the types of the entire subtree, so evaluating a deep chain of n
nodes becomes expensive. Such chains can be used in pruning predicates.
The result type is now derived from the types of the evaluated children by
calling `BinaryTypeCoercer` directly, which still rejects invalid operand types.
The new `disjunction_chain` benchmark evaluates `c0 = 0 OR c0 = 1 OR ...`:
My results:
```txt
Gnuplot not found, using plotters backend
disjunction_chain/terms=8/null
time: [6.3965 µs 6.4890 µs 6.5927 µs]
change: [−35.569% −33.802% −32.025%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
disjunction_chain/terms=32/null
time: [27.333 µs 27.837 µs 28.394 µs]
change: [−67.714% −66.742% −65.811%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
disjunction_chain/terms=128/null
time: [100.49 µs 101.77 µs 103.33 µs]
change: [−88.644% −88.416% −88.172%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
6 (6.00%) high mild
3 (3.00%) high severe
disjunction_chain/terms=8/8192_rows
time: [38.680 µs 39.223 µs 39.796 µs]
change: [−12.180% −10.017% −7.7466%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
disjunction_chain/terms=32/8192_rows
time: [164.24 µs 168.68 µs 173.39 µs]
change: [−29.078% −27.019% −24.855%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
disjunction_chain/terms=128/8192_rows
time: [592.83 µs 596.89 µs 602.42 µs]
change: [−58.399% −56.868% −55.236%] (p = 0.00 <
0.05)
Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
8 (8.00%) high mild
7 (7.00%) high severe
```
Happy to extract the benchmarks into a separate PR if necessary.
## What changes are included in this PR?
- Add `benchmark_disjunction_chain`
- Directly call `BinaryTypeCoercer` in `BinaryExpr::evaluate`
## What is the testing strategy for this PR?
- Existing tests
- Performance via benchmark
https://github.com/tschwarzinger/datafusion/actions/runs/43971312261
## Are there any user-facing changes?
- No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]