shrivasshankar commented on code in PR #27148:
URL: https://github.com/apache/datafusion/pull/27148#discussion_r4516388014
##########
datafusion/physical-plan/src/filter.rs:
##########
@@ -389,7 +389,14 @@ impl FilterExec {
} else {
let null_rejecting_columns =
collect_null_rejecting_columns(predicate);
- if check_support(predicate, schema) {
+ if check_statistics_support(predicate, schema)
Review Comment:
Thanks for the repro! Went with your suggestion: `analyze` now returns no
selectivity when a column's cardinality is unknown, and `FilterExec` falls back
to the default for any type, so `decimal_columns_have_bounds` is gone. (I did
it in `calculate_selectivity` since `cardinality_ratio` is public.) Your four
cases now estimate 33, 220, 220 and 220, and they're in the slt.
Since it's not decimal-only anymore, TPC-DS q8, q34, q70 and q73 change
plans. Their `count(*)`/`rank()` filters were at 100% before. Details are in
the description; all four ran the same or faster locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
