fwojciec opened a new pull request, #20391: URL: https://github.com/apache/datafusion/pull/20391
## Which issue does this PR close? - Closes #20388. ## Rationale for this change `collect_new_statistics` in `FilterExec` wraps NULL interval bounds in `Precision::Exact`, converting what should be `Precision::Absent` column statistics into `Precision::Exact(ScalarValue::Int32(None))`. Downstream, `estimate_disjoint_inputs` treats these as real bounds and incorrectly concludes join inputs are disjoint, forcing Partitioned join mode and disabling dynamic filter pushdown for Parquet row group pruning. ## What changes are included in this PR? Single change to `collect_new_statistics` in `filter.rs`: check `is_null()` on interval bounds before wrapping in `Precision`, mapping NULL bounds back to `Absent`. ## Are these changes tested? Yes — includes a regression test (`test_filter_statistics_absent_columns_stay_absent`) that fails on current main and passes with the fix. ## Are there any user-facing changes? No API changes. Corrects statistics propagation for tables/views with absent column statistics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
