timsaucer opened a new issue, #22329: URL: https://github.com/apache/datafusion/issues/22329
## Gap `FFI_ExecutionPlan` in `datafusion/ffi/src/execution_plan.rs` does not plumb many defaulted methods of `ExecutionPlan`. Producer overrides are silently lost on the consumer side, breaking optimizer decisions that depend on them. ## Missing methods - `required_input_distribution` - `required_input_ordering` - `maintains_input_order` - `benefits_from_input_partitioning` - `check_invariants` - `apply_expressions` - `reset_state` - `with_fetch` - `fetch` - `supports_limit_pushdown` - `cardinality_effect` - `partition_statistics` - `try_swapping_with_projection` - `gather_filters_for_pushdown` - `handle_child_pushdown_result` - `with_new_state` - `try_pushdown_sort` - `with_preserve_order` ## Why it matters **Severity: critical, largest single gap by method count.** Distribution / ordering requirements drive repartition + sort insertion. Filter pushdown (`gather_filters_for_pushdown` / `handle_child_pushdown_result`) is entirely disabled across the FFI boundary today. Limit pushdown, projection swapping, cardinality-based costing, and partition statistics all silently fall back to trait defaults. ## Implementation notes - Plumb each as a plain `unsafe extern \"C\" fn`; wrapper body calls the trait method on inner `Arc<dyn ExecutionPlan>` and dispatch picks override-or-default. - Layout change → `api change` label, target `main` only, no back-port to `branch-<major>`. - Add unit tests (local-bypass + `mock_foreign_marker_id` forced-foreign) **and** integration tests under `datafusion/ffi/tests/` — layout change makes integration coverage mandatory. - Consider splitting into sub-PRs by theme (pushdown / ordering / fetch) if a single PR is unwieldy, but ship them all into one major bump. --- Generated from an audit performed for PR #22327 (datafusion-ffi agent skill). If a PR addressing this finds any item to be a false positive (e.g., a method intentionally omitted for a documented reason), please also propose an update to the `datafusion-ffi` skill so future audits do not re-flag it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
