rgwood opened a new issue, #6237: URL: https://github.com/apache/arrow-datafusion/issues/6237
### Describe the bug Hello and thank you for writing+maintaining DataFusion! When upgrading from DataFusion 18, we noticed an unexpected warning that started in #4465 by @jackwener (which shipped in DataFusion 20): > WARN datafusion_optimizer::optimizer: Skipping optimizer rule 'push_down_projection' due to unexpected error: Error during planning: required columns can't push down ### To Reproduce I've put together a minimal Rust repro here: https://github.com/rgwood/df-repro In that repro, I use a [`VarProvider`](https://docs.rs/datafusion/latest/datafusion/variable/trait.VarProvider.html) to define an integer variable `@var`. I then use that variable in the WHERE clause of a SQL query: `SELECT foo FROM csv_table WHERE bar > @var` When evaluating the query, a warning indicates that `push_down_projection` was skipped: > WARN datafusion_optimizer::optimizer: Skipping optimizer rule 'push_down_projection' due to unexpected error: Error during planning: required columns can't push down, columns: {Column { relation: Some(Bare { table: "csv_table" }), name: "bar" }, Column { relation: None, name: "@var" }, Column { relation: Some(Bare { table: "csv_table" }), name: "foo" }} ### Expected behavior I expect the `push_down_projection` optimization to succeed without a warning. ### Additional context I'm relatively new to DataFusion, but it seems like `push_down_projection` is incorrectly interpreting the variable `@var` as a column when it should not be. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
