goldmedal commented on PR #13267: URL: https://github.com/apache/datafusion/pull/13267#issuecomment-2466215869
I enabled `optimize_projections_preserve_existing_projections` to run the sqllogictests. I found that many projection push-downs do not work (the physical plan changes), and some queries fail. ### Push-down doesn’t work, but the query still succeeds (execution behavior may vary): - Join - Union - Predicate - Sort - Window ### Query failures: - `same_column_name_cross_join.slt` It appears the physical planner struggles with additional projections. I believe `optimize_projections_preserve_existing_projections` is an unparser-specific config and shouldn't be used in query execution. However, I see the value in the additional projection for the unparser, as it helps identify the SQL body (`SELECT`) more easily. As @alamb mentioned, having a config that changes part of an optimization can be confusing. Here are some potential solutions: - Instead of modifying the existing optimization, create a separate `optimize_projection` rule for unparsing, allowing users to optimize their plan for SQL generation. - Provide a config to switch `optimize_projection` into unparsing mode (though this might complicate the codebase). - Improve the unparser to handle the original optimized plan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
