sgrebnov commented on PR #13267: URL: https://github.com/apache/datafusion/pull/13267#issuecomment-2465870217
@alamb - thank you for the feedback, the main challenge is that optimized plan can't be converted back to SQL after optimization (it is both added/moved and also removed projections). For example, the way how aggregation functions are unprojected rely on some assumptions about plan structure / etc. The end goal is to get unnecessary columns pruned for the cases when plan needs to be split on sub-plans and executed separately. In this scenario pushing projections/columns downs ensure only required for final execution data is fetched. Keeping layout simple and human readable is nice but second. Would implementing own optimizer using `optimize_projections` as a reference be a recommended approach in this case? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
