nuno-faria opened a new issue, #26706: URL: https://github.com/apache/datafusion/issues/26706
### Is your feature request related to a problem or challenge? Building and optimizing the physical plan can be relatively expensive when we execute small queries. Currently, DataFusion's prepared statements only reuse the logical plan. <img width="637" height="349" alt="Image" src="https://github.com/user-attachments/assets/7a891ce3-996d-46a8-8550-0713ec0dc87b"; /> In a proof of concept, I found that we can improve the performance of small queries in >3x if we just reuse the physical plan. <img width="508" height="329" alt="Image" src="https://github.com/user-attachments/assets/94173913-3f28-41f2-a771-0c5798d66834"; /> This was done in a hacky way that first created the physical plan using placeholder arguments and at runtime replaced nodes like `DataSourceExec` and `FilterExec` with the passed arguments. ### Describe the solution you'd like For a clean implementation, I think we need the following: - Build `ExecutionPlan` from queries with placeholder values. - Add a `ExecutionPlan::set_placeholder` (or similar) method that is implemented by every operator that uses placeholder values, changing only what is necessary and returning a new plan that can be consumed. This would certainly be a breaking change. ### Describe alternatives you've considered Hacky downstream implementation. ### Additional context Related: - https://github.com/apache/datafusion/issues/14342 - https://github.com/apache/datafusion/issues/19351 ### Willingness to contribute None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
