askalt commented on issue #14342: URL: https://github.com/apache/datafusion/issues/14342#issuecomment-3210652680
> [@askalt](https://github.com/askalt) It's been a while - just curious if you happened to make progress on reusing physical plans with placeholders? Yes, I implemented reuse approximately according to the plan outlined here. This is used to implement prepared statements in our project. For example, consider: `prepare query(INT) AS SELECT * FROM t WHERE a = $1`. Initially, a logical plan is stored along with the statement. As soon as a client executes the request `EXECUTE query(params)`, our planner checks if there is a physical plan in the cache. The cache can be imagined as a map from planning options (for example, a target partitions count) to `Arc<dyn ExecutionPlan>`. If there is a physical plan built with the necessary parameters, it is simply used. Placeholder substitution occurs in the execute function—currently, each plan does this independently by iterating over its expressions. A `PlaceholderExpr` is used to represent the placeholder, for which `PhysicalExpr` is implemented. The version we are using is based on release 42 and can be found at the link below. Recently, we also added support for placeholders in `ValuesExec` (it's still there) to reuse the plan for queries like `INSERT INTO t VALUES ($1, $2, ..., $n)`. https://github.com/tarantool/datafusion/tree/release-42.0.0 I would be glad if we could contribute the changes upstream. I am willing to work on this if we reach an agreement on the implementation. For example, in our fork, metrics are moved out of the plans into TaskContext and stored there in a map: plan address -> set of metrics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
