neilconway commented on PR #21240:
URL: https://github.com/apache/datafusion/pull/21240#issuecomment-4187695529
> Any chance we can break this PR into smaller pieces (e.g. move benchmarks,
for example) to make it easier to review?
Hmmm, that might be a bit tricky. The benchmarks are pretty trivial and
could easily be omitted. Here's how Claude summarizes the PR:
1. `ScalarSubqueryExec` execution plan — A new physical plan node that wraps
a main input plan and a set of subquery plans.
2. `ScalarSubqueryExpr` physical expression — A PhysicalExpr that reads a
scalar value from a shared `OnceLock`-based results container, populated by
`ScalarSubqueryExec`.
4. `ExecutionProps` as the bridge — Carries `subquery_indexes` (mapping
logical `Subquery` → result slot) and subquery_results (the shared `OnceLock`
container) so that `create_physical_expr` can convert `Expr::ScalarSubquery`
into `ScalarSubqueryExpr`.
3. Physical planner integration — `create_initial_plan` collects
uncorrelated scalar subqueries at each plan level, plans them, allocates a
shared results container, and wraps the main plan in `ScalarSubqueryExec`.
5. `scalar_subquery_to_join` scoped to correlated subqueries only —
Uncorrelated scalar subqueries are no longer rewritten to joins by the
optimizer; they flow through to the physical planner instead.
6. Protobuf serialization — Round-trip serde support for the new plan
nodes and expressions.
7. Tree traversal helpers — LogicalPlan::map_uncorrelated_subqueries and
Expr::contains_scalar_subquery.
8. Benchmarks (trivial, would be fine to omit)
9. Tests and updates to expected query plans
If it is helpful, I could prepare two PRs that have a split like:
1. `ScalarSubqueryExec`, `ScalarSubqueryExpr`, `ExecutionProps` change,
protobuf serialization, benchmarks (or omit them)
2. Planner and optimizer changes, tree traversal helpers, test updates
If you think that would be easier to review, lmk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
