andygrove opened a new issue, #2841: URL: https://github.com/apache/datafusion-ballista/issues/2841
Follow-up from #2721. When the planner decides whether a `UnionExec` branch can stay inline in its parent's stage (`can_stay_inline` / `restricts_to_nothing` in `ballista/scheduler/src/planner.rs`), it calls `restrict_plan_to_partitions` with no resolver. That is safe: a branch with a node Ballista doesn't know gets a stage of its own. But it means a union branch with a custom node that a configured resolver *can* restrict (Comet native, Iceberg scans) still gets its own stage and loses the inline parallelism. Passing the scheduler's `partition_sliceable_resolver` through to the planner would let such branches stay inline. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
