danielhumanmod commented on issue #1293:
URL:
https://github.com/apache/datafusion-ballista/issues/1293#issuecomment-3218378622
Hey @milenkovicm , I read through the codebase and have an initial proposal
for EXPLAIN (I will start work on EXPLAIN first, then EXPLAN ANALYZE)
1. In Ballista's scheduler phase (when logical → physical), detect the
`ExecExplain` (DF one) node, replace with new custom `BallistaExecPlan` node
2. Implement `BallistaExecPlan` returns three rows:
- Logical plan: reuse DF’s final logical plan from ExplainExec.
- physical plan: reuse DF’s final physical plan from ExplainExec
- distributed plan: derive from Ballista’s stage DAG
(DefaultDistributedPlanner::plan_query_stages), showing stages/shuffles/deps.
**PS: Why a new exec instead of reusing ExplainExec?**
DF’s
[ExplainExec](https://github.com/apache/datafusion/blob/main/datafusion/physical-plan/src/explain.rs#L156)
filters rows via PlanType::should_display(). In non-VERBOSE mode it only shows
logical_plan and physical_plan. Our distributed_plan is unknown to DF’s
PlanType, so:
• Plan A: either we “fake” it as another physical_plan row (but
potentially confusing to user), or
• Plan B: we introduce a Ballista-specific exec that appends a
third distributed_plan row explicitly.
Happy to know your thought here, is the solution sounds good to you, and do
you prefer plan A or B in this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
