This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new dc0a14e3aae9 [SPARK-54594][UI] Skip duplicated edges starting from
subqueries in plan dot
dc0a14e3aae9 is described below
commit dc0a14e3aae973c0a9da19e57e2a139466b7e056
Author: Kent Yao <[email protected]>
AuthorDate: Thu Dec 4 08:37:22 2025 -0800
[SPARK-54594][UI] Skip duplicated edges starting from subqueries in plan dot
### What changes were proposed in this pull request?
Skip duplicated edges for subqueries in the plan dot
### Why are the changes needed?
Subquery as a parent/from node generates duplicated edges to the same
child/to node.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- After
<img width="650" height="1224" alt="image"
src="https://github.com/user-attachments/assets/05fc1c2d-1e97-498b-b09d-0c6db7d8b70e";
/>
- Before
<img width="1936" height="882" alt="image"
src="https://github.com/user-attachments/assets/971ebde6-1794-4220-a815-578bd08b59a4";
/>
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #53325 from yaooqinn/SPARK-54594.
Authored-by: Kent Yao <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 7bea0b785f1005fca8cc355c9bbd95b32465791a)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../scala/org/apache/spark/sql/execution/ui/SparkPlanGraph.scala | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SparkPlanGraph.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SparkPlanGraph.scala
index ced4b6224c88..3ddf0b69e762 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SparkPlanGraph.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SparkPlanGraph.scala
@@ -113,9 +113,12 @@ object SparkPlanGraph {
// Subquery should not be included in WholeStageCodegen
buildSparkPlanGraphNode(planInfo, nodeIdGenerator, nodes, edges,
parent, null, exchanges)
case "Subquery" if exchanges.contains(planInfo) =>
- // Point to the re-used subquery
val node = exchanges(planInfo)
- edges += SparkPlanGraphEdge(node.id, parent.id)
+ val newEdge = SparkPlanGraphEdge(node.id, parent.id)
+ if (!edges.contains(newEdge)) {
+ // Point to the re-used subquery
+ edges += newEdge
+ }
case "ReusedSubquery" =>
// Re-used subquery might appear before the original subquery, so skip
this node and let
// the previous `case` make sure the re-used and the original point to
the same node.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
