This is an automated email from the ASF dual-hosted git repository.
yaooqinn pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new defde6b580d4 [SPARK-56799][SQL] Search and highlight nodes in SQL plan
visualization
defde6b580d4 is described below
commit defde6b580d493aa87fce8b33cadd3f5054444a9
Author: Kent Yao <[email protected]>
AuthorDate: Wed May 13 00:38:43 2026 +0800
[SPARK-56799][SQL] Search and highlight nodes in SQL plan visualization
### What changes were proposed in this pull request?
This PR adds an in-graph node search to the SQL execution detail page,
sitting next to the zoom toolbar introduced in SPARK-56792.
Behavior:
- A magnifying-glass button in the plan-viz toolbar (or the `/` keyboard
shortcut while hovering the plan) opens a compact search input with a match
counter and prev/next/close buttons.
- Typing performs a case-insensitive substring match against the operator
name (`SparkPlanGraphNode.name` for plain nodes, `SparkPlanGraphCluster.name`
for WholeStageCodegen clusters).
- Matches are highlighted with an orange outline; the active match is
filled with the existing 'linked' accent color and the viewport is zoomed to
fit it. `Enter` / `Shift+Enter` and the up/down buttons cycle through matches
in DOM order; the viewport pans smoothly.
- Non-matching nodes and clusters are dimmed to opacity 0.3, but a cluster
that contains a match is never dimmed (so the matched child stays visible).
When the query has no matches, the plan is left fully visible and the counter
shows '0/0' in red, mirroring familiar find-in-page UX.
- `Esc` (or the close button) clears the search and collapses the toolbar
without resetting the user's manual zoom/pan position.
- Toggling detailed mode (which re-parses the dot file and rebuilds the
SVG) automatically reapplies the active query against the new DOM.
Implementation notes:
- Matching uses `getNodeDetails()[domId].name` so detailed-mode HTML labels
(which embed metric tables) are not searched as raw HTML.
- Cluster ancestors of a match are tracked via the graphlib instance's
`parent(v)` to drive the 'do not dim' rule.
- An `!important` on the dimming rule is required because dagre-d3 writes
an inline `opacity: 1` on each `<g class="node">` / `<g class="cluster">` at
render time.
### Why are the changes needed?
For wide SparkPlans (large joins, AQE plans with many shuffle/exchange
nodes, generated WholeStageCodegen clusters) it is currently hard to locate a
specific operator. With zoom/pan from SPARK-56792 the plan can be navigated,
but the user still needs a way to ask 'where is the BroadcastHashJoin in this
graph?'.
### Does this PR introduce _any_ user-facing change?
Yes. A new search control appears in the SQL execution detail page's
plan-viz toolbar (next to the existing zoom controls). No public APIs, configs,
metrics, or persisted data are affected. Screenshots are attached in a
follow-up comment.
### How was this patch tested?
- `build/sbt sql/Compile/compile sql/scalastyle`
- `dev/lint-js`
- `node --check` on the modified JS
- Manual smoke test with a multi-stage join + aggregate query, verified in
both basic and detailed modes:
- toolbar collapses/expands as expected
- typing 'hash' matches BroadcastHashJoin + HashAggregate (×2), counter
shows 1/3, viewport zooms to first match
- cycling with Enter/up/down updates the active match and pans
- non-matching nodes (Project, Exchange, AQEShuffleRead, LocalTableScan,
BroadcastExchange) dim to 0.3
- the WholeStageCodegen cluster containing matched HashAggregate is not
dimmed
- toggling detailed mode preserves the search
- typing a non-matching string shows '0/0' in red and leaves the plan
fully visible
- Esc clears highlights and collapses the toolbar without resetting the
zoom level
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI 1.0.44 with Claude Opus 4.7
Closes #55778 from yaooqinn/SPARK-56799.
Authored-by: Kent Yao <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
(cherry picked from commit bae8a599dead4299c334797e68c48f2f87c210fe)
Signed-off-by: Kent Yao <[email protected]>
---
.../sql/execution/ui/static/spark-sql-viz.css | 43 ++++
.../spark/sql/execution/ui/static/spark-sql-viz.js | 263 +++++++++++++++++++++
.../spark/sql/execution/ui/ExecutionPage.scala | 21 ++
3 files changed, 327 insertions(+)
diff --git
a/sql/core/src/main/resources/org/apache/spark/sql/execution/ui/static/spark-sql-viz.css
b/sql/core/src/main/resources/org/apache/spark/sql/execution/ui/static/spark-sql-viz.css
index dd0c9fc81bd5..262b2f733242 100644
---
a/sql/core/src/main/resources/org/apache/spark/sql/execution/ui/static/spark-sql-viz.css
+++
b/sql/core/src/main/resources/org/apache/spark/sql/execution/ui/static/spark-sql-viz.css
@@ -23,6 +23,7 @@
--spark-sql-selected-fill: #E25A1CFF;
--spark-sql-selected-stroke: #317EACFF;
--spark-sql-linked-fill: #FFC106FF;
+ --spark-sql-search-stroke: #FD7E14;
}
[data-bs-theme="dark"] {
--spark-sql-cluster-fill: #1a5276;
@@ -32,6 +33,7 @@
--spark-sql-selected-fill: #c0470fff;
--spark-sql-selected-stroke: #5dade2ff;
--spark-sql-linked-fill: #d4a00aff;
+ --spark-sql-search-stroke: #ffa94d;
}
svg g.label {
@@ -166,6 +168,8 @@ svg path.linked {
top: 8px;
right: 16px;
z-index: 10;
+ display: flex;
+ align-items: center;
}
.plan-viz-zoom-toolbar #plan-viz-zoom-level {
@@ -173,3 +177,42 @@ svg path.linked {
min-width: 3.25rem;
font-variant-numeric: tabular-nums;
}
+
+#plan-viz-search-expanded {
+ width: auto;
+}
+
+#plan-viz-search-input {
+ width: 12rem;
+}
+
+#plan-viz-search-count {
+ min-width: 4.5rem;
+ justify-content: center;
+ font-variant-numeric: tabular-nums;
+ font-size: 0.75rem;
+}
+
+#plan-viz-search-count.no-match {
+ color: var(--bs-danger);
+}
+
+/* Search result highlight: outline matched node/cluster with the search
color. */
+svg g.node rect.search-match,
+svg g.cluster.search-match > rect {
+ stroke: var(--spark-sql-search-stroke);
+ stroke-width: 3px;
+}
+
+/* Currently active match: keep the outline visible and add an accent fill. */
+svg g.node rect.search-match.search-active {
+ fill: var(--spark-sql-linked-fill);
+}
+
+/* Dim non-matching nodes/clusters when a search is active.
+ `!important` is needed because dagre-d3 writes an inline `opacity: 1`
+ style on each <g class="node">/<g class="cluster"> at render time. */
+svg g.node.search-dimmed,
+svg g.cluster.search-dimmed {
+ opacity: 0.3 !important;
+}
diff --git
a/sql/core/src/main/resources/org/apache/spark/sql/execution/ui/static/spark-sql-viz.js
b/sql/core/src/main/resources/org/apache/spark/sql/execution/ui/static/spark-sql-viz.js
index 037877f0dda6..81d4562cbb35 100644
---
a/sql/core/src/main/resources/org/apache/spark/sql/execution/ui/static/spark-sql-viz.js
+++
b/sql/core/src/main/resources/org/apache/spark/sql/execution/ui/static/spark-sql-viz.js
@@ -32,6 +32,12 @@ var cachedNodeDetails = null;
// d3.zoom behavior for the current SVG; reinitialized on each (re)render.
var planVizZoom = null;
+// Current dagre graph; kept so node search can re-iterate names without
re-parsing dot.
+var currentPlanGraph = null;
+
+// Node search state. matches[] is in DOM (top-down) order, index is the
active match.
+var planVizSearchState = { query: "", matches: [], index: -1 };
+
function shouldRenderPlanViz() {
return planVizContainer().selectAll("svg").empty();
}
@@ -47,6 +53,7 @@ function renderPlanViz() {
var graph = zoomLayer.append("g");
var g = graphlibDot.read(dot);
+ currentPlanGraph = g;
preprocessGraphLayout(g);
var renderer = new dagreD3.render();
renderer(graph, g);
@@ -64,6 +71,7 @@ function renderPlanViz() {
postprocessForAdditionalMetrics();
setupDetailedLabelsToggle();
setupZoomAndPan(svg, zoomLayer);
+ reapplyPlanVizSearch();
}
/* -------------------- *
@@ -679,6 +687,7 @@ function rerenderWithDetailedLabels() {
var graph = zoomLayer.append("g");
var g = graphlibDot.read(dot);
+ currentPlanGraph = g;
// If detailed mode, inject HTML labels with metrics tables
var detailed = document.getElementById("detailed-labels-checkbox");
@@ -714,6 +723,7 @@ function rerenderWithDetailedLabels() {
resizeSvg(svg);
postprocessForAdditionalMetrics();
setupZoomAndPan(svg, zoomLayer);
+ reapplyPlanVizSearch();
}
/* ---------------------- *
@@ -776,6 +786,257 @@ function planVizZoomReset() {
}
}
+/* ---------------------- *
+ * | Node search | *
+ * ---------------------- */
+
+/*
+ * Wire the search toolbar (toggle, input, navigation, close) and the global
+ * `/` shortcut. Idempotent: subsequent calls do nothing.
+ */
+function setupPlanVizSearch() {
+ var toggle = document.getElementById("plan-viz-search-toggle");
+ var input = document.getElementById("plan-viz-search-input");
+ var prevBtn = document.getElementById("plan-viz-search-prev");
+ var nextBtn = document.getElementById("plan-viz-search-next");
+ var closeBtn = document.getElementById("plan-viz-search-close");
+ if (!toggle || !input || !prevBtn || !nextBtn || !closeBtn) return;
+ if (toggle.dataset.searchWired === "true") return;
+ toggle.dataset.searchWired = "true";
+
+ toggle.addEventListener("click", function () {
+ expandPlanVizSearch();
+ });
+ closeBtn.addEventListener("click", function () {
+ collapsePlanVizSearch();
+ });
+
+ var debounceTimer = null;
+ input.addEventListener("input", function () {
+ if (debounceTimer) clearTimeout(debounceTimer);
+ debounceTimer = setTimeout(function () {
+ runPlanVizSearch(input.value, true);
+ }, 80);
+ });
+
+ input.addEventListener("keydown", function (event) {
+ if (event.key === "Enter") {
+ event.preventDefault();
+ planVizSearchGoTo(event.shiftKey ? -1 : 1);
+ } else if (event.key === "Escape") {
+ event.preventDefault();
+ collapsePlanVizSearch();
+ }
+ });
+
+ prevBtn.addEventListener("click", function () { planVizSearchGoTo(-1); });
+ nextBtn.addEventListener("click", function () { planVizSearchGoTo(1); });
+
+ document.addEventListener("keydown", function (event) {
+ if (event.ctrlKey || event.metaKey || event.altKey) return;
+ if (event.key !== "/") return;
+ var tag = event.target && event.target.tagName;
+ if (tag === "INPUT" || tag === "TEXTAREA" ||
event.target.isContentEditable) {
+ return;
+ }
+ var graphEl = document.getElementById("plan-viz-graph");
+ if (!graphEl || !graphEl.matches(":hover")) return;
+ event.preventDefault();
+ expandPlanVizSearch();
+ });
+}
+
+function expandPlanVizSearch() {
+ var collapsed = document.getElementById("plan-viz-search-collapsed");
+ var expanded = document.getElementById("plan-viz-search-expanded");
+ var input = document.getElementById("plan-viz-search-input");
+ if (!collapsed || !expanded || !input) return;
+ collapsed.classList.add("d-none");
+ expanded.classList.remove("d-none");
+ input.value = planVizSearchState.query;
+ input.focus();
+ input.select();
+}
+
+function collapsePlanVizSearch() {
+ var collapsed = document.getElementById("plan-viz-search-collapsed");
+ var expanded = document.getElementById("plan-viz-search-expanded");
+ var input = document.getElementById("plan-viz-search-input");
+ clearPlanVizSearchHighlights();
+ planVizSearchState.query = "";
+ planVizSearchState.matches = [];
+ planVizSearchState.index = -1;
+ updatePlanVizSearchCount();
+ if (input) input.value = "";
+ if (collapsed) collapsed.classList.remove("d-none");
+ if (expanded) expanded.classList.add("d-none");
+}
+
+/*
+ * Recompute matches against the current query, update DOM classes, and zoom
+ * to the first match. When `zoomToFirst` is false (e.g. re-applying after a
+ * detailed-mode rerender), the viewport is left untouched.
+ */
+function runPlanVizSearch(rawQuery, zoomToFirst) {
+ var query = (rawQuery || "").trim();
+ planVizSearchState.query = query;
+ planVizSearchState.matches = [];
+ planVizSearchState.index = -1;
+
+ clearPlanVizSearchHighlights();
+
+ if (query === "" || !currentPlanGraph) {
+ updatePlanVizSearchCount();
+ return;
+ }
+
+ var lower = query.toLowerCase();
+ var nodeDetails = getNodeDetails();
+ var matchedDomIds = Object.create(null);
+ var ancestorsOfMatch = Object.create(null);
+
+ currentPlanGraph.nodes().forEach(function (v) {
+ var node = currentPlanGraph.node(v);
+ if (!node) return;
+ var domId = node.id || (node.isCluster ? v : "node" + v);
+ var displayName;
+ if (nodeDetails[domId] && nodeDetails[domId].name) {
+ displayName = String(nodeDetails[domId].name);
+ } else {
+ displayName = String(node.label || "");
+ }
+ if (displayName.toLowerCase().indexOf(lower) >= 0) {
+ matchedDomIds[domId] = true;
+ // Walk up the compound-graph hierarchy so we don't dim a cluster that
+ // contains a match (which would visually hide the matched child).
+ var parent = currentPlanGraph.parent(v);
+ while (parent) {
+ var parentNode = currentPlanGraph.node(parent);
+ var parentDomId = (parentNode && parentNode.id) || parent;
+ ancestorsOfMatch[parentDomId] = true;
+ parent = currentPlanGraph.parent(parent);
+ }
+ }
+ });
+
+ var svg = planVizContainer().select("svg");
+ var nodeEls = svg.selectAll("g.node, g.cluster").nodes();
+ var orderedMatches = [];
+ var anyMatch = false;
+ Object.keys(matchedDomIds).forEach(function () { anyMatch = true; });
+ nodeEls.forEach(function (el) {
+ if (matchedDomIds[el.id]) {
+ orderedMatches.push(el.id);
+ var rect = el.querySelector(":scope > rect");
+ if (rect) rect.classList.add("search-match");
+ if (el.classList.contains("cluster")) el.classList.add("search-match");
+ } else if (anyMatch && !ancestorsOfMatch[el.id]) {
+ // Only dim when at least one match exists; otherwise leave the plan
+ // fully visible so the user can adjust their query without obscuring
+ // context (matches familiar find-in-page UX).
+ el.classList.add("search-dimmed");
+ }
+ });
+
+ planVizSearchState.matches = orderedMatches;
+ if (orderedMatches.length > 0) {
+ planVizSearchState.index = 0;
+ markActiveMatch(orderedMatches[0]);
+ if (zoomToFirst) zoomToNode(orderedMatches[0]);
+ }
+ updatePlanVizSearchCount();
+}
+
+function planVizSearchGoTo(delta) {
+ var matches = planVizSearchState.matches;
+ if (matches.length === 0) return;
+ var idx = (planVizSearchState.index + delta + matches.length) %
matches.length;
+ planVizSearchState.index = idx;
+ markActiveMatch(matches[idx]);
+ zoomToNode(matches[idx]);
+ updatePlanVizSearchCount();
+}
+
+function markActiveMatch(domId) {
+ planVizContainer().selectAll("rect.search-active")
+ .classed("search-active", false);
+ if (!domId) return;
+ var el = document.getElementById(domId);
+ if (!el) return;
+ var rect = el.querySelector(":scope > rect");
+ if (rect) rect.classList.add("search-active");
+}
+
+function clearPlanVizSearchHighlights() {
+ var container = planVizContainer();
+ container.selectAll(".search-match").classed("search-match", false);
+ container.selectAll(".search-active").classed("search-active", false);
+ container.selectAll(".search-dimmed").classed("search-dimmed", false);
+}
+
+function updatePlanVizSearchCount() {
+ var el = document.getElementById("plan-viz-search-count");
+ if (!el) return;
+ var matches = planVizSearchState.matches;
+ if (planVizSearchState.query === "") {
+ el.textContent = "";
+ el.classList.remove("no-match");
+ } else if (matches.length === 0) {
+ el.textContent = "0/0";
+ el.classList.add("no-match");
+ } else {
+ el.textContent = (planVizSearchState.index + 1) + "/" + matches.length;
+ el.classList.remove("no-match");
+ }
+}
+
+/* Center and scale the viewport on the given DOM element using d3-zoom. */
+function zoomToNode(domId) {
+ var el = document.getElementById(domId);
+ if (!el || !planVizZoom) return;
+ var svg = planVizContainer().select("svg");
+ var svgNode = svg.node();
+ if (!svgNode) return;
+ var vb = svgNode.viewBox && svgNode.viewBox.baseVal;
+ if (!vb || vb.width === 0 || vb.height === 0) return;
+
+ var bbox;
+ try {
+ bbox = el.getBBox();
+ } catch (e) {
+ return;
+ }
+ if (!bbox || bbox.width === 0 || bbox.height === 0) return;
+
+ // Aim to fill ~50% of the viewport so the matched node is prominent but
+ // still in context. Clamp to the configured zoom range.
+ var scale = Math.min(
+ vb.width / bbox.width / 2,
+ vb.height / bbox.height / 2,
+ PlanVizConstants.zoomMax
+ );
+ scale = Math.max(scale, PlanVizConstants.zoomMin);
+
+ var bcx = bbox.x + bbox.width / 2;
+ var bcy = bbox.y + bbox.height / 2;
+ var vcx = vb.x + vb.width / 2;
+ var vcy = vb.y + vb.height / 2;
+ var transform = d3.zoomIdentity
+ .translate(vcx - bcx * scale, vcy - bcy * scale)
+ .scale(scale);
+
+ svg.transition().duration(400).call(planVizZoom.transform, transform);
+}
+
+/*
+ * After a render or detailed-mode toggle, reapply the active query against the
+ * fresh DOM so highlights survive re-layout. No-ops when no search is active.
+ */
+function reapplyPlanVizSearch() {
+ if (!planVizSearchState.query) return;
+ runPlanVizSearch(planVizSearchState.query, false);
+}
+
document.addEventListener("DOMContentLoaded", function () {
if (shouldRenderPlanViz()) {
renderPlanViz();
@@ -798,6 +1059,8 @@ document.addEventListener("DOMContentLoaded", function () {
zoomResetBtn.addEventListener("click", planVizZoomReset);
}
+ setupPlanVizSearch();
+
// Keyboard shortcuts when the SVG is focused or the user hovers over it.
document.addEventListener("keydown", function (event) {
if (event.ctrlKey || event.metaKey || event.altKey) return;
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
index 827e0f92dfc9..9ff410e829e2 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
@@ -169,6 +169,27 @@ class ExecutionPage(parent: SQLTab) extends
WebUIPage("execution") with Logging
<span>Show metrics in graph nodes (detailed mode)</span>
</div>
<div id="plan-viz-zoom-toolbar" class="plan-viz-zoom-toolbar">
+ <div id="plan-viz-search-collapsed" class="btn-group
btn-group-sm me-2"
+ role="group" aria-label="Search">
+ <button id="plan-viz-search-toggle" type="button"
+ class="btn btn-light border" title="Find node
(/)">🔍</button>
+ </div>
+ <div id="plan-viz-search-expanded" class="input-group
input-group-sm me-2 d-none"
+ role="group" aria-label="Search">
+ <input id="plan-viz-search-input" type="search"
autocomplete="off"
+ class="form-control form-control-sm border"
+ placeholder="Find node..." aria-label="Find node"/>
+ <span id="plan-viz-search-count" class="input-group-text
bg-light"></span>
+ <button id="plan-viz-search-prev" type="button"
+ class="btn btn-light border"
+ title="Previous match (Shift+Enter)">↑</button>
+ <button id="plan-viz-search-next" type="button"
+ class="btn btn-light border"
+ title="Next match (Enter)">↓</button>
+ <button id="plan-viz-search-close" type="button"
+ class="btn btn-light border"
+ title="Close search (Esc)">×</button>
+ </div>
<div class="btn-group btn-group-sm" role="group"
aria-label="Zoom controls">
<button id="plan-viz-zoom-out" type="button"
class="btn btn-light border" title="Zoom out
(-)">−</button>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
