yaooqinn opened a new pull request, #55787: URL: https://github.com/apache/spark/pull/55787
### What changes were proposed in this pull request? This PR restores sub-execution grouping on the SQL tab listing that was silently dropped when the listing was switched to server-side pagination in SPARK-56140. - Backend (`SqlResource.sqlTable`): accept a new `groupSubExecution` query parameter (default = cluster config `spark.ui.groupSQLSubExecutionEnabled`). When enabled, paginate over root executions only and embed each root's children as `subExecutions: [...]` on the row. Sub-executions whose root is missing from the filtered set surface as roots so they are never hidden. Fix `recordsTotal` / `recordsFiltered` to count root rows in grouped mode so the DataTables "Showing X to Y of Z entries" matches the visible rows. - Frontend (`allexecutionspage.js`): read the existing `group-sub-exec-config` data attribute, forward `groupSubExecution=true|false` to the server, and append a trailing **Sub Executions** column showing a `+N sub` toggle on roots that have children. Toggling expands the row using DataTables `row().child()` with a nested table — matches the SPARK-41752 / Spark 4.1 layout. - CSS (`webui-dataTables.css`): minimal styling for the toggle link and the nested child table (indent + tertiary background). ### Why are the changes needed? SPARK-41752 introduced sub-execution grouping in Spark 4.1, and SPARK-55875 carried it over when the listing moved to client-side DataTables. SPARK-56140 then switched the listing to server-side pagination but didn't carry over the grouping logic — every sub-execution now shows up as its own flat row, regressing the UX for queries such as `CACHE TABLE` and nested CTAS. ### Does this PR introduce _any_ user-facing change? Yes — the SQL tab listing again folds sub-executions under their root, with a `+N sub` toggle to expand. Default behaviour is controlled by the existing `spark.ui.groupSQLSubExecutionEnabled` config (default `true`). | Before (regression) | After (this PR, collapsed) | After (this PR, expanded) | | --- | --- | --- | | flat rows | trailing `+N sub` toggle | nested child table | ### How was this patch tested? - New unit test in `SqlResourceWithActualMetricsSuite` covering both grouped and flat modes against a session that runs `CACHE TABLE` (which produces a root + sub-execution pair). - Existing `SqlResourceWithActualMetricsSuite` and `AllExecutionsPageWithInMemoryStoreSuite` continue to pass. - Manual verification in a local Spark UI: ran a workload with two sub-execution-producing statements (CACHE TABLE, CTAS) and confirmed both grouped and flat modes render correctly. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor 1.0.43 with Claude Opus 4.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
