ulysses-you opened a new pull request, #57751: URL: https://github.com/apache/spark/pull/57751
### What changes were proposed in this pull request? Add a **Total Task Time** column to the SQL / DataFrame tab, shown right after the existing **Duration** column. The value is the SQL-level total task time aggregated across all stages of the execution: it sums `executorRunTime` (the "Total Time Across All Tasks" stage metric) over **every attempt of every stage**, since a retried/failed attempt also genuinely consumed task time. The same column also appears in the summary table on the SQL execution detail page, because the list page and the detail page share one set of column definitions. When an execution has no stages to aggregate (e.g. a query that never launched a job), the value is reported as `-1` and rendered as **N/A** in the UI, so it is not confused with a genuine `0 ms`. ### Why are the changes needed? `Duration` mixes scheduling/queueing overhead together with the actual compute time, so it cannot tell how much real task time a query consumed. The per-stage `executorRunTime` is already collected and shown on each stage page, but the SQL tab did not aggregate it up to the query level. A SQL-level total task time lets users compare the real compute cost of queries at a glance and identify compute-heavy plans. ### Does this PR introduce _any_ user-facing change? Yes: - The SQL / DataFrame tab and the SQL execution detail page gain a new "Total Task Time" column (rendered as `N/A` when the execution has no stages). - The SQL v1 REST API's `ExecutionData` gains a `totalTaskTime` field (in milliseconds, `-1` when unknown). The field defaults to `-1`, so existing API consumers are unaffected. ### How was this patch tested? - New unit test in `SqlResourceSuite` — `totalTaskTime aggregates executorRunTime across all attempts of all stages` — verifies the aggregation across multiple stages and attempts (including a retried attempt). - Extended `SqlResourceWithActualMetricsSuite` to assert the `totalTaskTime` field of the `sqlTable` endpoint with real queries. - Both suites pass: `SqlResourceSuite` (9 tests) and `SqlResourceWithActualMetricsSuite` (10 tests). - `dev/lint-js` passes. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
