[
https://issues.apache.org/jira/browse/SPARK-59487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59487:
-----------------------------------
Labels: pull-request-available (was: )
> Add task-level spans to correlate Spark tasks with executor profiling samples
> -----------------------------------------------------------------------------
>
> Key: SPARK-59487
> URL: https://issues.apache.org/jira/browse/SPARK-59487
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 5.0.0
> Reporter: Zhen Wang
> Priority: Major
> Labels: pull-request-available
>
> The executor profiler records samples associated with executor threads, but
> the profiling output currently lacks an explicit mapping between Spark task
> attempts and those threads over time. Since each executor thread can run
> multiple tasks sequentially, the thread identity alone is insufficient to
> determine which task produced a profiling sample. This makes it difficult to
> connect a slow task observed in the Spark UI with its corresponding profile.
> Async-profiler introduced the Span API in [PR
> #1755|https://github.com/async-profiler/async-profiler/pull/1755]. A span
> records an execution interval on a thread with a user-defined tag, providing
> the thread and time information needed to associate profiling samples with a
> task attempt.
> Add task-level spans to {{{}ProfilerExecutorPlugin{}}}, recording each task
> attempt’s interval with its stage, partition, task attempt identifiers, and
> completion status. These spans establish the task-to-thread mapping within
> the same JFR recording, allowing users to identify and analyze samples
> belonging to a specific task.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
