This is an automated email from the ASF dual-hosted git repository.
pan3793 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 07e77e6e17dd [SPARK-60612][CORE] Remove redundant dump command from
async profiler
07e77e6e17dd is described below
commit 07e77e6e17dd67ae55e8792c418238bd0b25d0bf
Author: Zhen Wang <[email protected]>
AuthorDate: Fri Oct 30 16:04:40 2026 +0800
[SPARK-60612][CORE] Remove redundant dump command from async profiler
### What changes were proposed in this pull request?
Remove the redundant `dump` command and its definition from
`SparkAsyncProfiler`, and update the related comment.
### Why are the changes needed?
`SparkAsyncProfiler.writeChunk` executes `dump` immediately after `stop`,
but this call has no effect for JFR output.
In async-profiler:
-
[`Profiler::stop`](https://github.com/async-profiler/async-profiler/blob/v4.5/src/profiler.cpp#L1096)
stops the JFR recording and sets `_state = IDLE`.
-
[`Profiler::dump`](https://github.com/async-profiler/async-profiler/blob/v4.5/src/profiler.cpp#L1144-L1150)
only flushes JFR data when `_state == RUNNING`.
Since the profiler is already in the `IDLE` state, the subsequent `dump`
does nothing.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually verified that profile files are successfully generated after
applying this change and can be parsed into flame graphs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex (GPT-6)
Closes #59858 from wForget/SPARK-60612.
Authored-by: Zhen Wang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
.../main/scala/org/apache/spark/profiler/SparkAsyncProfiler.scala | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git
a/connector/profiler/src/main/scala/org/apache/spark/profiler/SparkAsyncProfiler.scala
b/connector/profiler/src/main/scala/org/apache/spark/profiler/SparkAsyncProfiler.scala
index f86b47e98d8f..39ff33cea9ee 100644
---
a/connector/profiler/src/main/scala/org/apache/spark/profiler/SparkAsyncProfiler.scala
+++
b/connector/profiler/src/main/scala/org/apache/spark/profiler/SparkAsyncProfiler.scala
@@ -52,7 +52,6 @@ private[spark] class SparkAsyncProfiler(conf: SparkConf,
executorId: String) ext
private val startcmd =
s"start,$profilerOptions,file=$profilerLocalDir/$profileFile"
private val stopcmd =
s"stop,$profilerOptions,file=$profilerLocalDir/$profileFile"
- private val dumpcmd =
s"dump,$profilerOptions,file=$profilerLocalDir/$profileFile"
private val resumecmd =
s"resume,$profilerOptions,file=$profilerLocalDir/$profileFile"
private val PROFILER_FOLDER_PERMISSIONS = new
FsPermission(Integer.parseInt("770", 8).toShort)
@@ -172,11 +171,10 @@ private[spark] class SparkAsyncProfiler(conf: SparkConf,
executorId: String) ext
}
try {
- // stop (pause) the profiler, dump the results and then resume. This is
not ideal as we miss
- // the events while the file is being dumped, but that is the only way
to make sure that
+ // stop (pause) the profiler and then resume. This is not ideal as we
miss
+ // the events while the profiler is stopped, but that is the only way to
make sure that
// the chunk of data we are copying to dfs is in a consistent state.
profiler.get.execute(stopcmd)
- profiler.get.execute(dumpcmd)
var remaining = inputStream.available()
if (!lastChunk) {
profiler.get.execute(resumecmd)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]