Tian Gao created SPARK-60951:
--------------------------------
Summary: Support configurable aggregation level for the Python UDF
sampling profiler
Key: SPARK-60951
URL: https://issues.apache.org/jira/browse/SPARK-60951
Project: Spark
Issue Type: Improvement
Components: PySpark
Affects Versions: 5.0.0
Reporter: Tian Gao
Add spark.sql.pyspark.udf.profiler.samplingAggregation (query | stage | task,
default stage) to control how the sampling profiler aggregates call stacks.
Results are merged by the chosen key so the driver keeps one call tree per
query, stage, or task attempt instead of one per task. Sampled call trees are
also keyed by thread name instead of thread id, so threads from different tasks
merge correctly.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
