Gabor Roczei created SPARK-52185:
------------------------------------
Summary: Automate the thread dump collection for Spark applications
Key: SPARK-52185
URL: https://issues.apache.org/jira/browse/SPARK-52185
Project: Spark
Issue Type: New Feature
Components: Spark Core
Affects Versions: 4.1.0
Reporter: Gabor Roczei
Obtaining thread dumps for hung Spark executors and Spark drivers are not
simple tasks in YARN or Kubernetes environments.
Actual solutions which are available for debugging:
1)
We need to find out where the Java Virtual Machine (JVM) is running then run
the jstack command manually.
2)
Download the thread dumps from the Spark UI. For example:
http://localhost:4040/executors/threadDump/?executorId=driver
3)
Download the thread dumps via Spark API. For example:
{code:java}
curl
"http://localhost:4040/api/v1/applications/local-1747400853731/executors/driver/threads"{code}
The purpose of this feature request is to automate the thread dump collection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
