Nishanth created SPARK-54152:
--------------------------------
Summary: Spark | `Add Spark task context information to
FileScanRDD and PythonRunner logs for improved UDF debugging `
Key: SPARK-54152
URL: https://issues.apache.org/jira/browse/SPARK-54152
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 4.0.1, 4.0.0, 3.5.2, 3.5.1
Reporter: Nishanth
Currently, log messages in FileScanRDD.compute() and PythonRunner do not
include Spark task context information (e.g., Task ID, Partition ID).
This change adds task context details to file reading and Python UDF execution
logs to improve traceability and debugging when UDFs hang or perform slowly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
