Susheel Aroskar created SPARK-54314:
---------------------------------------
Summary: Improve Server-Side debuggability in Spark Connect by
capturing client application's file name and line numbers in PySpark
Key: SPARK-54314
URL: https://issues.apache.org/jira/browse/SPARK-54314
Project: Spark
Issue Type: Improvement
Components: PySpark
Affects Versions: 3.5.7, 4.0.1, 4.0.0, 3.5.6, 3.5.5, 3.5.4
Reporter: Susheel Aroskar
Fix For: 4.0.2
Make Spark Connect server logs and observability tooling more useful by
transmitting client-side code location details (function name, file name and
line number) along with actions in PySpark. This will help developers debug
errors in production pipelines and improve observability integrations (e.g.,
OpenTelemetry).
SPIP:
[https://docs.google.com/document/d/1Sqa2QtY4tTVEwy-Tuh42gzL5RyuENmcPhPqETkpZgk0/edit?tab=t.0]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
