dimtiris kanoute created SPARK-39580:
----------------------------------------
Summary: Write event logs in a continuous manner
Key: SPARK-39580
URL: https://issues.apache.org/jira/browse/SPARK-39580
Project: Spark
Issue Type: New Feature
Components: Spark Core
Affects Versions: 3.3.0
Reporter: dimtiris kanoute
We are using Spark Thrift Server as a service to run Spark SQL queries.
Currently Spark writes *eventlogs* once the job is done/ gracefully killed.
This means that in case of Spark thrift Server when it runs as one job in case
of unexpected error ( i.e. OOM ) we cannot access the logs after, as it doesn't
write them due to the unexpected shut down.
We would like to suggest as an improvement the functionality for sparks *to
write the event logs to a cloud storage in a continuous manner.*
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
