Zhongwei Zhu created SPARK-36793:
------------------------------------
Summary: [K8S] Support write container stdout/stderr to file
Key: SPARK-36793
URL: https://issues.apache.org/jira/browse/SPARK-36793
Project: Spark
Issue Type: Improvement
Components: Kubernetes
Affects Versions: 3.1.2
Reporter: Zhongwei Zhu
Currently, executor and driver pod only redirect stdout/stderr. If users want
to sidecar logging agent to send stdout/stderr to external log storage, only
way is to change entrypoint.sh, which might break compatibility with community
version.
We should support this feature, and this feature could be enabled by spark
config. Related spark configs are:
|Key|Default|Desc|
|Spark.kubernetes.logToFile.enabled|false|Whether to write executor/driver
stdout/stderr as log file|
|Spark.kubernetes.logToFile.path|/var/log/spark|The path to write
executor/driver stdout/stderr as log file|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
