Chao Sun created SPARK-48392:
--------------------------------
Summary: (Optionally) Load `spark-defaults.conf` when passing
configurations to `spark-submit` through `--properties-file`
Key: SPARK-48392
URL: https://issues.apache.org/jira/browse/SPARK-48392
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.5.1
Reporter: Chao Sun
Currently, when user pass configurations to `spark-submit.sh` via
`--properties-file`, the `spark-defaults.conf` will be completely ignored. This
poses issues for some people, for instance, those using [Spark on K8S operator
from kubeflow|https://github.com/kubeflow/spark-operator]. See related issues:
- https://github.com/kubeflow/spark-operator/issues/1183
- https://github.com/kubeflow/spark-operator/issues/1321
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
