Ali Sabir created SPARK-51020:
---------------------------------
Summary: User-specified executor pod templates are not working in
Spark 3.3.1.
Key: SPARK-51020
URL: https://issues.apache.org/jira/browse/SPARK-51020
Project: Spark
Issue Type: Bug
Components: Kubernetes, Spark Core
Affects Versions: 3.3.1
Reporter: Ali Sabir
We are attempting to use a pod template for the executor to expose a JMX port
for exporting metrics. While the pod template works for the driver — with the
JMX port being created and the template successfully mounted at
/opt/spark/pod-template — the same template is not being applied to the
executor.
Both the driver and executor pod templates are provided in the attached files.
The following Spark properties are being passed during job submission:
spark.kubernetes.driver.podTemplateFile=s3a://sgs-eng-prd-eh-rmo-gold/code/driver-pod-template.yaml
spark.kubernetes.executor.podTemplateFile=s3a://sgs-eng-prd-eh-rmo-gold/code/exec-pod-template.yaml
Despite these configurations, the executor pod template does not appear to take
effect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
