[
https://issues.apache.org/jira/browse/SPARK-51020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ali Sabir updated SPARK-51020:
------------------------------
Attachment: spark-driver-properties.txt
exec-pod-details
driver-pod-details.txt
> User-specified executor pod templates are not working in Spark 3.3.1.
> ---------------------------------------------------------------------
>
> Key: SPARK-51020
> URL: https://issues.apache.org/jira/browse/SPARK-51020
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes, Spark Core
> Affects Versions: 3.3.1
> Reporter: Ali Sabir
> Priority: Minor
> Attachments: driver-pod-details.txt, driver-pod-template.yml,
> exec-pod-details, exec-pod-template.yml, spark-driver-properties.txt
>
>
> We are attempting to use a pod template for the executor to expose a JMX port
> for exporting metrics. While the pod template works for the driver — with the
> JMX port being created and the template successfully mounted at
> /opt/spark/pod-template — the same template is not being applied to the
> executor.
> Both the driver and executor pod templates are provided in the attached
> files. The following Spark properties are being passed during job submission:
> spark.kubernetes.driver.podTemplateFile=s3a://mybucket/code/driver-pod-template.yaml
> spark.kubernetes.executor.podTemplateFile=s3a://mybucket/code/exec-pod-template.yaml
> Despite these configurations, the executor pod template does not appear to
> take effect.
>
> I have attached the details of the executor and driver pods as they appear
> when launched. Additionally, I am including the Spark configuration passed to
> the driver during job submission.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
