Hi, I am trying to run Spark on the Kubernetes cluster, but that cluster has certain validation to deploy any pod that is not allowing me to run my Spark submit.
for example, I need to add liveness, readiness probes and certain security capability restrictions, which we usually do for all outer pods via yaml file. not sure how to get that in Spark submit k8s. I tried the driver and executor template file, but it's not working. at the same time, I do not have a sample template file from the documentation except below lines --conf spark.kubernetes.driver.podTemplateFile=s3a://bucket/driver.yml --conf spark.kubernetes.executor.podTemplateFile=s3a://bucket/executor.yml Can some one provide directions how to proceed further. Thanks, Jilani
