Hello,
Adding the dev mailing list maybe there is someone here that can help to
have/show a valid/accepted pod template for spark 3?
Thanks in advance,
Michel
Le ven. 26 juin 2020 à 14:03, Michel Sumbul <michelsum...@gmail.com> a
écrit :
> Hi Jorge,
> If I set that in the spark submit command it works but I want it only in
> the pod template file.
>
> Best regards,
> Michel
>
> Le ven. 26 juin 2020 à 14:01, Jorge Machado <jom...@me.com> a écrit :
>
>> Try to set spark.kubernetes.container.image
>>
>> On 26. Jun 2020, at 14:58, Michel Sumbul <michelsum...@gmail.com> wrote:
>>
>> Hi guys,
>>
>> I try to use Spark 3 on top of Kubernetes and to specify a pod template
>> for the driver.
>>
>> Here is my pod manifest or the driver and when I do a spark-submit with
>> the option:
>> --conf
>> spark.kubernetes.driver.podTemplateFile=/data/k8s/podtemplate_driver3.yaml
>>
>> I got the error message that I need to specify an image, but it's the
>> manifest.
>> Does my manifest file is wrong, How should it look like?
>>
>> Thanks for your help,
>> Michel
>>
>> --------
>> The pod manifest:
>>
>> apiVersion: v1
>> kind: Pod
>> metadata:
>> name: mySpark3App
>> labels:
>> app: mySpark3App
>> customlabel/app-id: "1"
>> spec:
>> securityContext:
>> runAsUser: 1000
>> volumes:
>> - name: "test-volume"
>> emptyDir: {}
>> containers:
>> - name: spark3driver
>> image: mydockerregistry.example.com/images/dev/spark3:latest
>> instances: 1
>> resources:
>> requests:
>> cpu: "1000m"
>> memory: "512Mi"
>> limits:
>> cpu: "1000m"
>> memory: "512Mi"
>> volumeMounts:
>> - name: "test-volume"
>> mountPath: "/tmp"
>>
>>
>>
