[ https://issues.apache.org/jira/browse/SPARK-31666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150621#comment-17150621 ]
Dongjoon Hyun commented on SPARK-31666: --------------------------------------- Hi, [~hopper-signifyd]. I found what is going on there. SPARK-23529 works correctly like the following. {code} # minikube ssh ls /data SPARK-31666.txt {code} {code} export HTTP2_DISABLE=true bin/spark-submit \ --master k8s://$K8S_MASTER \ --deploy-mode cluster \ --name spark-pi \ --class org.apache.spark.examples.SparkPi \ --conf spark.kubernetes.driverEnv.HTTP2_DISABLE=true \ --conf spark.executor.instances=1 \ --conf spark.kubernetes.container.image=spark/spark:v2.4.6 \ --conf spark.kubernetes.executor.volumes.hostPath.data.mount.path=/data \ --conf spark.kubernetes.executor.volumes.hostPath.data.options.path=/data \ local:///opt/spark/examples/jars/spark-examples_2.11-2.4.6.jar 10000 {code} {code} # k exec po/spark-pi-1593729363998-exec-1 -- ls /data SPARK-31666.txt {code} Please see the error message `Invalid value: "/tmp1": must be unique.`. The error message occurs because `spark-local-dir-x` is already mounted as volume name by Spark. You should not use the same name. {code} 20/07/02 15:38:39 INFO LoggingPodStatusWatcherImpl: State changed, new state: pod name: spark-pi-1593729518015-driver namespace: default labels: spark-app-selector -> spark-74b65a9a61cc46fd8bfc5e03e4b28bb8, spark-role -> driver pod uid: d838532b-eaa9-4b11-8eba-655f66965580 creation time: 2020-07-02T22:38:39Z service account name: default volumes: spark-local-dir-1, spark-conf-volume, default-token-n5wwg node name: N/A start time: N/A container images: N/A phase: Pending status: [] {code} > Cannot map hostPath volumes to container > ---------------------------------------- > > Key: SPARK-31666 > URL: https://issues.apache.org/jira/browse/SPARK-31666 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core > Affects Versions: 2.4.5 > Reporter: Stephen Hopper > Priority: Major > > I'm trying to mount additional hostPath directories as seen in a couple of > places: > [https://aws.amazon.com/blogs/containers/optimizing-spark-performance-on-kubernetes/] > [https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/docs/user-guide.md#using-volume-for-scratch-space] > [https://spark.apache.org/docs/latest/running-on-kubernetes.html#using-kubernetes-volumes] > > However, whenever I try to submit my job, I run into this error: > {code:java} > Uncaught exception in thread kubernetes-executor-snapshots-subscribers-1 │ > io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: > POST at: https://kubernetes.default.svc/api/v1/namespaces/my-spark-ns/pods. > Message: Pod "spark-pi-1588970477877-exec-1" is invalid: > spec.containers[0].volumeMounts[1].mountPath: Invalid value: "/tmp1": must be > unique. Received status: Status(apiVersion=v1, code=422, > details=StatusDetails(causes=[StatusCause(field=spec.containers[0].volumeMounts[1].mountPath, > message=Invalid value: "/tmp1": must be unique, reason=FieldValueInvalid, > additionalProperties={})], group=null, kind=Pod, > name=spark-pi-1588970477877-exec-1, retryAfterSeconds=null, uid=null, > additionalProperties={}), kind=Status, message=Pod > "spark-pi-1588970477877-exec-1" is invalid: > spec.containers[0].volumeMounts[1].mountPath: Invalid value: "/tmp1": must be > unique, metadata=ListMeta(_continue=null, remainingItemCount=null, > resourceVersion=null, selfLink=null, additionalProperties={}), > reason=Invalid, status=Failure, additionalProperties={}).{code} > > This is my spark-submit command (note: I've used my own build of spark for > kubernetes as well as a few other images that I've seen floating around (such > as this one seedjeffwan/spark:v2.4.5) and they all have this same issue): > {code:java} > bin/spark-submit \ > --master k8s://https://my-k8s-server:443 \ > --deploy-mode cluster \ > --name spark-pi \ > --class org.apache.spark.examples.SparkPi \ > --conf spark.executor.instances=2 \ > --conf spark.kubernetes.container.image=my-spark-image:my-tag \ > --conf spark.kubernetes.driver.pod.name=sparkpi-test-driver \ > --conf spark.kubernetes.namespace=my-spark-ns \ > --conf > spark.kubernetes.executor.volumes.hostPath.spark-local-dir-2.mount.path=/tmp1 > \ > --conf > spark.kubernetes.executor.volumes.hostPath.spark-local-dir-2.options.path=/tmp1 > \ > --conf spark.local.dir="/tmp1" \ > --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark > local:///opt/spark/examples/jars/spark-examples_2.11-2.4.5.jar 20000{code} > Any ideas on what's causing this? > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org