holdenk commented on a change in pull request #25748: [SPARK-28904][K8S] Create
mount for PvTestSuire
URL: https://github.com/apache/spark/pull/25748#discussion_r326725007
##########
File path:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala
##########
@@ -42,23 +48,14 @@ private[spark] trait PVTestsSuite { k8sSuite:
KubernetesSuite =>
.withKind("PersistentVolume")
.withApiVersion("v1")
.withNewMetadata()
- .withName("test-local-pv")
+ .withName(PV_NAME)
.endMetadata()
.withNewSpec()
.withCapacity(Map("storage" -> new
QuantityBuilder().withAmount("1Gi").build()).asJava)
.withAccessModes("ReadWriteOnce")
.withPersistentVolumeReclaimPolicy("Retain")
- .withStorageClassName("test-local-storage")
- .withLocal(new LocalVolumeSourceBuilder().withPath(VM_PATH).build())
- .withNewNodeAffinity()
- .withNewRequired()
- .withNodeSelectorTerms(new NodeSelectorTermBuilder()
- .withMatchExpressions(new NodeSelectorRequirementBuilder()
- .withKey("kubernetes.io/hostname")
- .withOperator("In")
- .withValues("minikube", "docker-for-desktop",
"docker-desktop").build()).build())
Review comment:
I did some more poking and running docker-for-desktop is kind of a pain for
me, 0% of the tests pass with the default configuration. This test is already
tagged for minikube only, so I'd argue the correct run path for running the
docker-for-desktop-tests would be
`resource-managers/kubernetes/integration-tests/dev/dev-run-integration-tests.sh
--deploy-mode docker-for-desktop --spark-tgz $PWD/spark-*.tgz --exclude-tags
minikube` and then the docker for desktop test suite passes.
I'm down to spend a few hours trying to get docker for desktop working on my
machine if folks actually use this for their testing rather than MiniKube, but
I think given that the
https://github.com/apache/spark/blob/master/resource-managers/kubernetes/integration-tests/README.md
suggests `MiniKube` as the way to run our integration tests and the test is
already tagged for MiniKube only, just supporting MiniKube and leaving a follow
up issue for someone who cares about docker-4-desktop is an OK path forward.
If that is not OK I could switch this into a README PR that tells people to
set up the bind mounts manually, would that be an ok option with you?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
