shaneknapp commented on issue #23514: [SPARK-24902][K8s] Add PV integration tests URL: https://github.com/apache/spark/pull/23514#issuecomment-463323885 > > this will work, but you're just missing an intermediate step (the mount on the VM). > > @shaneknapp I am using --vm-driver=none, I was not aware how things run on the test boxes and I underlined that a few comments above. And I know it has been a fuss to debug, thank you for your patience. > `none` makes life easier and it seemed more realistic to me. On the other hand, after I noticed we have that difference in the setups, we were trying to make this work on the test boxes. > So I know the tests need an adaption the question is what that is ;) > i won't (really can't) run minikube on the jenkins workers w/`--vm-driver=none` because the user launching minikube that way needs to be either root, or have root privs. and i am NOT giving jenkins root privs for builds. :) instead, i'm using the kvm2 driver as documented here: https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#kvm2-driver these boxes are running ubuntu 16.04.5LTS: ``` sknapp@ubuntu-testing:~$ uname -a Linux ubuntu-testing 4.15.0-33-generic #36~16.04.1-Ubuntu SMP Wed Aug 15 17:21:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux sknapp@ubuntu-testing:~$ cat /etc/issue Ubuntu 16.04.5 LTS ``` we're on an older version of minikube (0.25.0), and running k8s 1.10.0. i could stop work on this, upgrade to the latest and see if the regular integration tests will pass. if they do, then it might behoove us to wait until we're current before trying fancy new things like PVs. :) minikube w/a VM driver is pretty seamless once you get it up and running. i'm not sure about portability (running in a vm instead of bare metal, macos, etc) but it can be a PITA to initially get working. > > however, when you're creating the PV and linking it to the volume, you're also using `HOST_PATH`... > > but shouldn't it be mount as seen on the VM layer (eg: mount 2), and not the bare metal FS? > > So I guess that is the root cause of this. The mount should point to the path in the VM. If the path is the same as the HOST_PATH wouldnt that work anyway? yes, it *should* work, but since you're not mounting from the pod to the VM things seem to get confused. i'm going to write up some yaml to create a pod, pv, mounts, 2 containers, etc and see if i can get it to work. once i have that done, i'll give you the yaml so you can either port that in to unreadable scala, or perhaps read that in and set the test's config vars. it would be great to have you look in to things on your end, and hopefully get a test env set up. we'll need to update some docs, i think, so that people in the future understand how these features are tested, and that `sudo minikube --vm-driver=none` is the Easy Button[tm]. ;) > If you cannot do that then: > Touch should happen as you say to: `/home/jenkins/src/spark/tmp`, tmp file will be available in `/tmp/testing-k8s` in the vm and the pod should use `/tmp/testing-k8s` to do the mounting and mount `/tmp/testing-k8s` at `/opt/spark/pv-tests`. Just le tme know what path should I use on the test boxes and I will update the tests. ;) > Should I use `/home/jenkins/src/spark/tmp` and `/tmp/testing-k8s` ? use: `/home/jenkins/src/spark/tmp` (local filesystem) and `/tmp/spark-k8s-integration-tests/` (the mount that minikube advertises). please push a change that will create this dir if it's not present, and wipe it if it is. once the test is run, remove the dir. since we only allow one run of this test per worker, we can assume if the dir is there that it's cruft and safe to wipe. i will test your pushes manually, and have a bunch of tooling set up so i can build the spark dist from any PR, then run integration tests and inject my test code mods in before the tests are run. sound good?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
