Yikun opened a new pull request, #37244: URL: https://github.com/apache/spark/pull/37244
### What changes were proposed in this pull request? Enable Spark on K8S integration tests in Github Action based on minikube: - Only trigger the K8S intergration test when `./dev/is-changed.py -m kubernetes` is `true`. - The K8S IT will triggered in user fork repo and `apache/spark` merged commits to master branch - This PR does NOT contains Volcano related test due to limited resource of github action. - minikube installation is allowed by Apache Infra: [INFRA-23000](https://issues.apache.org/jira/projects/INFRA/issues/INFRA-23000) - Why setting driver 0.5 cpu, executor 0.2 cpu? * Github-hosted runner hardware limited: [2U7G](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources), so cpu resource is very limited. * IT Job available CPU = 2U - 0.85U (K8S deploy) = 1.15U * There are 1.15 cpu left after k8s installation, to meet the requirement of K8S tests (one driver + max to 3 executors). * For memory - Time cost info: * 14 mins to compile related code. * 3 mins to build docker images. * 20-30 mins to test * Total: about 30-40 mins ### Why are the changes needed? This will also improve the efficiency of K8S development and guarantee the quality of spark on K8S and spark docker image in some level. ### Does this PR introduce _any_ user-facing change? No, dev only. ### How was this patch tested? CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
