Yikun opened a new pull request, #38789: URL: https://github.com/apache/spark/pull/38789
### What changes were proposed in this pull request? This patch makes Spark K8s volcano IT can be ran in Github Action resource limited env. It will help downstream community like volcano to enable spark IT test in github action. BTW, there is no plan to enable volcano test in Spark community, this patch only make test work but don't enable the volcano test. - Change parallel job number from 4 to 2. (Only 1 job in each queue) if in github action env. - Get specified `spark.kubernetes.[driver|executor].request.cores` - Set queue limit according specified [driver|executor].request.cores just like we done in normal test: https://github.com/apache/spark/commit/883a481e44a1f91ef3fc3aea2838a598cbd6cf0f ### Why are the changes needed? It helps downstream community like volcano to enable spark IT test in github action (Currently, volcano community are using self host runner). ### Does this PR introduce _any_ user-facing change? No, test only. ### How was this patch tested? - Complete test volcano IT in github action:https://github.com/Yikun/spark/pull/187 https://github.com/Yikun/spark/actions/runs/3540512071/jobs/5943666821 <img width="1073" alt="image" src="https://user-images.githubusercontent.com/1736354/203789751-a8d80b59-173c-46eb-bd18-e59aaff65d91.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/1736354/203789751-a8d80b59-173c-46eb-bd18-e59aaff65d91.png"](https://user-images.githubusercontent.com/1736354/203789751-a8d80b59-173c-46eb-bd18-e59aaff65d91.png"){kind=link}