kbendick opened a new issue, #5153: URL: https://github.com/apache/iceberg/issues/5153
We not have 30 Github Actions that run as part of the CI test suite, and it's starting to have a noticeable impact. We test Spark with a large number of combinations of Java versions and Scala versions. We previously only tested the "latest" Spark version (i.e. Spark 3.2) with Scala 2.13. We are now testing: - Spark 2 with Java 8 - Spark 3.0, 3.1, 3.2, 3.3 with Java 8 and Scala 2.12 - Spark 3.0, 3.1, 3.2, 3.3 with Java 11 and Scala 2.12 - Spark 3.2, 3.3 with Java 8 and Scala 2.13 - Spark 3.2, 3.3 with Java 11 and Scala 2.13 We should consider reducing the large number of combinations of JRE versions with Scala versions that are run for the various Spark versions, as CI is starting to take a good while longer. We should also look into (again) having all tests stem from one root test (by using Github Action callable workflows), so that if any one test fails, they all stop. This will reduce the number of CI slots that are running for tests that will have to be run again (as something else failed). We can also set up the faster tests first, to ensure they pass, before then calling out to the more expensive tests (such as Spark / Flink etc). I tried before with the callable workflow, but at the time it wasn't worth the effort. I think now it probably is. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
