rdblue commented on issue #1826: URL: https://github.com/apache/iceberg/issues/1826#issuecomment-733981188
> Decide when we need to run the integration tests I think that we would want to create a new GitHub action that runs Hive integration tests separately and for just one JVM profile. That way, we have an action for JDK8 unit tests, an action for JDK11 unit tests, and a Hive integration test action that runs in parallel with those. That would save overall time by running in parallel, and would save resources by running the expensive tests just once. I would still want to run the tests on each PR because we don't want changes to core breaking the Hive integration tests. Also, we could possibly run the integration tests with Hive 3 if anything in iceberg-mr or iceberg-hive-metastore changes. > Run the actual tests on the CI When we moved to GitHub actions, I updated the python tests to only run when a file in `python/` changes. We could use the same strategy for these tests. But the main thing is moving expensive tests into an `integrationTest` task and calling it from a GitHub action with `./gradlew iceberg-mr:integrationTest -x test`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
