wypoon commented on pull request #2512: URL: https://github.com/apache/iceberg/pull/2512#issuecomment-855470756
@RussellSpitzer I have changed the build to build spark3 and spark3-extensions against Spark 3.0, and added tasks to build and run the tests in those two modules against Spark 3.1. spark3-runtime is thus also built against Spark 3.0, but a task is added that builds and runs the integration test against Spark 3.1 using the runtime jar (built against 3.0). The build.gradle is based on the one in your PR, but I had to make some modifications in order to get the Spark 3.1 tests in spark3 and spark3-extensions to actually run (before that, they were not getting run, so we were only testing Spark 3.0). @rdblue please take a look and see if you are fine with this approach. I have made additional changes to the code to account for runtime invocations that were not caught before at compile-time. As I have stated, there could be more incompatibilities that we have not caught because our tests don't exercise some code paths. If you are fine with this, I'll rebase on master and make sure that changes made to spark3 and spark3-extensions since I started this PR are accounted for as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
