rdblue opened a new pull request #3256: URL: https://github.com/apache/iceberg/pull/3256
This is another option for refactoring the Spark build. This is similar to #3237, but it also moves Spark projects into separate build.gradle files and allows building multiple Spark versions in parallel. This removes Spark 3.1 testing from the Spark 3.0 project because 3.0 will be copied to produce a 3.1 build. Otherwise, Spark builds are unchanged to keep changes small. There should also be further refactoring after these build changes to copy the contents of `iceberg-spark` into each version to ensure all classes are compiled against the correct version of Spark and Scala. This is the new layout: ``` ├── spark │ ├── v2.4 │ │ ├── spark2 │ │ └── spark-runtime │ └── v3.0 │ ├── spark3 │ │ └── src │ ├── spark3-extensions │ │ └── src │ └── spark3-runtime │ └── src ``` Work to do after this: * Refactor `iceberg-spark` and move into version-specific modules * Add a v3.1 build * Refactoring to remove modules that are no longer needed, like `spark3-extensions` * Apply similar refactoring to `jmh.gradle` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
