anuragmantri opened a new pull request, #19025: URL: https://github.com/apache/iceberg/pull/19025
Draft for discussion, not for merge, and not a replacement for #18749. It is code to check out question of whether shared Spark source is better than per-version copies. The 545 common source files in 4.1 and 4.2 move to `spark/common`. Each member compiles them against its own Spark and publishes its own jars. Files that differ stay full per-version copies with no shim layer, and a `sparkCommonCheck` task enforces the split. Spark 4.3 is added on top with the same adaptations as #18749; two shared files fork. Test plan: - `sparkCommonCheck` fails on a file in both `spark/common` and a version directory, and on a copy identical in every version (checked by planting both). - All source sets of 4.1, 4.2 and 4.3 compile in one build; spotless, checkstyle and scalastyle pass. - The 4.1 and 4.2 runtime jars match jars built from main entry for entry, with identical CRCs. - Full 4.1, 4.2 and 4.3 suites (core, extensions, runtime integration): [results]. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: I did not review this fully as this is just a prototype - Prompt Summary: Rebuild the shared-source Spark prototype on current main as identical-file sharing with no shim layer, add Spark 4.3 on it for comparison with #18749, and measure the trade-offs from main's commit history. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
