steveloughran opened a new issue, #15100: URL: https://github.com/apache/iceberg/issues/15100
### Feature Request / Improvement #14125 set the version of the hadoop libraries iceberg is built with to 3.4.2 However, spark 3.4.4 and 3.5.8 are built with Hadoop 3.3.5. There's a risk of accidental use of APIs, enum values, overridden methods and the like such that iceberg doesn't actually run properly in these versions. And while the iceberg testing _should_ find problems, there's inevitably code paths, which don't get tested, especially those related to different deployment scenarios. That's independent of the challenge of using new APIs (#12055) in the code. It would be much safer to fix the hadoop version in builds to be that of the oldest supported spark release. Those developers who want to adopt new features will have to use reflection. Easy to fix * Downgrade hadoop3 version in gradle/libs.versions.toml * Add a comment saying "must be in sync with oldest supported spark release" ### Query engine None ### Willingness to contribute - [x] I can contribute this improvement/feature independently - [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
