pan3793 commented on PR #10460: URL: https://github.com/apache/iceberg/pull/10460#issuecomment-2268888543
@pvary Unfortunately, I think this is not an issue specify to Hive 2.3.10, the whole Hive 2.3 series have issues on loading Jackson 2.x classes. > Do we have a clear understanding that why and when was the dependency updated? Are the owners of the change aware of this issue? Are there any alternatives to accepting that the Hive integration is not working anymore? As explained above, the root cause is `avatica:1.8.0` ships old un-relocated Jackson classes. Before https://github.com/apache/hive/commit/e45d5ffa8674418ffd031db5b9d66c28c6883805 (Hive 2.3.10), those classes won't be initialized by normal bootstrap process of HS2 and HMS, while this patch (https://github.com/apache/hive/commit/e45d5ffa8674418ffd031db5b9d66c28c6883805), switched Hive built-in JSON UDFs from Jackson 1.x to Jackson 2.x (for security purpose), which made Hive eagerly initialize Jackson 2.x classes, thus exposing Jackson 2.x class conflict issues. This is not identified on the Hive CI, I guess it might be caused by different classpath orders for jars. As you can see, in https://github.com/apache/iceberg/pull/10460/commits/1a1e5fc0e4b08fbe6e11f1d052c97c96aae303ba, even I do nothing but only initialize a Jackson 2.x class, the UT fails. To make the Iceberg Hive 2.3.10 integration tests work normally, a workaround I can thinking is, creating a `avatica:1.8.0-patched`(this is pretty easy in Gradle, or I can do it in a sepreate repo and publish to Maven Central) to replace `avatica:1.8.0` to address the Jackson 2.x issue. As this is an Hive side issue, and Hive 2.3 is EOL, for users who want to use Hive 2.3.x, they should fix the `avatica:1.8.0` by themselves. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
