pan3793 commented on PR #7900: URL: https://github.com/apache/hudi/pull/7900#issuecomment-1423650689
Thanks for fixing this issue. And I think curator should be relocated/removed as well. The issue happens on Kyuubi IT because 1. Kyuubi invokes the curator to access ZK 2. During the testing, Kyuubi loads the vanilla curator classes from the maven test scope classpath which includes both vanilla curator jars and Hudi bundle jars 3. Since Hudi bundle jar contains vanilla curator classes, there is a chance to be loaded by Kyuubi engine (a Spark job) 4. On production workloads, Kyuubi uses the shaded and relocated curator classes(shipped by Kyuubi fat jar) so there is no problem. Based on the above facts, I think Hudi bundle jar should exclude the curator because the spark binary dist already shipped it, or relocated it to avoid class conflicts. Has a quick glance, rocksdb and some other libs are in the same position w/ curator, please consider excluding or relocating them to avoid potential class conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
