pan3793 commented on issue #838: URL: https://github.com/apache/incubator-kyuubi/issues/838#issuecomment-883471959
I'd like to talk about what I thought about this issue. Actually, we have tried to do something to solve the jar conflicts and try our best to keep the dependencies up-to-date but also be compatible with most components. You can find some previous discussion at #488, but unfortunately, the PR got reverted finally because it's not friendly to developers. (`mvn clean install` must be run first, then IDEA and `mvn clean package` works) And I think shade is the right direction to solve this issue. If we shade `curator 2.12` and `zookeeper 3.4` then Kyuubi will be compatible with zookeeper 3.4/3.5/3.6, and it can co-exist with the normal `curator 4.2` and `zookeeper 3.5/3.6`(i.e. kyuubi engine running with spark 3.2) Then the problem becomes which is the best way to shade and use shaded jars? There are some options I have thought of. 1. shade jar in independent modules (#488) 2. shade jar in independent repos, like https://github.com/apache/flink-shaded 3. migrate from maven to gradle, and shade jar in independent sub-project, like https://github.com/apache/iceberg Now, as Spark 3.2 is on the way, and more people reported the issues, I think it's time to make the decision, and I'd like to hear what other people thought. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
