Hi Kyuubi developers and users, I propose to introduce a new repo apache/kyuubi-shaded in this discussion.
Why? To address the following issues: Case 1: Some dependencies which used by both Kyuubi and other dependencies, have known CVEs, but due to the breaking changes of API, we can not upgrade it to the security version. For example, currently, Kyuubi uses thrift 0.9.3, which is affected by CVE-2018-11798, CVE-2020-13949, CVE-2019-0205, but due to its breaking API change breaks compatible w/ Hive 2.3 and Hive 3.1, we can not upgrade it to 0.16.0. Case 2: For some components, we decide to stay to a lower version of client to compatible w/ an adopted widely version of server, hence we lack of some newer functionality e.g. JDK 17 support. For example, currently, Kyuubi uses Curator 2.12 and Zookeeper 3.4, because Zookeeper 3.4 is adopted by CDH 5 and CDH 6, AWS EMR(although the recent EMR distributions upgraded to Zookeeper 3.5). How? Kyuubi uses Apache Maven as the building tool, which is not good to handle shaded stuff in multiple modules project. I believe most of us felt the bad experience of kyuubi-hive-jdbc-shaded, so a separated repo should be a good idea. All shaded classes should be relocated to the new package which starting w/ `org.apache.kyuubi.shaded.`(or something else), to avoid the confliction between the existing vanilla classes. In the meanwhile, we can override some classes, which is kind of a patch to the existing third party libraries. What’s the scope? The shading/relocation technology should be the final solution, we must avoid using it as much as possible, because the relocation is not always safe, it may break the reflection invoking, and the relocated class is not fully tested. Currently, I propose to limit the scope to Thrift, Curator, Zookeeper, please list your reasons if you want to add other components. Thanks, Cheng Pan
