+1
---- Replied Message ---- | From | Fei Wang<feiw...@apache.org> | | Date | 05/18/2022 15:05 | | To | dev@kyuubi.apache.org<dev@kyuubi.apache.org> | | Cc | | | Subject | Re: [DISCUSS] Kyuubi Hive JDBC shaded client and development experience | +1 for separating Kyuubi Hive JDBC shaded client to an independent Git repo. Regards, Fei On 2022/05/18 05:45:00 Cheng Pan wrote: > Hi Kyuubi developers, > > Recently, we notice that many new contributors are confused about > Kyuubi's development and debugging process, because they can not run > `mvn test` or run test on IDE(e.g. Jetbrains IDEA) directly like most > maven based Java projects, instead, they should run `mvn install` > before doing that. > > The root cause is Kyuubi Hive JDBC shaded client. We introduce this > module because the upstream Hive JDBC client has lots of transitive > dependencies which may pollute the user classpath and cause class > conflict. The Kyuubi Hive JDBC shaded client shaded and relocated all > Hive transitive classes to make it friendly to downstream projects and > users. > > The problem is, Maven and IDEA[1] do not work well with shaded > modules, the current workaround requires the developer to run `mvn > install` before testing. > > I have 2 ideas to solve the problem. > > 1. Some Kyuubi modules use Kyuubi Hive JDBC shaded client for testing, > we let them depend on the latest released version of Kyuubi Hive JDBC > shaded client rather than the project module, and introduce a profile > to switch back on CI environment for verification. > > 2. Separate the Kyuubi Hive JDBC shaded client to an independent Git > repo, and make it have an independent release cycle with the Kyuubi > main project. > > Further, now the Kyuubi Hive JDBC shaded client has room to improve, > i.e. it's easy to deserialize the metadata and record from thrift > format to Java format directly, we don't need to use Hive serde to do > such things, then we can drop those Hive dependencies rather than > shade them; as Kyuubi proposes to support etcd as service discover > server, we also need to support that in Kyuubi Hive JDBC client. > > What do you guys think of that? > > [1] https://youtrack.jetbrains.com/issue/IDEA-126596 > > Thanks, > Cheng Pan >
