> You completely misunderstood what I meant. I don’t get your point, I tried my best to answer/explain each of your questions/concerns.
> kyuubi-shaded-hive-service-rpc, > kyuubi-hive-service-rpc-shaded > > I just wanted to know why these two kinds of naming policies can make > both end-users and kyuubi developers happy? Just follow the common practice, I believe I have listed sufficient examples to prove that. >> Personally, I think it’s a quite common practice, especially in Apache >> projects, thus it’s obvious to me, for example: >> >> - >> https://mvnrepository.com/artifact/org.apache.hbase.thirdparty/hbase-shaded-netty >> - >> https://mvnrepository.com/artifact/org.apache.hadoop.thirdparty/hadoop-shaded-guava >> - https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-jackson >> - https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-bundled-guava >> Also, I just follow the most popular name pattern >> `<project_name>-shaded-<thrid_party_name>` to name Kyuubi shaded artifacts. ==================================================================================================== > Is it said that if an end-users use kyuubi-hive-jdbc-shaded as a JDBC > Driver, he/she could not use kyuubi-shaded-hive-service-rpc as > as a thrift client? We encourage users to use `kyuubi-hive-jdbc-shaded` and do not recommend users to use `kyuubi-shaded-hive-service-rpc`, `kyuubi-shaded-hive-service-rpc` is designed for Kyuubi project internal usage, we should not expose those shaded classes to public API. But as I said before > Technically, we can not stop the user from using such shaded API directly. Thanks, Cheng Pan > On Nov 30, 2023, at 14:34, Kent Yao <y...@apache.org> wrote: > > Hi Pan, > > You completely misunderstood what I meant. > > kyuubi-shaded-hive-service-rpc, > kyuubi-hive-service-rpc-shaded > > I just wanted to know why these two kinds of naming policies can make > both end-users and kyuubi developers happy? > > Why is even this necessary? > > Is it said that if an end-users use kyuubi-hive-jdbc-shaded as a JDBC > Driver, he/she could not use kyuubi-shaded-hive-service-rpc as > as a thrift client? > > Cheng Pan <pan3...@gmail.com> 于2023年11月30日周四 11:25写道： >> >> >>>> The artifacts end with “-shaded” are for end users. >>> >>> This looks like an explanation for mistakes we've already made. >> >> I don’t think they're mistakes, only facts. In practice, there are various >> ways to name this type of jar, we just picked one of them. >> >> There are some examples: >> >> - https://mvnrepository.com/artifact/com.datastax.oss/java-driver-core-shaded >> - >> https://mvnrepository.com/artifact/org.apache.zeppelin/zeppelin-interpreter-shaded >> - >> https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-spark-runtime-3.3 >> - https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client-api >> - https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client-runtime >> - >> https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kinesis-asl-assembly >> >> Currently, Kyuubi has two such modules >> >> - kyuubi-hive-jdbc-shaded >> - kyuubi-spark-authz-shaded >> >> The main purpose of this kind of artifact is to assemble all transitive >> dependencies into one jar, usually with proper third-party class relocation >> but WITHOUT changing any public API, e.g. for the JDBC driver, the public >> API is JDBC API, it also means that the user can migrate from `A` to >> `A-shaded` smoothly if only public API is consumed. >> >>> I've never been heard of such policy being discussed, documented. >> >> There hasn't been any specific topic of discussion about name policy before. >> It's impossible to discuss everything in detail. If someone thinks something >> matters, then we should discuss it. >> >>> Furthermore, since we've already posted them publicly in the maven >>> repository, >>> how can we decide that they won't be used by end-users? >> >> Technically, we can not stop the user from using such shaded API directly. >> Personally, I think it’s a quite common practice, especially in Apache >> projects, thus it’s obvious to me, for example: >> >> - >> https://mvnrepository.com/artifact/org.apache.hbase.thirdparty/hbase-shaded-netty >> - >> https://mvnrepository.com/artifact/org.apache.hadoop.thirdparty/hadoop-shaded-guava >> - https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-jackson >> - https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-bundled-guava >> >> Also, I just follow the most popular name pattern >> `<project_name>-shaded-<thrid_party_name>` to name Kyuubi shaded artifacts. >> >>> Did we explicitly remove the public APIs from the original artifact？ >> >> >> No, in most cases, just simple relocation and repackaging of the original >> jar, with class overwriting if necessary. >> >> Thanks, >> Cheng Pan >> >> >>> On Nov 29, 2023, at 11:13, Kent Yao <y...@apache.org> wrote: >>> >>>> The artifacts end with “-shaded” are for end users. >>> >>> This looks like an explanation for mistakes we've already made. >>> >>> I've never been heard of such policy being discussed, documented. >>> >>> Furthermore, since we've already posted them publicly in the maven >>> repository, >>> how can we decide that they won't be used by end-users? Did we explicitly >>> remove >>> the public APIs from the original artifact？ >>> >>> Kent >>> >>> Cheng Pan <pan3...@gmail.com> 于2023年11月29日周三 10:25写道： >>>> >>>> Kent >>>> >>>> They are for different purposes. >>>> >>>> The artifacts start with "kyuubi-shaded” is for the Kyuubi project's >>>> internal usage, see [1] and [2] >>>> >>>> The artifacts end with “-shaded” are for end users. >>>> >>>> [1] https://github.com/apache/kyuubi-shaded >>>> [2] >>>> https://mvnrepository.com/artifact/org.apache.kyuubi/kyuubi-shaded-zookeeper-34 >>>> >>>> Thanks, >>>> Cheng Pan >>>> >>>> >>>>> On Nov 28, 2023, at 18:36, Kent Yao <y...@apache.org> wrote: >>>>> >>>>> -1 for the arbitrary naming >>>>> >>>>> >>>>> FYI, >>>>> https://mvnrepository.com/artifact/org.apache.kyuubi/kyuubi-hive-jdbc-shaded >>>>> >>>>> XiDuo You <ulyssesyo...@gmail.com> 于2023年11月28日周二 18:03写道： >>>>>> >>>>>> +1, looks fine to me >>>>>> >>>>>> Zhen Wang <wangz...@apache.org> 于2023年11月28日周二 12:53写道： >>>>>>> >>>>>>> +1, I like this idea, it avoids the possibility of engine thrift class >>>>>>> conflicts in different environments. >>>>>>> >>>>>>> >>>>>>> Kind Regards, >>>>>>> Zhen Wang >>>>>>> >>>>>>> Cheng Pan <pan3...@gmail.com> 于2023年11月28日周二 01:00写道： >>>>>>>> >>>>>>>> Hi Kyuubi developers, >>>>>>>> >>>>>>>> Recently, I've been looking at the code of the Hive engine, one of my >>>>>>>> goals id to make it support multiple versions of Hive runtime, >>>>>>>> including >>>>>>>> - 3.1.3, which is the latest stable version of Apache Hive, already >>>>>>>> supported >>>>>>>> - 2.3.9, which is the latest stable version of Apache Hive 2.x, is >>>>>>>> adopted widely, including Apache Spark, Apache Flink >>>>>>>> - 2.1.1-cdh6.3.2, the latest free version of CDH, has lots of users >>>>>>>> >>>>>>>> When I tried to run the Hive engine built against Hive 3.x with Hive >>>>>>>> 2.3.9/2.1.1-cdh6.3.2 runtime, I encountered some thrift class conflict >>>>>>>> issues which were hard to resolve, thus I propose to create a >>>>>>>> pre-shaded hive-service-rpc to tackles such issue. >>>>>>>> >>>>>>>> I have created two PRs[1][2] to demonstrate and verify my idea, it >>>>>>>> also introduces additional benefits, e.g. speed up the engine >>>>>>>> packaging by reducing relocation classes. >>>>>>>> >>>>>>>> Looking forward to the community feedback and PR reviews. >>>>>>>> >>>>>>>> Once the idea is accepted, and [1] gets merged, I will start the >>>>>>>> kyuubi-shaded release voting. >>>>>>>> >>>>>>>> [1] https://github.com/apache/kyuubi-shaded/pull/20 >>>>>>>> [2] https://github.com/apache/kyuubi/pull/5783 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Cheng Pan >>>> >>
