>> The artifacts end with “-shaded” are for end users. > > This looks like an explanation for mistakes we've already made.
I don’t think they're mistakes, only facts. In practice, there are various ways to name this type of jar, we just picked one of them. There are some examples: - https://mvnrepository.com/artifact/com.datastax.oss/java-driver-core-shaded - https://mvnrepository.com/artifact/org.apache.zeppelin/zeppelin-interpreter-shaded - https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-spark-runtime-3.3 - https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client-api - https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client-runtime - https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kinesis-asl-assembly Currently, Kyuubi has two such modules - kyuubi-hive-jdbc-shaded - kyuubi-spark-authz-shaded The main purpose of this kind of artifact is to assemble all transitive dependencies into one jar, usually with proper third-party class relocation but WITHOUT changing any public API, e.g. for the JDBC driver, the public API is JDBC API, it also means that the user can migrate from `A` to `A-shaded` smoothly if only public API is consumed. > I've never been heard of such policy being discussed, documented. There hasn't been any specific topic of discussion about name policy before. It's impossible to discuss everything in detail. If someone thinks something matters, then we should discuss it. > Furthermore, since we've already posted them publicly in the maven repository, > how can we decide that they won't be used by end-users? Technically, we can not stop the user from using such shaded API directly. Personally, I think it’s a quite common practice, especially in Apache projects, thus it’s obvious to me, for example: - https://mvnrepository.com/artifact/org.apache.hbase.thirdparty/hbase-shaded-netty - https://mvnrepository.com/artifact/org.apache.hadoop.thirdparty/hadoop-shaded-guava - https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-jackson - https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-bundled-guava Also, I just follow the most popular name pattern `<project_name>-shaded-<thrid_party_name>` to name Kyuubi shaded artifacts. > Did we explicitly remove the public APIs from the original artifact？ No, in most cases, just simple relocation and repackaging of the original jar, with class overwriting if necessary. Thanks, Cheng Pan > On Nov 29, 2023, at 11:13, Kent Yao <y...@apache.org> wrote: > >> The artifacts end with “-shaded” are for end users. > > This looks like an explanation for mistakes we've already made. > > I've never been heard of such policy being discussed, documented. > > Furthermore, since we've already posted them publicly in the maven repository, > how can we decide that they won't be used by end-users? Did we explicitly > remove > the public APIs from the original artifact？ > > Kent > > Cheng Pan <pan3...@gmail.com> 于2023年11月29日周三 10:25写道： >> >> Kent >> >> They are for different purposes. >> >> The artifacts start with "kyuubi-shaded” is for the Kyuubi project's >> internal usage, see [1] and [2] >> >> The artifacts end with “-shaded” are for end users. >> >> [1] https://github.com/apache/kyuubi-shaded >> [2] >> https://mvnrepository.com/artifact/org.apache.kyuubi/kyuubi-shaded-zookeeper-34 >> >> Thanks, >> Cheng Pan >> >> >>> On Nov 28, 2023, at 18:36, Kent Yao <y...@apache.org> wrote: >>> >>> -1 for the arbitrary naming >>> >>> >>> FYI, >>> https://mvnrepository.com/artifact/org.apache.kyuubi/kyuubi-hive-jdbc-shaded >>> >>> XiDuo You <ulyssesyo...@gmail.com> 于2023年11月28日周二 18:03写道： >>>> >>>> +1, looks fine to me >>>> >>>> Zhen Wang <wangz...@apache.org> 于2023年11月28日周二 12:53写道： >>>>> >>>>> +1, I like this idea, it avoids the possibility of engine thrift class >>>>> conflicts in different environments. >>>>> >>>>> >>>>> Kind Regards, >>>>> Zhen Wang >>>>> >>>>> Cheng Pan <pan3...@gmail.com> 于2023年11月28日周二 01:00写道： >>>>>> >>>>>> Hi Kyuubi developers, >>>>>> >>>>>> Recently, I've been looking at the code of the Hive engine, one of my >>>>>> goals id to make it support multiple versions of Hive runtime, >>>>>> including >>>>>> - 3.1.3, which is the latest stable version of Apache Hive, already >>>>>> supported >>>>>> - 2.3.9, which is the latest stable version of Apache Hive 2.x, is >>>>>> adopted widely, including Apache Spark, Apache Flink >>>>>> - 2.1.1-cdh6.3.2, the latest free version of CDH, has lots of users >>>>>> >>>>>> When I tried to run the Hive engine built against Hive 3.x with Hive >>>>>> 2.3.9/2.1.1-cdh6.3.2 runtime, I encountered some thrift class conflict >>>>>> issues which were hard to resolve, thus I propose to create a >>>>>> pre-shaded hive-service-rpc to tackles such issue. >>>>>> >>>>>> I have created two PRs[1][2] to demonstrate and verify my idea, it >>>>>> also introduces additional benefits, e.g. speed up the engine >>>>>> packaging by reducing relocation classes. >>>>>> >>>>>> Looking forward to the community feedback and PR reviews. >>>>>> >>>>>> Once the idea is accepted, and [1] gets merged, I will start the >>>>>> kyuubi-shaded release voting. >>>>>> >>>>>> [1] https://github.com/apache/kyuubi-shaded/pull/20 >>>>>> [2] https://github.com/apache/kyuubi/pull/5783 >>>>>> >>>>>> Thanks, >>>>>> Cheng Pan >>
