For me, since we shade netty, there will be no conflict with other netty dependencies, so we can make sure the netty version we used is exactly the one we shaded in hbase-thirdparty.
If later we want to upgrade netty version in hbase-thirdparty, and it breaks the api, then we can release a new hbase-thirdparty, and upgrade the main hbase repo to depend on the new hbase-thirdparty version, and also modify the code in hbase accordingly. There will be no user visible affects. Anyway, let me have a try first. Even with netty's PaltformDependent class, since it still references sun.misc.Unsafe, maybe we will still get an compile error... Thanks. Andrew Purtell <apurt...@apache.org> 于2022年2月22日周二 08:08写道： > Thanks for that. I understand better what you are proposing now. It is > clear that you have thought it through. > > So then the only question I have is this: Do you think it is a concern that > Netty's PlatformDependent class is in their "io.netty.util.internal" > package? The 'internal' designation suggests to me they won't apply the > same care to not breaking users of it as they would for their public > (non-"internal") API. I noticed this earlier but had other questions at > that time. > > If it is a concern, we have the option of doing the same thing that Netty > has done with their PlatformDependent class as a new utility class of our > own in hbase-thirdparty. This may impose special requirements on building > hbase-thirdparty, or, at least, the module containing this wrapper, but it > is an option that would help us avoid external breaking changes. For your > consideration. > > On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang) <palomino...@gmail.com> > wrote: > > > What I proposed here is for solving the problem… > > > > The problem is that when using —release 8, javac will not export the > > jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a compile > > error. But at runtime if you export jdk.unsupported you can still use > > sun.misc.Unsafe. The related jdk issue said —release is only for public > API > > so not exporting jdk.unsupported is the expected behavior. > > > > So what I proposed here is to remove direct dependency on sun.misc.Unsafe > > in HBase code, so at compile time there will be no problem. Betty has a > > wrapper of Unsafe so I think we could have a try. At runtime if we have > > jdk.unsupported exported we could still actually make use of > > sun.misc.Unsafe. > > > > Feel free to correct me if I missed something or there are still > uncleared > > things. > > > > Thanks. > > > > > > > > Andrew Purtell <andrew.purt...@gmail.com>于2022年2月22日 周二00:56写道： > > > > > As Nick discovered ‘—release’ doesn’t work for version 8 anyway. > > > > > > > > > > On Feb 20, 2022, at 3:47 PM, 张铎 <palomino...@gmail.com> wrote: > > > > > > > > I know, with —release 8 ByteBuffer will not be an problem but then > > > > sun.misc.Unsafe can not be used any more right? So the problem here > is > > > how > > > > to make use of Unsafe with —release 8. What I proposed here is that > we > > > just > > > > do not use it, then we can make use of —release 8 to address the > > > ByteBuffer > > > > problem. > > > > > > > > Thanks. > > > > > > > > Andrew Purtell <apurt...@apache.org>于2022年2月21日 周一03:32写道： > > > > > > > >> The problem that leads us to consider '--release 8', or what it > would > > > >> promise if it worked, isn't Unsafe, it is ByteBuffer. > > > >> > > > > > > https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/ > > > >> . Although to your point Netty has ByteBuf... > > > >> > > > >> > > > >>> On Sun, Feb 20, 2022 at 5:23 AM 张铎(Duo Zhang) < > palomino...@gmail.com > > > > > > >>> wrote: > > > >>> > > > >>> Since we have shaded netty, I think we could make use of > > > >>> netty's PlatformDependent[1] class to avoid referencing Unsafe > > > directly, > > > >>> then there will be no problem for us to use the '--release 8' > option. > > > >>> > > > >>> If no big concerns, I will file an issue to remove all the > > > >> sun.misc.Unsafe > > > >>> references in our code base and make use of PlatformDependent from > > the > > > >>> shaded netty. > > > >>> > > > >>> Thanks. > > > >>> > > > >>> 1. > > > >>> > > > >>> > > > >> > > > > > > https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java > > > >>> > > > >>> 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年2月16日周三 14:12写道： > > > >>> > > > >>>> I think this could be done by some module tricks, where we build a > > > >>> special > > > >>>> module with -source 8 and -target 8.We have done similar things in > > the > > > >>> past > > > >>>> to have hadoop1-compat and hadoop2-compact. > > > >>>> > > > >>>> Let me have a try. > > > >>>> > > > >>>> Sean Busbey <bus...@apache.org> 于2022年2月16日周三 13:31写道： > > > >>>> > > > >>>>> Unfortunately if we want to keep jdk8 working we can't rely on > > > >> building > > > >>>>> with the jdk release target option on newer jdks. > > > >>>>> > > > >>>>> We ran into this recently in HBASE-25465 where a JDK bug came up. > > > >>>>> > > > >>>>> On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang) < > > palomino...@gmail.com > > > > > > > >>>>> wrote: > > > >>>>> > > > >>>>>> +1 on moving the minimum required java version for compiling > HBase > > > >> to > > > >>>>> Java > > > >>>>>> 11. But we could still use --release=8 to still support jdk8 at > > > >>> runtime. > > > >>>>>> > > > >>>>>> Java 8 is still widely used, and I believe this will last for > even > > > >>> more > > > >>>>>> years, as lots of big companies such as RedHat, Microsoft and > > Amazon > > > >>> are > > > >>>>>> still shipping new jdk8 releases, I do not think it is time to > > fully > > > >>>>> drop > > > >>>>>> the support of jdk8. > > > >>>>>> > > > >>>>>> Thanks. > > > >>>>>> > > > >>>>>> Sean Busbey <bus...@apache.org> 于2022年2月16日周三 09:57写道： > > > >>>>>> > > > >>>>>>> If we set the minJDK to 11 I believe that will effectively > remove > > > >>> jdk8 > > > >>>>>>> support, rather than "just" deprecate it. > > > >>>>>>> > > > >>>>>>> As we build out more robust testing I would be in favor of > > running > > > >>>>> less > > > >>>>>> of > > > >>>>>>> it on deprecated JDKs. > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> On Tue, Feb 15, 2022, 16:10 Josh Elser <els...@apache.org> > > wrote: > > > >>>>>>> > > > >>>>>>>> Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems > > > >>>>> reasonable > > > >>>>>> to > > > >>>>>>>> me. > > > >>>>>>>> > > > >>>>>>>> Gotta start pushing the issue somehow. > > > >>>>>>>> > > > >>>>>>>> On 2/15/22 1:47 PM, Sean Busbey wrote: > > > >>>>>>>>> Hi folks! > > > >>>>>>>>> > > > >>>>>>>>> It's been some time since we decided to stick to LTS JDK > > > >>> releases > > > >>>>> as > > > >>>>>> a > > > >>>>>>>> way > > > >>>>>>>>> of getting a handle on the JDK treadmill. > > > >>>>>>>>> > > > >>>>>>>>> What do folks think about deprecating JDK8? The openjdk8u > > > >>> project > > > >>>>> is > > > >>>>>>>> still > > > >>>>>>>>> going and there are commercial support options at least > > > >> through > > > >>>>> 2030. > > > >>>>>>>>> > > > >>>>>>>>> Deprecating it in HBase 3 would mean we could remove it in > > > >> HBase > > > >>>>> 4, > > > >>>>>> not > > > >>>>>>>>> that we would _have_ to remove it. The way I think about > > > >> likely > > > >>>>>> timing > > > >>>>>>> of > > > >>>>>>>>> these events goes like this: > > > >>>>>>>>> > > > >>>>>>>>> * HBase 2 started alphas in June 2017, betas in January 2018, > > > >>> and > > > >>>>>> came > > > >>>>>>>> out > > > >>>>>>>>> in April 2018 > > > >>>>>>>>> * HBase 3 started alphas in July 2021, and as of Feb 2022 we > > > >>>>> haven't > > > >>>>>>>>> discussed how close we are to our stated beta goals (upgrades > > > >>> from > > > >>>>>>> active > > > >>>>>>>>> 2.x releases and removal of not-ready features). > > > >>>>>>>>> > > > >>>>>>>>> Given the above, in the absence of us specifically pushing to > > > >>> roll > > > >>>>>>>> through > > > >>>>>>>>> major version numbers for some reason, I think a reasonably > > > >>>>>>> conservative > > > >>>>>>>>> estimate is for HBase 3 to arrive in late 2022 or early 2023 > > > >> and > > > >>>>> then > > > >>>>>>>> HBase > > > >>>>>>>>> 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems > > > >>>>> unlikely. > > > >>>>>>>>> > > > >>>>>>>>> That all said, our current reference guide section on java > > > >>>>> versions > > > >>>>>>> does > > > >>>>>>>>> not sound very confident about JDK11 support. > > > >>>>>>>>> > > > >>>>>>>>>> A Note on JDK11 * > > > >>>>>>>>>> Preliminary support for JDK11 is introduced with HBase > 2.3.0. > > > >>>>> This > > > >>>>>>>>> support is limited to > > > >>>>>>>>>> compilation and running the full test suite. There are open > > > >>>>>> questions > > > >>>>>>>>> regarding the runtime > > > >>>>>>>>>> compatibility of JDK11 with Apache ZooKeeper and Apache > > > >> Hadoop > > > >>>>>>>>> (HADOOP-15338). > > > >>>>>>>>>> Significantly, neither project has yet released a version > > > >> with > > > >>>>>>> explicit > > > >>>>>>>>> runtime support for > > > >>>>>>>>>> JDK11. The remaining known issues in HBase are catalogued in > > > >>>>>>>> HBASE-22972. > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> Since that blurb was written, Hadoop has added JDK11 support > > > >> [1] > > > >>>>> as > > > >>>>>> has > > > >>>>>>>>> ZooKeeper[2]. As a part of buttoning up our JDK11 support we > > > >>> could > > > >>>>>>> update > > > >>>>>>>>> our minimum supported versions of these projects to match > that > > > >>>>>> support. > > > >>>>>>>>> > > > >>>>>>>>> What do folks think? > > > >>>>>>>>> > > > >>>>>>>>> [1]: https://hadoop.apache.org/docs/r3.3.0/index.html > > > >>>>>>>>> [2]: > > > >>>>>>>>> > > > >>>>>>> > > > >>>>> > > > >>> > > > > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>>> > > > >>>> > > > >>> > > > >> > > > >> > > > >> -- > > > >> Best regards, > > > >> Andrew > > > >> > > > >> Unrest, ignorance distilled, nihilistic imbeciles - > > > >> It's what we’ve earned > > > >> Welcome, apocalypse, what’s taken you so long? > > > >> Bring us the fitting end that we’ve been counting on > > > >> - A23, Welcome, Apocalypse > > > >> > > > > > > > > -- > Best regards, > Andrew > > Unrest, ignorance distilled, nihilistic imbeciles - > It's what we’ve earned > Welcome, apocalypse, what’s taken you so long? > Bring us the fitting end that we’ve been counting on > - A23, Welcome, Apocalypse >
