Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

Filed HBASE-26793 for making a new hbase-thirdparty release.

Then will put the PR for HBASE-25465.

Thanks.

张铎(Duo Zhang)  于2022年3月2日周三 11:19写道：

> Good news, with HBASE-26781 in place[1], I successfully built hbase with
> java 11 --release 8. Of course I need to modify the hbase main code a bit,
> and also make use of netty's PlatformDependent for processing
> DirectByteBuffer.
>
> 1. https://github.com/apache/hbase-thirdparty/pull/79
>
> 张铎(Duo Zhang)  于2022年3月1日周二 22:07写道：
>
>> So after HBASE-26773, we still have another problem that we can not
>> use sun.nio.ch.DirectBuffer. The main usage is to get the address.
>>
>> Let me provide a new util method in the hbase-unsafe module.
>>
>> Andrew Purtell  于2022年2月26日周六 13:58写道：
>>
>>> That sounds good, looking forward to it. Please let me know if you want
>>> help.
>>>
>>> > On Feb 25, 2022, at 9:56 PM, 张铎  wrote:
>>> >
>>> > Filed HBASE-26773 and tried, the result is not very good. Netty's
>>> > PlatformDependent can not meet all the requirements.
>>> >
>>> > So now I prefer we have a new hbase-thirdparty-unsafe module to
>>> provide the
>>> > Unsafe ability without exposing sun.misc.Unsafe directly.
>>> >
>>> > Will give it a try.
>>> >
>>> > Thanks.
>>> >
>>> > Andrew Purtell  于2022年2月24日周四 10:46写道：
>>> >
>>> >> Ok, a bridge in one place would minimize the risk.
>>> >>
>>> >>> On Wed, Feb 23, 2022 at 6:07 PM 张铎(Duo Zhang) >> >
>>> >>> wrote:
>>> >>>
>>> >>> I think this is a reasonable concern, we could do this on our own.
>>> >>>
>>> >>> But checking the commit history of PlatformDependent[1], the API is
>>> >> pretty
>>> >>> stable, so I think the risk of hitting a security issue but we can
>>> not
>>> >>> upgrade due to API conflicts is very low. And if a security issue
>>> >> requires
>>> >>> big changes on PlatformDependent, then I assume we have to fix our
>>> own
>>> >>> version as well...
>>> >>>
>>> >>> And we will not spread PlatformDependent references everywhere in our
>>> >> code,
>>> >>> we will still have a bridge, like the current UnsafeAccess. So if
>>> later
>>> >> we
>>> >>> decide to implement our own version, it will not introduce a big
>>> impact
>>> >> to
>>> >>> our code base.
>>> >>>
>>> >>> Thanks.
>>> >>>
>>> >>>
>>> >>> [1]
>>> >>>
>>> >>>
>>> >>
>>> https://github.com/netty/netty/commits/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
>>> >>>
>>> >>> Andrew Purtell  于2022年2月24日周四 01:12写道：
>>> >>>
>>>  We often upgrade netty after there is a CVE announcement. This
>>> >> produces a
>>>  pressure to upgrade in production. Use of an internal API is not a
>>> good
>>>  practice because it can cause friction because of breaking changes
>>> when
>>>  there is a need to upgrade all of a sudden. This is painful and
>>> >> hopefully
>>>  we can exclude it as a possibility now when coming up with the
>>> plan. We
>>> >>> can
>>>  do the same thing they do in our own hbase-thirdparty without taking
>>> >> the
>>>  dependency and achieve the same goal, right?
>>> 
>>>  On Mon, Feb 21, 2022 at 7:35 PM 张铎(Duo Zhang) <
>>> palomino...@gmail.com>
>>>  wrote:
>>> 
>>> > For me, since we shade netty, there will be no conflict with other
>>> >>> netty
>>> > dependencies, so we can make sure the netty version we used is
>>> >> exactly
>>>  the
>>> > one we shaded in hbase-thirdparty.
>>> >
>>> > If later we want to upgrade netty version in hbase-thirdparty, and
>>> it
>>> > breaks the api, then we can release a new hbase-thirdparty, and
>>> >> upgrade
>>>  the
>>> > main hbase repo to depend on the new hbase-thirdparty version, and
>>> >> also
>>> > modify the code in hbase accordingly. There will be no user visible
>>> > affects.
>>> >
>>> > Anyway, let me have a try first. Even with netty's
>>> PaltformDependent
>>>  class,
>>> > since it still references sun.misc.Unsafe, maybe we will still get
>>> an
>>> > compile error...
>>> >
>>> > Thanks.
>>> >
>>> > Andrew Purtell  于2022年2月22日周二 08:08写道：
>>> >
>>> >> Thanks for that. I understand better what you are proposing now.
>>> It
>>> >>> is
>>> >> clear that you have thought it through.
>>> >>
>>> >> So then the only question I have is this: Do you think it is a
>>> >>> concern
>>> > that
>>> >> Netty's PlatformDependent class is in their
>>> >> "io.netty.util.internal"
>>> >> package? The 'internal' designation suggests to me they won't
>>> apply
>>> >>> the
>>> >> same care to not breaking users of it as they would for their
>>> >> public
>>> >> (non-"internal") API. I noticed this earlier but had other
>>> >> questions
>>> >>> at
>>> >> that time.
>>> >>
>>> >> If it is a concern, we have the option of doing the same thing
>>> that
>>>  Netty
>>> >> has done with their PlatformDependent class as a new utility class
>>> >> of
>>>  our
>>> >> own in hbase-thirdparty. This may 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

Good news, with HBASE-26781 in place[1], I successfully built hbase with
java 11 --release 8. Of course I need to modify the hbase main code a bit,
and also make use of netty's PlatformDependent for processing
DirectByteBuffer.

1. https://github.com/apache/hbase-thirdparty/pull/79

张铎(Duo Zhang)  于2022年3月1日周二 22:07写道：

> So after HBASE-26773, we still have another problem that we can not
> use sun.nio.ch.DirectBuffer. The main usage is to get the address.
>
> Let me provide a new util method in the hbase-unsafe module.
>
> Andrew Purtell  于2022年2月26日周六 13:58写道：
>
>> That sounds good, looking forward to it. Please let me know if you want
>> help.
>>
>> > On Feb 25, 2022, at 9:56 PM, 张铎  wrote:
>> >
>> > Filed HBASE-26773 and tried, the result is not very good. Netty's
>> > PlatformDependent can not meet all the requirements.
>> >
>> > So now I prefer we have a new hbase-thirdparty-unsafe module to provide
>> the
>> > Unsafe ability without exposing sun.misc.Unsafe directly.
>> >
>> > Will give it a try.
>> >
>> > Thanks.
>> >
>> > Andrew Purtell  于2022年2月24日周四 10:46写道：
>> >
>> >> Ok, a bridge in one place would minimize the risk.
>> >>
>> >>> On Wed, Feb 23, 2022 at 6:07 PM 张铎(Duo Zhang) 
>> >>> wrote:
>> >>>
>> >>> I think this is a reasonable concern, we could do this on our own.
>> >>>
>> >>> But checking the commit history of PlatformDependent[1], the API is
>> >> pretty
>> >>> stable, so I think the risk of hitting a security issue but we can not
>> >>> upgrade due to API conflicts is very low. And if a security issue
>> >> requires
>> >>> big changes on PlatformDependent, then I assume we have to fix our own
>> >>> version as well...
>> >>>
>> >>> And we will not spread PlatformDependent references everywhere in our
>> >> code,
>> >>> we will still have a bridge, like the current UnsafeAccess. So if
>> later
>> >> we
>> >>> decide to implement our own version, it will not introduce a big
>> impact
>> >> to
>> >>> our code base.
>> >>>
>> >>> Thanks.
>> >>>
>> >>>
>> >>> [1]
>> >>>
>> >>>
>> >>
>> https://github.com/netty/netty/commits/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
>> >>>
>> >>> Andrew Purtell  于2022年2月24日周四 01:12写道：
>> >>>
>>  We often upgrade netty after there is a CVE announcement. This
>> >> produces a
>>  pressure to upgrade in production. Use of an internal API is not a
>> good
>>  practice because it can cause friction because of breaking changes
>> when
>>  there is a need to upgrade all of a sudden. This is painful and
>> >> hopefully
>>  we can exclude it as a possibility now when coming up with the plan.
>> We
>> >>> can
>>  do the same thing they do in our own hbase-thirdparty without taking
>> >> the
>>  dependency and achieve the same goal, right?
>> 
>>  On Mon, Feb 21, 2022 at 7:35 PM 张铎(Duo Zhang) > >
>>  wrote:
>> 
>> > For me, since we shade netty, there will be no conflict with other
>> >>> netty
>> > dependencies, so we can make sure the netty version we used is
>> >> exactly
>>  the
>> > one we shaded in hbase-thirdparty.
>> >
>> > If later we want to upgrade netty version in hbase-thirdparty, and
>> it
>> > breaks the api, then we can release a new hbase-thirdparty, and
>> >> upgrade
>>  the
>> > main hbase repo to depend on the new hbase-thirdparty version, and
>> >> also
>> > modify the code in hbase accordingly. There will be no user visible
>> > affects.
>> >
>> > Anyway, let me have a try first. Even with netty's PaltformDependent
>>  class,
>> > since it still references sun.misc.Unsafe, maybe we will still get
>> an
>> > compile error...
>> >
>> > Thanks.
>> >
>> > Andrew Purtell  于2022年2月22日周二 08:08写道：
>> >
>> >> Thanks for that. I understand better what you are proposing now. It
>> >>> is
>> >> clear that you have thought it through.
>> >>
>> >> So then the only question I have is this: Do you think it is a
>> >>> concern
>> > that
>> >> Netty's PlatformDependent class is in their
>> >> "io.netty.util.internal"
>> >> package? The 'internal' designation suggests to me they won't apply
>> >>> the
>> >> same care to not breaking users of it as they would for their
>> >> public
>> >> (non-"internal") API. I noticed this earlier but had other
>> >> questions
>> >>> at
>> >> that time.
>> >>
>> >> If it is a concern, we have the option of doing the same thing that
>>  Netty
>> >> has done with their PlatformDependent class as a new utility class
>> >> of
>>  our
>> >> own in hbase-thirdparty. This may impose special requirements on
>>  building
>> >> hbase-thirdparty, or, at least, the module containing this wrapper,
>> >>> but
>> > it
>> >> is an option that would help us avoid external breaking changes.
>> >> For
>>  your
>> >> consideration.
>> >>
>> >> On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang) <
>> >> 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

So after HBASE-26773, we still have another problem that we can not
use sun.nio.ch.DirectBuffer. The main usage is to get the address.

Let me provide a new util method in the hbase-unsafe module.

Andrew Purtell  于2022年2月26日周六 13:58写道：

> That sounds good, looking forward to it. Please let me know if you want
> help.
>
> > On Feb 25, 2022, at 9:56 PM, 张铎  wrote:
> >
> > Filed HBASE-26773 and tried, the result is not very good. Netty's
> > PlatformDependent can not meet all the requirements.
> >
> > So now I prefer we have a new hbase-thirdparty-unsafe module to provide
> the
> > Unsafe ability without exposing sun.misc.Unsafe directly.
> >
> > Will give it a try.
> >
> > Thanks.
> >
> > Andrew Purtell  于2022年2月24日周四 10:46写道：
> >
> >> Ok, a bridge in one place would minimize the risk.
> >>
> >>> On Wed, Feb 23, 2022 at 6:07 PM 张铎(Duo Zhang) 
> >>> wrote:
> >>>
> >>> I think this is a reasonable concern, we could do this on our own.
> >>>
> >>> But checking the commit history of PlatformDependent[1], the API is
> >> pretty
> >>> stable, so I think the risk of hitting a security issue but we can not
> >>> upgrade due to API conflicts is very low. And if a security issue
> >> requires
> >>> big changes on PlatformDependent, then I assume we have to fix our own
> >>> version as well...
> >>>
> >>> And we will not spread PlatformDependent references everywhere in our
> >> code,
> >>> we will still have a bridge, like the current UnsafeAccess. So if later
> >> we
> >>> decide to implement our own version, it will not introduce a big impact
> >> to
> >>> our code base.
> >>>
> >>> Thanks.
> >>>
> >>>
> >>> [1]
> >>>
> >>>
> >>
> https://github.com/netty/netty/commits/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
> >>>
> >>> Andrew Purtell  于2022年2月24日周四 01:12写道：
> >>>
>  We often upgrade netty after there is a CVE announcement. This
> >> produces a
>  pressure to upgrade in production. Use of an internal API is not a
> good
>  practice because it can cause friction because of breaking changes
> when
>  there is a need to upgrade all of a sudden. This is painful and
> >> hopefully
>  we can exclude it as a possibility now when coming up with the plan.
> We
> >>> can
>  do the same thing they do in our own hbase-thirdparty without taking
> >> the
>  dependency and achieve the same goal, right?
> 
>  On Mon, Feb 21, 2022 at 7:35 PM 张铎(Duo Zhang) 
>  wrote:
> 
> > For me, since we shade netty, there will be no conflict with other
> >>> netty
> > dependencies, so we can make sure the netty version we used is
> >> exactly
>  the
> > one we shaded in hbase-thirdparty.
> >
> > If later we want to upgrade netty version in hbase-thirdparty, and it
> > breaks the api, then we can release a new hbase-thirdparty, and
> >> upgrade
>  the
> > main hbase repo to depend on the new hbase-thirdparty version, and
> >> also
> > modify the code in hbase accordingly. There will be no user visible
> > affects.
> >
> > Anyway, let me have a try first. Even with netty's PaltformDependent
>  class,
> > since it still references sun.misc.Unsafe, maybe we will still get an
> > compile error...
> >
> > Thanks.
> >
> > Andrew Purtell  于2022年2月22日周二 08:08写道：
> >
> >> Thanks for that. I understand better what you are proposing now. It
> >>> is
> >> clear that you have thought it through.
> >>
> >> So then the only question I have is this: Do you think it is a
> >>> concern
> > that
> >> Netty's PlatformDependent class is in their
> >> "io.netty.util.internal"
> >> package? The 'internal' designation suggests to me they won't apply
> >>> the
> >> same care to not breaking users of it as they would for their
> >> public
> >> (non-"internal") API. I noticed this earlier but had other
> >> questions
> >>> at
> >> that time.
> >>
> >> If it is a concern, we have the option of doing the same thing that
>  Netty
> >> has done with their PlatformDependent class as a new utility class
> >> of
>  our
> >> own in hbase-thirdparty. This may impose special requirements on
>  building
> >> hbase-thirdparty, or, at least, the module containing this wrapper,
> >>> but
> > it
> >> is an option that would help us avoid external breaking changes.
> >> For
>  your
> >> consideration.
> >>
> >> On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang) <
> >> palomino...@gmail.com
> 
> >> wrote:
> >>
> >>> What I proposed here is for solving the problem…
> >>>
> >>> The problem is that when using —release 8, javac will not export
> >>> the
> >>> jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a
> > compile
> >>> error. But at runtime if you export jdk.unsupported you can still
> >>> use
> >>> sun.misc.Unsafe. The related jdk issue said —release is only for
>  public
> >> 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Andrew Purtell]

That sounds good, looking forward to it. Please let me know if you want help. 

> On Feb 25, 2022, at 9:56 PM, 张铎  wrote:
> 
> Filed HBASE-26773 and tried, the result is not very good. Netty's
> PlatformDependent can not meet all the requirements.
> 
> So now I prefer we have a new hbase-thirdparty-unsafe module to provide the
> Unsafe ability without exposing sun.misc.Unsafe directly.
> 
> Will give it a try.
> 
> Thanks.
> 
> Andrew Purtell  于2022年2月24日周四 10:46写道：
> 
>> Ok, a bridge in one place would minimize the risk.
>> 
>>> On Wed, Feb 23, 2022 at 6:07 PM 张铎(Duo Zhang) 
>>> wrote:
>>> 
>>> I think this is a reasonable concern, we could do this on our own.
>>> 
>>> But checking the commit history of PlatformDependent[1], the API is
>> pretty
>>> stable, so I think the risk of hitting a security issue but we can not
>>> upgrade due to API conflicts is very low. And if a security issue
>> requires
>>> big changes on PlatformDependent, then I assume we have to fix our own
>>> version as well...
>>> 
>>> And we will not spread PlatformDependent references everywhere in our
>> code,
>>> we will still have a bridge, like the current UnsafeAccess. So if later
>> we
>>> decide to implement our own version, it will not introduce a big impact
>> to
>>> our code base.
>>> 
>>> Thanks.
>>> 
>>> 
>>> [1]
>>> 
>>> 
>> https://github.com/netty/netty/commits/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
>>> 
>>> Andrew Purtell  于2022年2月24日周四 01:12写道：
>>> 
 We often upgrade netty after there is a CVE announcement. This
>> produces a
 pressure to upgrade in production. Use of an internal API is not a good
 practice because it can cause friction because of breaking changes when
 there is a need to upgrade all of a sudden. This is painful and
>> hopefully
 we can exclude it as a possibility now when coming up with the plan. We
>>> can
 do the same thing they do in our own hbase-thirdparty without taking
>> the
 dependency and achieve the same goal, right?
 
 On Mon, Feb 21, 2022 at 7:35 PM 张铎(Duo Zhang) 
 wrote:
 
> For me, since we shade netty, there will be no conflict with other
>>> netty
> dependencies, so we can make sure the netty version we used is
>> exactly
 the
> one we shaded in hbase-thirdparty.
> 
> If later we want to upgrade netty version in hbase-thirdparty, and it
> breaks the api, then we can release a new hbase-thirdparty, and
>> upgrade
 the
> main hbase repo to depend on the new hbase-thirdparty version, and
>> also
> modify the code in hbase accordingly. There will be no user visible
> affects.
> 
> Anyway, let me have a try first. Even with netty's PaltformDependent
 class,
> since it still references sun.misc.Unsafe, maybe we will still get an
> compile error...
> 
> Thanks.
> 
> Andrew Purtell  于2022年2月22日周二 08:08写道：
> 
>> Thanks for that. I understand better what you are proposing now. It
>>> is
>> clear that you have thought it through.
>> 
>> So then the only question I have is this: Do you think it is a
>>> concern
> that
>> Netty's PlatformDependent class is in their
>> "io.netty.util.internal"
>> package? The 'internal' designation suggests to me they won't apply
>>> the
>> same care to not breaking users of it as they would for their
>> public
>> (non-"internal") API. I noticed this earlier but had other
>> questions
>>> at
>> that time.
>> 
>> If it is a concern, we have the option of doing the same thing that
 Netty
>> has done with their PlatformDependent class as a new utility class
>> of
 our
>> own in hbase-thirdparty. This may impose special requirements on
 building
>> hbase-thirdparty, or, at least, the module containing this wrapper,
>>> but
> it
>> is an option that would help us avoid external breaking changes.
>> For
 your
>> consideration.
>> 
>> On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang) <
>> palomino...@gmail.com
 
>> wrote:
>> 
>>> What I proposed here is for solving the problem…
>>> 
>>> The problem is that when using —release 8, javac will not export
>>> the
>>> jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a
> compile
>>> error. But at runtime if you export jdk.unsupported you can still
>>> use
>>> sun.misc.Unsafe. The related jdk issue said —release is only for
 public
>> API
>>> so not exporting jdk.unsupported is the expected behavior.
>>> 
>>> So what I proposed here is to remove direct dependency on
> sun.misc.Unsafe
>>> in HBase code, so at compile time there will be no problem. Betty
 has a
>>> wrapper of Unsafe so I think we could have a try. At runtime if
>> we
 have
>>> jdk.unsupported exported we could still actually make use of
>>> sun.misc.Unsafe.
>>> 
>>> Feel free to correct me if I missed 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

Filed HBASE-26773 and tried, the result is not very good. Netty's
PlatformDependent can not meet all the requirements.

So now I prefer we have a new hbase-thirdparty-unsafe module to provide the
Unsafe ability without exposing sun.misc.Unsafe directly.

Will give it a try.

Thanks.

Andrew Purtell  于2022年2月24日周四 10:46写道：

> Ok, a bridge in one place would minimize the risk.
>
> On Wed, Feb 23, 2022 at 6:07 PM 张铎(Duo Zhang) 
> wrote:
>
> > I think this is a reasonable concern, we could do this on our own.
> >
> > But checking the commit history of PlatformDependent[1], the API is
> pretty
> > stable, so I think the risk of hitting a security issue but we can not
> > upgrade due to API conflicts is very low. And if a security issue
> requires
> > big changes on PlatformDependent, then I assume we have to fix our own
> > version as well...
> >
> > And we will not spread PlatformDependent references everywhere in our
> code,
> > we will still have a bridge, like the current UnsafeAccess. So if later
> we
> > decide to implement our own version, it will not introduce a big impact
> to
> > our code base.
> >
> > Thanks.
> >
> >
> > [1]
> >
> >
> https://github.com/netty/netty/commits/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
> >
> > Andrew Purtell  于2022年2月24日周四 01:12写道：
> >
> > > We often upgrade netty after there is a CVE announcement. This
> produces a
> > > pressure to upgrade in production. Use of an internal API is not a good
> > > practice because it can cause friction because of breaking changes when
> > > there is a need to upgrade all of a sudden. This is painful and
> hopefully
> > > we can exclude it as a possibility now when coming up with the plan. We
> > can
> > > do the same thing they do in our own hbase-thirdparty without taking
> the
> > > dependency and achieve the same goal, right?
> > >
> > > On Mon, Feb 21, 2022 at 7:35 PM 张铎(Duo Zhang) 
> > > wrote:
> > >
> > > > For me, since we shade netty, there will be no conflict with other
> > netty
> > > > dependencies, so we can make sure the netty version we used is
> exactly
> > > the
> > > > one we shaded in hbase-thirdparty.
> > > >
> > > > If later we want to upgrade netty version in hbase-thirdparty, and it
> > > > breaks the api, then we can release a new hbase-thirdparty, and
> upgrade
> > > the
> > > > main hbase repo to depend on the new hbase-thirdparty version, and
> also
> > > > modify the code in hbase accordingly. There will be no user visible
> > > > affects.
> > > >
> > > > Anyway, let me have a try first. Even with netty's PaltformDependent
> > > class,
> > > > since it still references sun.misc.Unsafe, maybe we will still get an
> > > > compile error...
> > > >
> > > > Thanks.
> > > >
> > > > Andrew Purtell  于2022年2月22日周二 08:08写道：
> > > >
> > > > > Thanks for that. I understand better what you are proposing now. It
> > is
> > > > > clear that you have thought it through.
> > > > >
> > > > > So then the only question I have is this: Do you think it is a
> > concern
> > > > that
> > > > > Netty's PlatformDependent class is in their
> "io.netty.util.internal"
> > > > > package? The 'internal' designation suggests to me they won't apply
> > the
> > > > > same care to not breaking users of it as they would for their
> public
> > > > > (non-"internal") API. I noticed this earlier but had other
> questions
> > at
> > > > > that time.
> > > > >
> > > > > If it is a concern, we have the option of doing the same thing that
> > > Netty
> > > > > has done with their PlatformDependent class as a new utility class
> of
> > > our
> > > > > own in hbase-thirdparty. This may impose special requirements on
> > > building
> > > > > hbase-thirdparty, or, at least, the module containing this wrapper,
> > but
> > > > it
> > > > > is an option that would help us avoid external breaking changes.
> For
> > > your
> > > > > consideration.
> > > > >
> > > > > On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang) <
> palomino...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > What I proposed here is for solving the problem…
> > > > > >
> > > > > > The problem is that when using —release 8, javac will not export
> > the
> > > > > > jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a
> > > > compile
> > > > > > error. But at runtime if you export jdk.unsupported you can still
> > use
> > > > > > sun.misc.Unsafe. The related jdk issue said —release is only for
> > > public
> > > > > API
> > > > > > so not exporting jdk.unsupported is the expected behavior.
> > > > > >
> > > > > > So what I proposed here is to remove direct dependency on
> > > > sun.misc.Unsafe
> > > > > > in HBase code, so at compile time there will be no problem. Betty
> > > has a
> > > > > > wrapper of Unsafe so I think we could have a try. At runtime if
> we
> > > have
> > > > > > jdk.unsupported exported we could still actually make use of
> > > > > > sun.misc.Unsafe.
> > > > > >
> > > > > > Feel free to correct me if I missed something or there 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Andrew Purtell]

Ok, a bridge in one place would minimize the risk.

On Wed, Feb 23, 2022 at 6:07 PM 张铎(Duo Zhang)  wrote:

> I think this is a reasonable concern, we could do this on our own.
>
> But checking the commit history of PlatformDependent[1], the API is pretty
> stable, so I think the risk of hitting a security issue but we can not
> upgrade due to API conflicts is very low. And if a security issue requires
> big changes on PlatformDependent, then I assume we have to fix our own
> version as well...
>
> And we will not spread PlatformDependent references everywhere in our code,
> we will still have a bridge, like the current UnsafeAccess. So if later we
> decide to implement our own version, it will not introduce a big impact to
> our code base.
>
> Thanks.
>
>
> [1]
>
> https://github.com/netty/netty/commits/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
>
> Andrew Purtell  于2022年2月24日周四 01:12写道：
>
> > We often upgrade netty after there is a CVE announcement. This produces a
> > pressure to upgrade in production. Use of an internal API is not a good
> > practice because it can cause friction because of breaking changes when
> > there is a need to upgrade all of a sudden. This is painful and hopefully
> > we can exclude it as a possibility now when coming up with the plan. We
> can
> > do the same thing they do in our own hbase-thirdparty without taking the
> > dependency and achieve the same goal, right?
> >
> > On Mon, Feb 21, 2022 at 7:35 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > For me, since we shade netty, there will be no conflict with other
> netty
> > > dependencies, so we can make sure the netty version we used is exactly
> > the
> > > one we shaded in hbase-thirdparty.
> > >
> > > If later we want to upgrade netty version in hbase-thirdparty, and it
> > > breaks the api, then we can release a new hbase-thirdparty, and upgrade
> > the
> > > main hbase repo to depend on the new hbase-thirdparty version, and also
> > > modify the code in hbase accordingly. There will be no user visible
> > > affects.
> > >
> > > Anyway, let me have a try first. Even with netty's PaltformDependent
> > class,
> > > since it still references sun.misc.Unsafe, maybe we will still get an
> > > compile error...
> > >
> > > Thanks.
> > >
> > > Andrew Purtell  于2022年2月22日周二 08:08写道：
> > >
> > > > Thanks for that. I understand better what you are proposing now. It
> is
> > > > clear that you have thought it through.
> > > >
> > > > So then the only question I have is this: Do you think it is a
> concern
> > > that
> > > > Netty's PlatformDependent class is in their "io.netty.util.internal"
> > > > package? The 'internal' designation suggests to me they won't apply
> the
> > > > same care to not breaking users of it as they would for their public
> > > > (non-"internal") API. I noticed this earlier but had other questions
> at
> > > > that time.
> > > >
> > > > If it is a concern, we have the option of doing the same thing that
> > Netty
> > > > has done with their PlatformDependent class as a new utility class of
> > our
> > > > own in hbase-thirdparty. This may impose special requirements on
> > building
> > > > hbase-thirdparty, or, at least, the module containing this wrapper,
> but
> > > it
> > > > is an option that would help us avoid external breaking changes. For
> > your
> > > > consideration.
> > > >
> > > > On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang)  >
> > > > wrote:
> > > >
> > > > > What I proposed here is for solving the problem…
> > > > >
> > > > > The problem is that when using —release 8, javac will not export
> the
> > > > > jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a
> > > compile
> > > > > error. But at runtime if you export jdk.unsupported you can still
> use
> > > > > sun.misc.Unsafe. The related jdk issue said —release is only for
> > public
> > > > API
> > > > > so not exporting jdk.unsupported is the expected behavior.
> > > > >
> > > > > So what I proposed here is to remove direct dependency on
> > > sun.misc.Unsafe
> > > > > in HBase code, so at compile time there will be no problem. Betty
> > has a
> > > > > wrapper of Unsafe so I think we could have a try. At runtime if we
> > have
> > > > > jdk.unsupported exported we could still actually make use of
> > > > > sun.misc.Unsafe.
> > > > >
> > > > > Feel free to correct me if I missed something or there are still
> > > > uncleared
> > > > > things.
> > > > >
> > > > > Thanks.
> > > > >
> > > > >
> > > > >
> > > > > Andrew Purtell 于2022年2月22日 周二00:56写道：
> > > > >
> > > > > > As Nick discovered ‘—release’ doesn’t work for version 8 anyway.
> > > > > >
> > > > > >
> > > > > > > On Feb 20, 2022, at 3:47 PM, 张铎  wrote:
> > > > > > >
> > > > > > > I know, with —release 8 ByteBuffer will not be an problem but
> > then
> > > > > > > sun.misc.Unsafe can not be used any more right? So the problem
> > here
> > > > is
> > > > > > how
> > > > > > > to make use of Unsafe with —release 8. What I proposed here is
> > that
> 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

I think this is a reasonable concern, we could do this on our own.

But checking the commit history of PlatformDependent[1], the API is pretty
stable, so I think the risk of hitting a security issue but we can not
upgrade due to API conflicts is very low. And if a security issue requires
big changes on PlatformDependent, then I assume we have to fix our own
version as well...

And we will not spread PlatformDependent references everywhere in our code,
we will still have a bridge, like the current UnsafeAccess. So if later we
decide to implement our own version, it will not introduce a big impact to
our code base.

Thanks.


[1]
https://github.com/netty/netty/commits/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java

Andrew Purtell  于2022年2月24日周四 01:12写道：

> We often upgrade netty after there is a CVE announcement. This produces a
> pressure to upgrade in production. Use of an internal API is not a good
> practice because it can cause friction because of breaking changes when
> there is a need to upgrade all of a sudden. This is painful and hopefully
> we can exclude it as a possibility now when coming up with the plan. We can
> do the same thing they do in our own hbase-thirdparty without taking the
> dependency and achieve the same goal, right?
>
> On Mon, Feb 21, 2022 at 7:35 PM 张铎(Duo Zhang) 
> wrote:
>
> > For me, since we shade netty, there will be no conflict with other netty
> > dependencies, so we can make sure the netty version we used is exactly
> the
> > one we shaded in hbase-thirdparty.
> >
> > If later we want to upgrade netty version in hbase-thirdparty, and it
> > breaks the api, then we can release a new hbase-thirdparty, and upgrade
> the
> > main hbase repo to depend on the new hbase-thirdparty version, and also
> > modify the code in hbase accordingly. There will be no user visible
> > affects.
> >
> > Anyway, let me have a try first. Even with netty's PaltformDependent
> class,
> > since it still references sun.misc.Unsafe, maybe we will still get an
> > compile error...
> >
> > Thanks.
> >
> > Andrew Purtell  于2022年2月22日周二 08:08写道：
> >
> > > Thanks for that. I understand better what you are proposing now. It is
> > > clear that you have thought it through.
> > >
> > > So then the only question I have is this: Do you think it is a concern
> > that
> > > Netty's PlatformDependent class is in their "io.netty.util.internal"
> > > package? The 'internal' designation suggests to me they won't apply the
> > > same care to not breaking users of it as they would for their public
> > > (non-"internal") API. I noticed this earlier but had other questions at
> > > that time.
> > >
> > > If it is a concern, we have the option of doing the same thing that
> Netty
> > > has done with their PlatformDependent class as a new utility class of
> our
> > > own in hbase-thirdparty. This may impose special requirements on
> building
> > > hbase-thirdparty, or, at least, the module containing this wrapper, but
> > it
> > > is an option that would help us avoid external breaking changes. For
> your
> > > consideration.
> > >
> > > On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang) 
> > > wrote:
> > >
> > > > What I proposed here is for solving the problem…
> > > >
> > > > The problem is that when using —release 8, javac will not export the
> > > > jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a
> > compile
> > > > error. But at runtime if you export jdk.unsupported you can still use
> > > > sun.misc.Unsafe. The related jdk issue said —release is only for
> public
> > > API
> > > > so not exporting jdk.unsupported is the expected behavior.
> > > >
> > > > So what I proposed here is to remove direct dependency on
> > sun.misc.Unsafe
> > > > in HBase code, so at compile time there will be no problem. Betty
> has a
> > > > wrapper of Unsafe so I think we could have a try. At runtime if we
> have
> > > > jdk.unsupported exported we could still actually make use of
> > > > sun.misc.Unsafe.
> > > >
> > > > Feel free to correct me if I missed something or there are still
> > > uncleared
> > > > things.
> > > >
> > > > Thanks.
> > > >
> > > >
> > > >
> > > > Andrew Purtell 于2022年2月22日 周二00:56写道：
> > > >
> > > > > As Nick discovered ‘—release’ doesn’t work for version 8 anyway.
> > > > >
> > > > >
> > > > > > On Feb 20, 2022, at 3:47 PM, 张铎  wrote:
> > > > > >
> > > > > > I know, with —release 8 ByteBuffer will not be an problem but
> then
> > > > > > sun.misc.Unsafe can not be used any more right? So the problem
> here
> > > is
> > > > > how
> > > > > > to make use of Unsafe with —release 8. What I proposed here is
> that
> > > we
> > > > > just
> > > > > > do not use it, then we can make use of —release 8 to address the
> > > > > ByteBuffer
> > > > > > problem.
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > Andrew Purtell 于2022年2月21日 周一03:32写道：
> > > > > >
> > > > > >> The problem that leads us to consider '--release 8', or what it
> > > would
> > > > > >> promise if it 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Andrew Purtell]

We often upgrade netty after there is a CVE announcement. This produces a
pressure to upgrade in production. Use of an internal API is not a good
practice because it can cause friction because of breaking changes when
there is a need to upgrade all of a sudden. This is painful and hopefully
we can exclude it as a possibility now when coming up with the plan. We can
do the same thing they do in our own hbase-thirdparty without taking the
dependency and achieve the same goal, right?

On Mon, Feb 21, 2022 at 7:35 PM 张铎(Duo Zhang)  wrote:

> For me, since we shade netty, there will be no conflict with other netty
> dependencies, so we can make sure the netty version we used is exactly the
> one we shaded in hbase-thirdparty.
>
> If later we want to upgrade netty version in hbase-thirdparty, and it
> breaks the api, then we can release a new hbase-thirdparty, and upgrade the
> main hbase repo to depend on the new hbase-thirdparty version, and also
> modify the code in hbase accordingly. There will be no user visible
> affects.
>
> Anyway, let me have a try first. Even with netty's PaltformDependent class,
> since it still references sun.misc.Unsafe, maybe we will still get an
> compile error...
>
> Thanks.
>
> Andrew Purtell  于2022年2月22日周二 08:08写道：
>
> > Thanks for that. I understand better what you are proposing now. It is
> > clear that you have thought it through.
> >
> > So then the only question I have is this: Do you think it is a concern
> that
> > Netty's PlatformDependent class is in their "io.netty.util.internal"
> > package? The 'internal' designation suggests to me they won't apply the
> > same care to not breaking users of it as they would for their public
> > (non-"internal") API. I noticed this earlier but had other questions at
> > that time.
> >
> > If it is a concern, we have the option of doing the same thing that Netty
> > has done with their PlatformDependent class as a new utility class of our
> > own in hbase-thirdparty. This may impose special requirements on building
> > hbase-thirdparty, or, at least, the module containing this wrapper, but
> it
> > is an option that would help us avoid external breaking changes. For your
> > consideration.
> >
> > On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > What I proposed here is for solving the problem…
> > >
> > > The problem is that when using —release 8, javac will not export the
> > > jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a
> compile
> > > error. But at runtime if you export jdk.unsupported you can still use
> > > sun.misc.Unsafe. The related jdk issue said —release is only for public
> > API
> > > so not exporting jdk.unsupported is the expected behavior.
> > >
> > > So what I proposed here is to remove direct dependency on
> sun.misc.Unsafe
> > > in HBase code, so at compile time there will be no problem. Betty has a
> > > wrapper of Unsafe so I think we could have a try. At runtime if we have
> > > jdk.unsupported exported we could still actually make use of
> > > sun.misc.Unsafe.
> > >
> > > Feel free to correct me if I missed something or there are still
> > uncleared
> > > things.
> > >
> > > Thanks.
> > >
> > >
> > >
> > > Andrew Purtell 于2022年2月22日 周二00:56写道：
> > >
> > > > As Nick discovered ‘—release’ doesn’t work for version 8 anyway.
> > > >
> > > >
> > > > > On Feb 20, 2022, at 3:47 PM, 张铎  wrote:
> > > > >
> > > > > I know, with —release 8 ByteBuffer will not be an problem but then
> > > > > sun.misc.Unsafe can not be used any more right? So the problem here
> > is
> > > > how
> > > > > to make use of Unsafe with —release 8. What I proposed here is that
> > we
> > > > just
> > > > > do not use it, then we can make use of —release 8 to address the
> > > > ByteBuffer
> > > > > problem.
> > > > >
> > > > > Thanks.
> > > > >
> > > > > Andrew Purtell 于2022年2月21日 周一03:32写道：
> > > > >
> > > > >> The problem that leads us to consider '--release 8', or what it
> > would
> > > > >> promise if it worked, isn't Unsafe, it is ByteBuffer.
> > > > >>
> > > >
> > >
> >
> https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
> > > > >> . Although to your point Netty has ByteBuf...
> > > > >>
> > > > >>
> > > > >>> On Sun, Feb 20, 2022 at 5:23 AM 张铎(Duo Zhang) <
> > palomino...@gmail.com
> > > >
> > > > >>> wrote:
> > > > >>>
> > > > >>> Since we have shaded netty, I think we could make use of
> > > > >>> netty's PlatformDependent[1] class to avoid referencing Unsafe
> > > > directly,
> > > > >>> then there will be no problem for us to use the '--release 8'
> > option.
> > > > >>>
> > > > >>> If no big concerns, I will file an issue to remove all the
> > > > >> sun.misc.Unsafe
> > > > >>> references in our code base and make use of PlatformDependent
> from
> > > the
> > > > >>> shaded netty.
> > > > >>>
> > > > >>> Thanks.
> > > > >>>
> > > > >>> 1.
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

For me, since we shade netty, there will be no conflict with other netty
dependencies, so we can make sure the netty version we used is exactly the
one we shaded in hbase-thirdparty.

If later we want to upgrade netty version in hbase-thirdparty, and it
breaks the api, then we can release a new hbase-thirdparty, and upgrade the
main hbase repo to depend on the new hbase-thirdparty version, and also
modify the code in hbase accordingly. There will be no user visible affects.

Anyway, let me have a try first. Even with netty's PaltformDependent class,
since it still references sun.misc.Unsafe, maybe we will still get an
compile error...

Thanks.

Andrew Purtell  于2022年2月22日周二 08:08写道：

> Thanks for that. I understand better what you are proposing now. It is
> clear that you have thought it through.
>
> So then the only question I have is this: Do you think it is a concern that
> Netty's PlatformDependent class is in their "io.netty.util.internal"
> package? The 'internal' designation suggests to me they won't apply the
> same care to not breaking users of it as they would for their public
> (non-"internal") API. I noticed this earlier but had other questions at
> that time.
>
> If it is a concern, we have the option of doing the same thing that Netty
> has done with their PlatformDependent class as a new utility class of our
> own in hbase-thirdparty. This may impose special requirements on building
> hbase-thirdparty, or, at least, the module containing this wrapper, but it
> is an option that would help us avoid external breaking changes. For your
> consideration.
>
> On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang) 
> wrote:
>
> > What I proposed here is for solving the problem…
> >
> > The problem is that when using —release 8, javac will not export the
> > jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a compile
> > error. But at runtime if you export jdk.unsupported you can still use
> > sun.misc.Unsafe. The related jdk issue said —release is only for public
> API
> > so not exporting jdk.unsupported is the expected behavior.
> >
> > So what I proposed here is to remove direct dependency on sun.misc.Unsafe
> > in HBase code, so at compile time there will be no problem. Betty has a
> > wrapper of Unsafe so I think we could have a try. At runtime if we have
> > jdk.unsupported exported we could still actually make use of
> > sun.misc.Unsafe.
> >
> > Feel free to correct me if I missed something or there are still
> uncleared
> > things.
> >
> > Thanks.
> >
> >
> >
> > Andrew Purtell 于2022年2月22日 周二00:56写道：
> >
> > > As Nick discovered ‘—release’ doesn’t work for version 8 anyway.
> > >
> > >
> > > > On Feb 20, 2022, at 3:47 PM, 张铎  wrote:
> > > >
> > > > I know, with —release 8 ByteBuffer will not be an problem but then
> > > > sun.misc.Unsafe can not be used any more right? So the problem here
> is
> > > how
> > > > to make use of Unsafe with —release 8. What I proposed here is that
> we
> > > just
> > > > do not use it, then we can make use of —release 8 to address the
> > > ByteBuffer
> > > > problem.
> > > >
> > > > Thanks.
> > > >
> > > > Andrew Purtell 于2022年2月21日 周一03:32写道：
> > > >
> > > >> The problem that leads us to consider '--release 8', or what it
> would
> > > >> promise if it worked, isn't Unsafe, it is ByteBuffer.
> > > >>
> > >
> >
> https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
> > > >> . Although to your point Netty has ByteBuf...
> > > >>
> > > >>
> > > >>> On Sun, Feb 20, 2022 at 5:23 AM 张铎(Duo Zhang) <
> palomino...@gmail.com
> > >
> > > >>> wrote:
> > > >>>
> > > >>> Since we have shaded netty, I think we could make use of
> > > >>> netty's PlatformDependent[1] class to avoid referencing Unsafe
> > > directly,
> > > >>> then there will be no problem for us to use the '--release 8'
> option.
> > > >>>
> > > >>> If no big concerns, I will file an issue to remove all the
> > > >> sun.misc.Unsafe
> > > >>> references in our code base and make use of PlatformDependent from
> > the
> > > >>> shaded netty.
> > > >>>
> > > >>> Thanks.
> > > >>>
> > > >>> 1.
> > > >>>
> > > >>>
> > > >>
> > >
> >
> https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
> > > >>>
> > > >>> 张铎(Duo Zhang)  于2022年2月16日周三 14:12写道：
> > > >>>
> > >  I think this could be done by some module tricks, where we build a
> > > >>> special
> > >  module with -source 8 and -target 8.We have done similar things in
> > the
> > > >>> past
> > >  to have hadoop1-compat and hadoop2-compact.
> > > 
> > >  Let me have a try.
> > > 
> > >  Sean Busbey  于2022年2月16日周三 13:31写道：
> > > 
> > > > Unfortunately if we want to keep jdk8 working we can't rely on
> > > >> building
> > > > with the jdk release target option on newer jdks.
> > > >
> > > > We ran into this recently in HBASE-25465 where a JDK bug came up.
> > > >
> > > > On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang) <
> > 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Andrew Purtell]

Thanks for that. I understand better what you are proposing now. It is
clear that you have thought it through.

So then the only question I have is this: Do you think it is a concern that
Netty's PlatformDependent class is in their "io.netty.util.internal"
package? The 'internal' designation suggests to me they won't apply the
same care to not breaking users of it as they would for their public
(non-"internal") API. I noticed this earlier but had other questions at
that time.

If it is a concern, we have the option of doing the same thing that Netty
has done with their PlatformDependent class as a new utility class of our
own in hbase-thirdparty. This may impose special requirements on building
hbase-thirdparty, or, at least, the module containing this wrapper, but it
is an option that would help us avoid external breaking changes. For your
consideration.

On Mon, Feb 21, 2022 at 3:48 PM 张铎(Duo Zhang)  wrote:

> What I proposed here is for solving the problem…
>
> The problem is that when using —release 8, javac will not export the
> jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a compile
> error. But at runtime if you export jdk.unsupported you can still use
> sun.misc.Unsafe. The related jdk issue said —release is only for public API
> so not exporting jdk.unsupported is the expected behavior.
>
> So what I proposed here is to remove direct dependency on sun.misc.Unsafe
> in HBase code, so at compile time there will be no problem. Betty has a
> wrapper of Unsafe so I think we could have a try. At runtime if we have
> jdk.unsupported exported we could still actually make use of
> sun.misc.Unsafe.
>
> Feel free to correct me if I missed something or there are still uncleared
> things.
>
> Thanks.
>
>
>
> Andrew Purtell 于2022年2月22日 周二00:56写道：
>
> > As Nick discovered ‘—release’ doesn’t work for version 8 anyway.
> >
> >
> > > On Feb 20, 2022, at 3:47 PM, 张铎  wrote:
> > >
> > > I know, with —release 8 ByteBuffer will not be an problem but then
> > > sun.misc.Unsafe can not be used any more right? So the problem here is
> > how
> > > to make use of Unsafe with —release 8. What I proposed here is that we
> > just
> > > do not use it, then we can make use of —release 8 to address the
> > ByteBuffer
> > > problem.
> > >
> > > Thanks.
> > >
> > > Andrew Purtell 于2022年2月21日 周一03:32写道：
> > >
> > >> The problem that leads us to consider '--release 8', or what it would
> > >> promise if it worked, isn't Unsafe, it is ByteBuffer.
> > >>
> >
> https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
> > >> . Although to your point Netty has ByteBuf...
> > >>
> > >>
> > >>> On Sun, Feb 20, 2022 at 5:23 AM 张铎(Duo Zhang)  >
> > >>> wrote:
> > >>>
> > >>> Since we have shaded netty, I think we could make use of
> > >>> netty's PlatformDependent[1] class to avoid referencing Unsafe
> > directly,
> > >>> then there will be no problem for us to use the '--release 8' option.
> > >>>
> > >>> If no big concerns, I will file an issue to remove all the
> > >> sun.misc.Unsafe
> > >>> references in our code base and make use of PlatformDependent from
> the
> > >>> shaded netty.
> > >>>
> > >>> Thanks.
> > >>>
> > >>> 1.
> > >>>
> > >>>
> > >>
> >
> https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
> > >>>
> > >>> 张铎(Duo Zhang)  于2022年2月16日周三 14:12写道：
> > >>>
> >  I think this could be done by some module tricks, where we build a
> > >>> special
> >  module with -source 8 and -target 8.We have done similar things in
> the
> > >>> past
> >  to have hadoop1-compat and hadoop2-compact.
> > 
> >  Let me have a try.
> > 
> >  Sean Busbey  于2022年2月16日周三 13:31写道：
> > 
> > > Unfortunately if we want to keep jdk8 working we can't rely on
> > >> building
> > > with the jdk release target option on newer jdks.
> > >
> > > We ran into this recently in HBASE-25465 where a JDK bug came up.
> > >
> > > On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang) <
> palomino...@gmail.com
> > >
> > > wrote:
> > >
> > >> +1 on moving the minimum required java version for compiling HBase
> > >> to
> > > Java
> > >> 11. But we could still use --release=8 to still support jdk8 at
> > >>> runtime.
> > >>
> > >> Java 8 is still widely used, and I believe this will last for even
> > >>> more
> > >> years, as lots of big companies such as RedHat, Microsoft and
> Amazon
> > >>> are
> > >> still shipping new jdk8 releases, I do not think it is time to
> fully
> > > drop
> > >> the support of jdk8.
> > >>
> > >> Thanks.
> > >>
> > >> Sean Busbey  于2022年2月16日周三 09:57写道：
> > >>
> > >>> If we set the minJDK to 11 I believe that will effectively remove
> > >>> jdk8
> > >>> support, rather than "just" deprecate it.
> > >>>
> > >>> As we build out more robust testing I would be in favor of
> running
> > > less
> > >> of
> > >>> it on 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

What I proposed here is for solving the problem…

The problem is that when using —release 8, javac will not export the
jdk.unsupported symbols, so using sun.misc.Unsafe will trigger a compile
error. But at runtime if you export jdk.unsupported you can still use
sun.misc.Unsafe. The related jdk issue said —release is only for public API
so not exporting jdk.unsupported is the expected behavior.

So what I proposed here is to remove direct dependency on sun.misc.Unsafe
in HBase code, so at compile time there will be no problem. Betty has a
wrapper of Unsafe so I think we could have a try. At runtime if we have
jdk.unsupported exported we could still actually make use of
sun.misc.Unsafe.

Feel free to correct me if I missed something or there are still uncleared
things.

Thanks.



Andrew Purtell 于2022年2月22日 周二00:56写道：

> As Nick discovered ‘—release’ doesn’t work for version 8 anyway.
>
>
> > On Feb 20, 2022, at 3:47 PM, 张铎  wrote:
> >
> > I know, with —release 8 ByteBuffer will not be an problem but then
> > sun.misc.Unsafe can not be used any more right? So the problem here is
> how
> > to make use of Unsafe with —release 8. What I proposed here is that we
> just
> > do not use it, then we can make use of —release 8 to address the
> ByteBuffer
> > problem.
> >
> > Thanks.
> >
> > Andrew Purtell 于2022年2月21日 周一03:32写道：
> >
> >> The problem that leads us to consider '--release 8', or what it would
> >> promise if it worked, isn't Unsafe, it is ByteBuffer.
> >>
> https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
> >> . Although to your point Netty has ByteBuf...
> >>
> >>
> >>> On Sun, Feb 20, 2022 at 5:23 AM 张铎(Duo Zhang) 
> >>> wrote:
> >>>
> >>> Since we have shaded netty, I think we could make use of
> >>> netty's PlatformDependent[1] class to avoid referencing Unsafe
> directly,
> >>> then there will be no problem for us to use the '--release 8' option.
> >>>
> >>> If no big concerns, I will file an issue to remove all the
> >> sun.misc.Unsafe
> >>> references in our code base and make use of PlatformDependent from the
> >>> shaded netty.
> >>>
> >>> Thanks.
> >>>
> >>> 1.
> >>>
> >>>
> >>
> https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
> >>>
> >>> 张铎(Duo Zhang)  于2022年2月16日周三 14:12写道：
> >>>
>  I think this could be done by some module tricks, where we build a
> >>> special
>  module with -source 8 and -target 8.We have done similar things in the
> >>> past
>  to have hadoop1-compat and hadoop2-compact.
> 
>  Let me have a try.
> 
>  Sean Busbey  于2022年2月16日周三 13:31写道：
> 
> > Unfortunately if we want to keep jdk8 working we can't rely on
> >> building
> > with the jdk release target option on newer jdks.
> >
> > We ran into this recently in HBASE-25465 where a JDK bug came up.
> >
> > On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang)  >
> > wrote:
> >
> >> +1 on moving the minimum required java version for compiling HBase
> >> to
> > Java
> >> 11. But we could still use --release=8 to still support jdk8 at
> >>> runtime.
> >>
> >> Java 8 is still widely used, and I believe this will last for even
> >>> more
> >> years, as lots of big companies such as RedHat, Microsoft and Amazon
> >>> are
> >> still shipping new jdk8 releases, I do not think it is time to fully
> > drop
> >> the support of jdk8.
> >>
> >> Thanks.
> >>
> >> Sean Busbey  于2022年2月16日周三 09:57写道：
> >>
> >>> If we set the minJDK to 11 I believe that will effectively remove
> >>> jdk8
> >>> support, rather than "just" deprecate it.
> >>>
> >>> As we build out more robust testing I would be in favor of running
> > less
> >> of
> >>> it on deprecated JDKs.
> >>>
> >>>
> >>> On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
> >>>
>  Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems
> > reasonable
> >> to
>  me.
> 
>  Gotta start pushing the issue somehow.
> 
>  On 2/15/22 1:47 PM, Sean Busbey wrote:
> > Hi folks!
> >
> > It's been some time since we decided to stick to LTS JDK
> >>> releases
> > as
> >> a
>  way
> > of getting a handle on the JDK treadmill.
> >
> > What do folks think about deprecating JDK8? The openjdk8u
> >>> project
> > is
>  still
> > going and there are commercial support options at least
> >> through
> > 2030.
> >
> > Deprecating it in HBase 3 would mean we could remove it in
> >> HBase
> > 4,
> >> not
> > that we would _have_ to remove it. The way I think about
> >> likely
> >> timing
> >>> of
> > these events goes like this:
> >
> > * HBase 2 started alphas in June 2017, betas in January 2018,
> >>> and
> >> came
>  out
> > in April 2018
> > 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Andrew Purtell]

As Nick discovered ‘—release’ doesn’t work for version 8 anyway. 


> On Feb 20, 2022, at 3:47 PM, 张铎  wrote:
> 
> I know, with —release 8 ByteBuffer will not be an problem but then
> sun.misc.Unsafe can not be used any more right? So the problem here is how
> to make use of Unsafe with —release 8. What I proposed here is that we just
> do not use it, then we can make use of —release 8 to address the ByteBuffer
> problem.
> 
> Thanks.
> 
> Andrew Purtell 于2022年2月21日 周一03:32写道：
> 
>> The problem that leads us to consider '--release 8', or what it would
>> promise if it worked, isn't Unsafe, it is ByteBuffer.
>> https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
>> . Although to your point Netty has ByteBuf...
>> 
>> 
>>> On Sun, Feb 20, 2022 at 5:23 AM 张铎(Duo Zhang) 
>>> wrote:
>>> 
>>> Since we have shaded netty, I think we could make use of
>>> netty's PlatformDependent[1] class to avoid referencing Unsafe directly,
>>> then there will be no problem for us to use the '--release 8' option.
>>> 
>>> If no big concerns, I will file an issue to remove all the
>> sun.misc.Unsafe
>>> references in our code base and make use of PlatformDependent from the
>>> shaded netty.
>>> 
>>> Thanks.
>>> 
>>> 1.
>>> 
>>> 
>> https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
>>> 
>>> 张铎(Duo Zhang)  于2022年2月16日周三 14:12写道：
>>> 
 I think this could be done by some module tricks, where we build a
>>> special
 module with -source 8 and -target 8.We have done similar things in the
>>> past
 to have hadoop1-compat and hadoop2-compact.
 
 Let me have a try.
 
 Sean Busbey  于2022年2月16日周三 13:31写道：
 
> Unfortunately if we want to keep jdk8 working we can't rely on
>> building
> with the jdk release target option on newer jdks.
> 
> We ran into this recently in HBASE-25465 where a JDK bug came up.
> 
> On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang) 
> wrote:
> 
>> +1 on moving the minimum required java version for compiling HBase
>> to
> Java
>> 11. But we could still use --release=8 to still support jdk8 at
>>> runtime.
>> 
>> Java 8 is still widely used, and I believe this will last for even
>>> more
>> years, as lots of big companies such as RedHat, Microsoft and Amazon
>>> are
>> still shipping new jdk8 releases, I do not think it is time to fully
> drop
>> the support of jdk8.
>> 
>> Thanks.
>> 
>> Sean Busbey  于2022年2月16日周三 09:57写道：
>> 
>>> If we set the minJDK to 11 I believe that will effectively remove
>>> jdk8
>>> support, rather than "just" deprecate it.
>>> 
>>> As we build out more robust testing I would be in favor of running
> less
>> of
>>> it on deprecated JDKs.
>>> 
>>> 
>>> On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
>>> 
 Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems
> reasonable
>> to
 me.
 
 Gotta start pushing the issue somehow.
 
 On 2/15/22 1:47 PM, Sean Busbey wrote:
> Hi folks!
> 
> It's been some time since we decided to stick to LTS JDK
>>> releases
> as
>> a
 way
> of getting a handle on the JDK treadmill.
> 
> What do folks think about deprecating JDK8? The openjdk8u
>>> project
> is
 still
> going and there are commercial support options at least
>> through
> 2030.
> 
> Deprecating it in HBase 3 would mean we could remove it in
>> HBase
> 4,
>> not
> that we would _have_ to remove it. The way I think about
>> likely
>> timing
>>> of
> these events goes like this:
> 
> * HBase 2 started alphas in June 2017, betas in January 2018,
>>> and
>> came
 out
> in April 2018
> * HBase 3 started alphas in July 2021, and as of Feb 2022 we
> haven't
> discussed how close we are to our stated beta goals (upgrades
>>> from
>>> active
> 2.x releases and removal of not-ready features).
> 
> Given the above, in the absence of us specifically pushing to
>>> roll
 through
> major version numbers for some reason, I think a reasonably
>>> conservative
> estimate is for HBase 3 to arrive in late 2022 or early 2023
>> and
> then
 HBase
> 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems
> unlikely.
> 
> That all said, our current reference guide section on java
> versions
>>> does
> not sound very confident about JDK11 support.
> 
>> A Note on JDK11 *
>> Preliminary support for JDK11 is introduced with HBase 2.3.0.
> This
> support is limited to
>> compilation and running the full test suite. There are open
>> questions
> regarding the runtime
>> 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

I know, with —release 8 ByteBuffer will not be an problem but then
sun.misc.Unsafe can not be used any more right? So the problem here is how
to make use of Unsafe with —release 8. What I proposed here is that we just
do not use it, then we can make use of —release 8 to address the ByteBuffer
problem.

Thanks.

Andrew Purtell 于2022年2月21日 周一03:32写道：

> The problem that leads us to consider '--release 8', or what it would
> promise if it worked, isn't Unsafe, it is ByteBuffer.
> https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
> . Although to your point Netty has ByteBuf...
>
>
> On Sun, Feb 20, 2022 at 5:23 AM 张铎(Duo Zhang) 
> wrote:
>
> > Since we have shaded netty, I think we could make use of
> > netty's PlatformDependent[1] class to avoid referencing Unsafe directly,
> > then there will be no problem for us to use the '--release 8' option.
> >
> > If no big concerns, I will file an issue to remove all the
> sun.misc.Unsafe
> > references in our code base and make use of PlatformDependent from the
> > shaded netty.
> >
> > Thanks.
> >
> > 1.
> >
> >
> https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
> >
> > 张铎(Duo Zhang)  于2022年2月16日周三 14:12写道：
> >
> > > I think this could be done by some module tricks, where we build a
> > special
> > > module with -source 8 and -target 8.We have done similar things in the
> > past
> > > to have hadoop1-compat and hadoop2-compact.
> > >
> > > Let me have a try.
> > >
> > > Sean Busbey  于2022年2月16日周三 13:31写道：
> > >
> > >> Unfortunately if we want to keep jdk8 working we can't rely on
> building
> > >> with the jdk release target option on newer jdks.
> > >>
> > >> We ran into this recently in HBASE-25465 where a JDK bug came up.
> > >>
> > >> On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang) 
> > >> wrote:
> > >>
> > >> > +1 on moving the minimum required java version for compiling HBase
> to
> > >> Java
> > >> > 11. But we could still use --release=8 to still support jdk8 at
> > runtime.
> > >> >
> > >> > Java 8 is still widely used, and I believe this will last for even
> > more
> > >> > years, as lots of big companies such as RedHat, Microsoft and Amazon
> > are
> > >> > still shipping new jdk8 releases, I do not think it is time to fully
> > >> drop
> > >> > the support of jdk8.
> > >> >
> > >> > Thanks.
> > >> >
> > >> > Sean Busbey  于2022年2月16日周三 09:57写道：
> > >> >
> > >> > > If we set the minJDK to 11 I believe that will effectively remove
> > jdk8
> > >> > > support, rather than "just" deprecate it.
> > >> > >
> > >> > > As we build out more robust testing I would be in favor of running
> > >> less
> > >> > of
> > >> > > it on deprecated JDKs.
> > >> > >
> > >> > >
> > >> > > On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
> > >> > >
> > >> > > > Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems
> > >> reasonable
> > >> > to
> > >> > > > me.
> > >> > > >
> > >> > > > Gotta start pushing the issue somehow.
> > >> > > >
> > >> > > > On 2/15/22 1:47 PM, Sean Busbey wrote:
> > >> > > > > Hi folks!
> > >> > > > >
> > >> > > > > It's been some time since we decided to stick to LTS JDK
> > releases
> > >> as
> > >> > a
> > >> > > > way
> > >> > > > > of getting a handle on the JDK treadmill.
> > >> > > > >
> > >> > > > > What do folks think about deprecating JDK8? The openjdk8u
> > project
> > >> is
> > >> > > > still
> > >> > > > > going and there are commercial support options at least
> through
> > >> 2030.
> > >> > > > >
> > >> > > > > Deprecating it in HBase 3 would mean we could remove it in
> HBase
> > >> 4,
> > >> > not
> > >> > > > > that we would _have_ to remove it. The way I think about
> likely
> > >> > timing
> > >> > > of
> > >> > > > > these events goes like this:
> > >> > > > >
> > >> > > > > * HBase 2 started alphas in June 2017, betas in January 2018,
> > and
> > >> > came
> > >> > > > out
> > >> > > > > in April 2018
> > >> > > > > * HBase 3 started alphas in July 2021, and as of Feb 2022 we
> > >> haven't
> > >> > > > > discussed how close we are to our stated beta goals (upgrades
> > from
> > >> > > active
> > >> > > > > 2.x releases and removal of not-ready features).
> > >> > > > >
> > >> > > > > Given the above, in the absence of us specifically pushing to
> > roll
> > >> > > > through
> > >> > > > > major version numbers for some reason, I think a reasonably
> > >> > > conservative
> > >> > > > > estimate is for HBase 3 to arrive in late 2022 or early 2023
> and
> > >> then
> > >> > > > HBase
> > >> > > > > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems
> > >> unlikely.
> > >> > > > >
> > >> > > > > That all said, our current reference guide section on java
> > >> versions
> > >> > > does
> > >> > > > > not sound very confident about JDK11 support.
> > >> > > > >
> > >> > > > >> A Note on JDK11 *
> > >> > > > >> Preliminary support for JDK11 is introduced with HBase 2.3.0.
> > >> This
> > >> > > > > support is limited to
> > >> > > > >> 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Andrew Purtell]

The problem that leads us to consider '--release 8', or what it would
promise if it worked, isn't Unsafe, it is ByteBuffer.
https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
. Although to your point Netty has ByteBuf...


On Sun, Feb 20, 2022 at 5:23 AM 张铎(Duo Zhang)  wrote:

> Since we have shaded netty, I think we could make use of
> netty's PlatformDependent[1] class to avoid referencing Unsafe directly,
> then there will be no problem for us to use the '--release 8' option.
>
> If no big concerns, I will file an issue to remove all the sun.misc.Unsafe
> references in our code base and make use of PlatformDependent from the
> shaded netty.
>
> Thanks.
>
> 1.
>
> https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java
>
> 张铎(Duo Zhang)  于2022年2月16日周三 14:12写道：
>
> > I think this could be done by some module tricks, where we build a
> special
> > module with -source 8 and -target 8.We have done similar things in the
> past
> > to have hadoop1-compat and hadoop2-compact.
> >
> > Let me have a try.
> >
> > Sean Busbey  于2022年2月16日周三 13:31写道：
> >
> >> Unfortunately if we want to keep jdk8 working we can't rely on building
> >> with the jdk release target option on newer jdks.
> >>
> >> We ran into this recently in HBASE-25465 where a JDK bug came up.
> >>
> >> On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang) 
> >> wrote:
> >>
> >> > +1 on moving the minimum required java version for compiling HBase to
> >> Java
> >> > 11. But we could still use --release=8 to still support jdk8 at
> runtime.
> >> >
> >> > Java 8 is still widely used, and I believe this will last for even
> more
> >> > years, as lots of big companies such as RedHat, Microsoft and Amazon
> are
> >> > still shipping new jdk8 releases, I do not think it is time to fully
> >> drop
> >> > the support of jdk8.
> >> >
> >> > Thanks.
> >> >
> >> > Sean Busbey  于2022年2月16日周三 09:57写道：
> >> >
> >> > > If we set the minJDK to 11 I believe that will effectively remove
> jdk8
> >> > > support, rather than "just" deprecate it.
> >> > >
> >> > > As we build out more robust testing I would be in favor of running
> >> less
> >> > of
> >> > > it on deprecated JDKs.
> >> > >
> >> > >
> >> > > On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
> >> > >
> >> > > > Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems
> >> reasonable
> >> > to
> >> > > > me.
> >> > > >
> >> > > > Gotta start pushing the issue somehow.
> >> > > >
> >> > > > On 2/15/22 1:47 PM, Sean Busbey wrote:
> >> > > > > Hi folks!
> >> > > > >
> >> > > > > It's been some time since we decided to stick to LTS JDK
> releases
> >> as
> >> > a
> >> > > > way
> >> > > > > of getting a handle on the JDK treadmill.
> >> > > > >
> >> > > > > What do folks think about deprecating JDK8? The openjdk8u
> project
> >> is
> >> > > > still
> >> > > > > going and there are commercial support options at least through
> >> 2030.
> >> > > > >
> >> > > > > Deprecating it in HBase 3 would mean we could remove it in HBase
> >> 4,
> >> > not
> >> > > > > that we would _have_ to remove it. The way I think about likely
> >> > timing
> >> > > of
> >> > > > > these events goes like this:
> >> > > > >
> >> > > > > * HBase 2 started alphas in June 2017, betas in January 2018,
> and
> >> > came
> >> > > > out
> >> > > > > in April 2018
> >> > > > > * HBase 3 started alphas in July 2021, and as of Feb 2022 we
> >> haven't
> >> > > > > discussed how close we are to our stated beta goals (upgrades
> from
> >> > > active
> >> > > > > 2.x releases and removal of not-ready features).
> >> > > > >
> >> > > > > Given the above, in the absence of us specifically pushing to
> roll
> >> > > > through
> >> > > > > major version numbers for some reason, I think a reasonably
> >> > > conservative
> >> > > > > estimate is for HBase 3 to arrive in late 2022 or early 2023 and
> >> then
> >> > > > HBase
> >> > > > > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems
> >> unlikely.
> >> > > > >
> >> > > > > That all said, our current reference guide section on java
> >> versions
> >> > > does
> >> > > > > not sound very confident about JDK11 support.
> >> > > > >
> >> > > > >> A Note on JDK11 *
> >> > > > >> Preliminary support for JDK11 is introduced with HBase 2.3.0.
> >> This
> >> > > > > support is limited to
> >> > > > >> compilation and running the full test suite. There are open
> >> > questions
> >> > > > > regarding the runtime
> >> > > > >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> >> > > > > (HADOOP-15338).
> >> > > > >> Significantly, neither project has yet released a version with
> >> > > explicit
> >> > > > > runtime support for
> >> > > > >> JDK11. The remaining known issues in HBase are catalogued in
> >> > > > HBASE-22972.
> >> > > > >>
> >> > > > >
> >> > > > > Since that blurb was written, Hadoop has added JDK11 support [1]
> >> as
> >> > has
> >> > > > > ZooKeeper[2]. As a part of buttoning up our JDK11 support we
> 

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

Since we have shaded netty, I think we could make use of
netty's PlatformDependent[1] class to avoid referencing Unsafe directly,
then there will be no problem for us to use the '--release 8' option.

If no big concerns, I will file an issue to remove all the sun.misc.Unsafe
references in our code base and make use of PlatformDependent from the
shaded netty.

Thanks.

1.
https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java

张铎(Duo Zhang)  于2022年2月16日周三 14:12写道：

> I think this could be done by some module tricks, where we build a special
> module with -source 8 and -target 8.We have done similar things in the past
> to have hadoop1-compat and hadoop2-compact.
>
> Let me have a try.
>
> Sean Busbey  于2022年2月16日周三 13:31写道：
>
>> Unfortunately if we want to keep jdk8 working we can't rely on building
>> with the jdk release target option on newer jdks.
>>
>> We ran into this recently in HBASE-25465 where a JDK bug came up.
>>
>> On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang) 
>> wrote:
>>
>> > +1 on moving the minimum required java version for compiling HBase to
>> Java
>> > 11. But we could still use --release=8 to still support jdk8 at runtime.
>> >
>> > Java 8 is still widely used, and I believe this will last for even more
>> > years, as lots of big companies such as RedHat, Microsoft and Amazon are
>> > still shipping new jdk8 releases, I do not think it is time to fully
>> drop
>> > the support of jdk8.
>> >
>> > Thanks.
>> >
>> > Sean Busbey  于2022年2月16日周三 09:57写道：
>> >
>> > > If we set the minJDK to 11 I believe that will effectively remove jdk8
>> > > support, rather than "just" deprecate it.
>> > >
>> > > As we build out more robust testing I would be in favor of running
>> less
>> > of
>> > > it on deprecated JDKs.
>> > >
>> > >
>> > > On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
>> > >
>> > > > Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems
>> reasonable
>> > to
>> > > > me.
>> > > >
>> > > > Gotta start pushing the issue somehow.
>> > > >
>> > > > On 2/15/22 1:47 PM, Sean Busbey wrote:
>> > > > > Hi folks!
>> > > > >
>> > > > > It's been some time since we decided to stick to LTS JDK releases
>> as
>> > a
>> > > > way
>> > > > > of getting a handle on the JDK treadmill.
>> > > > >
>> > > > > What do folks think about deprecating JDK8? The openjdk8u project
>> is
>> > > > still
>> > > > > going and there are commercial support options at least through
>> 2030.
>> > > > >
>> > > > > Deprecating it in HBase 3 would mean we could remove it in HBase
>> 4,
>> > not
>> > > > > that we would _have_ to remove it. The way I think about likely
>> > timing
>> > > of
>> > > > > these events goes like this:
>> > > > >
>> > > > > * HBase 2 started alphas in June 2017, betas in January 2018, and
>> > came
>> > > > out
>> > > > > in April 2018
>> > > > > * HBase 3 started alphas in July 2021, and as of Feb 2022 we
>> haven't
>> > > > > discussed how close we are to our stated beta goals (upgrades from
>> > > active
>> > > > > 2.x releases and removal of not-ready features).
>> > > > >
>> > > > > Given the above, in the absence of us specifically pushing to roll
>> > > > through
>> > > > > major version numbers for some reason, I think a reasonably
>> > > conservative
>> > > > > estimate is for HBase 3 to arrive in late 2022 or early 2023 and
>> then
>> > > > HBase
>> > > > > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems
>> unlikely.
>> > > > >
>> > > > > That all said, our current reference guide section on java
>> versions
>> > > does
>> > > > > not sound very confident about JDK11 support.
>> > > > >
>> > > > >> A Note on JDK11 *
>> > > > >> Preliminary support for JDK11 is introduced with HBase 2.3.0.
>> This
>> > > > > support is limited to
>> > > > >> compilation and running the full test suite. There are open
>> > questions
>> > > > > regarding the runtime
>> > > > >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
>> > > > > (HADOOP-15338).
>> > > > >> Significantly, neither project has yet released a version with
>> > > explicit
>> > > > > runtime support for
>> > > > >> JDK11. The remaining known issues in HBase are catalogued in
>> > > > HBASE-22972.
>> > > > >>
>> > > > >
>> > > > > Since that blurb was written, Hadoop has added JDK11 support [1]
>> as
>> > has
>> > > > > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could
>> > > update
>> > > > > our minimum supported versions of these projects to match that
>> > support.
>> > > > >
>> > > > > What do folks think?
>> > > > >
>> > > > > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
>> > > > > [2]:
>> > > > >
>> > >
>> https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

so we would require a specific patch release of Hadoop as a minimum? It's
messy, but we've certainly done it before.

On Thu, Feb 17, 2022 at 1:31 PM Andrew Purtell  wrote:

> On Thu, Feb 17, 2022 at 10:50 AM Steve Loughran  wrote:
>
> > j
> > On 2022/02/15 22:27:50 Andrew Purtell wrote:
> >
> > > The primary consideration to my mind is the state of S3A: in what
> > version it can be said to be stable and feature complete. I think 3.3 is
> > the appropriate code line for that criteria but perhaps 3.2 could serve
> as
> > well.
> >
> > 3.3.1. i have not been near 3.2.x for many years except for a change for
> > it to be ok with people disabling dir marker deletion
> > (fs.s3a.marker.retention=keep).
> >
> > 3.3.1 is good, with one little surprise:
> > https://issues.apache.org/jira/browse/HADOOP-17771
> > moving to the s3 client builder API for region support (needed for AWS
> > AccessPoint VPN access) broke things unless you explicitly set a region,
> > were running in EC2 *or had the AWS CLI installed*. All tests running in
> > EC2 and all us developers with the CLI installed missed that completely.
> > see the jira for the trivial workaround.
> >
> >
> > you need 3.3.1 for up to date abfs connections too.
> >
> > there's a 3.3.2 release due real soon now...testing the latest RC against
> > HBase would be wonderful.
> >
> > -steve
> >
>
> Thank you for writing in Steve. I think by the time we bring this in for a
> landing over here 3.3.2 will be out, that gets my vote.
>
> --
> Best regards,
> Andrew
>
> Unrest, ignorance distilled, nihilistic imbeciles -
> It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>- A23, Welcome, Apocalypse
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Andrew Purtell]

On Thu, Feb 17, 2022 at 10:50 AM Steve Loughran  wrote:

> j
> On 2022/02/15 22:27:50 Andrew Purtell wrote:
>
> > The primary consideration to my mind is the state of S3A: in what
> version it can be said to be stable and feature complete. I think 3.3 is
> the appropriate code line for that criteria but perhaps 3.2 could serve as
> well.
>
> 3.3.1. i have not been near 3.2.x for many years except for a change for
> it to be ok with people disabling dir marker deletion
> (fs.s3a.marker.retention=keep).
>
> 3.3.1 is good, with one little surprise:
> https://issues.apache.org/jira/browse/HADOOP-17771
> moving to the s3 client builder API for region support (needed for AWS
> AccessPoint VPN access) broke things unless you explicitly set a region,
> were running in EC2 *or had the AWS CLI installed*. All tests running in
> EC2 and all us developers with the CLI installed missed that completely.
> see the jira for the trivial workaround.
>
>
> you need 3.3.1 for up to date abfs connections too.
>
> there's a 3.3.2 release due real soon now...testing the latest RC against
> HBase would be wonderful.
>
> -steve
>

Thank you for writing in Steve. I think by the time we bring this in for a
landing over here 3.3.2 will be out, that gets my vote.

-- 
Best regards,
Andrew

Unrest, ignorance distilled, nihilistic imbeciles -
It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
   - A23, Welcome, Apocalypse

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Steve Loughran]

j
On 2022/02/15 22:27:50 Andrew Purtell wrote:

> The primary consideration to my mind is the state of S3A: in what version it 
> can be said to be stable and feature complete. I think 3.3 is the appropriate 
> code line for that criteria but perhaps 3.2 could serve as well. 

3.3.1. i have not been near 3.2.x for many years except for a change for it to 
be ok with people disabling dir marker deletion (fs.s3a.marker.retention=keep).

3.3.1 is good, with one little surprise: 
https://issues.apache.org/jira/browse/HADOOP-17771
moving to the s3 client builder API for region support (needed for AWS 
AccessPoint VPN access) broke things unless you explicitly set a region, were 
running in EC2 *or had the AWS CLI installed*. All tests running in EC2 and all 
us developers with the CLI installed missed that completely. see the jira for 
the trivial workaround.


you need 3.3.1 for up to date abfs connections too.

there's a 3.3.2 release due real soon now...testing the latest RC against HBase 
would be wonderful.

-steve

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Josh Elser]

On 2/16/22 12:24 AM, Sean Busbey wrote:

Regarding the original question, I would be in favor of the proposal. Time
marches on. I assume just to state the obvious that our destination of
minimum LTS would shift from 8 to 11.


Yes, sorry I should have expressly stated JDK11 would become the minimum
with some release after HBase 3.

I got here because I wanted to start working on qualifying JDK17 as a
runtime environment but then realized we were putting more caveats on JDK11
than I expected.

Hadoop 2 isn’t exactly dead, at least the source branch is still receiving

occasional update, but is not releasing. We should probably consider it
effectively EOL.


IIRC we've already dropped Hadoop 2 support for HBase 3.


Correct.


The Hadoop minimum could become 3.3. The primary consideration to my mind

is the state of S3A: in what version it can be said to be stable and
feature complete. I think 3.3 is the appropriate code line for that
criteria but perhaps 3.2 could serve as well.


I really like this as a criteria. Anyone else have an idea on this?


I believe we've been benefiting from S3A changes from Hadoop 3.3 inside 
at Cloudera already. However, I believe that we'll actually see more 
"pains" once we get the storefile tracking feature solid (whereas today, 
transient/perf problems we might face in S3A would be hidden by the fact 
that we're doubling our I/O costs on compaction, memstore flushes, etc).


I have not been following super-closely, but let me see if I can bring 
this in front of Steve or someone else from Cloudera to chime in.

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

I think this could be done by some module tricks, where we build a special
module with -source 8 and -target 8.We have done similar things in the past
to have hadoop1-compat and hadoop2-compact.

Let me have a try.

Sean Busbey  于2022年2月16日周三 13:31写道：

> Unfortunately if we want to keep jdk8 working we can't rely on building
> with the jdk release target option on newer jdks.
>
> We ran into this recently in HBASE-25465 where a JDK bug came up.
>
> On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang) 
> wrote:
>
> > +1 on moving the minimum required java version for compiling HBase to
> Java
> > 11. But we could still use --release=8 to still support jdk8 at runtime.
> >
> > Java 8 is still widely used, and I believe this will last for even more
> > years, as lots of big companies such as RedHat, Microsoft and Amazon are
> > still shipping new jdk8 releases, I do not think it is time to fully drop
> > the support of jdk8.
> >
> > Thanks.
> >
> > Sean Busbey  于2022年2月16日周三 09:57写道：
> >
> > > If we set the minJDK to 11 I believe that will effectively remove jdk8
> > > support, rather than "just" deprecate it.
> > >
> > > As we build out more robust testing I would be in favor of running less
> > of
> > > it on deprecated JDKs.
> > >
> > >
> > > On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
> > >
> > > > Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems reasonable
> > to
> > > > me.
> > > >
> > > > Gotta start pushing the issue somehow.
> > > >
> > > > On 2/15/22 1:47 PM, Sean Busbey wrote:
> > > > > Hi folks!
> > > > >
> > > > > It's been some time since we decided to stick to LTS JDK releases
> as
> > a
> > > > way
> > > > > of getting a handle on the JDK treadmill.
> > > > >
> > > > > What do folks think about deprecating JDK8? The openjdk8u project
> is
> > > > still
> > > > > going and there are commercial support options at least through
> 2030.
> > > > >
> > > > > Deprecating it in HBase 3 would mean we could remove it in HBase 4,
> > not
> > > > > that we would _have_ to remove it. The way I think about likely
> > timing
> > > of
> > > > > these events goes like this:
> > > > >
> > > > > * HBase 2 started alphas in June 2017, betas in January 2018, and
> > came
> > > > out
> > > > > in April 2018
> > > > > * HBase 3 started alphas in July 2021, and as of Feb 2022 we
> haven't
> > > > > discussed how close we are to our stated beta goals (upgrades from
> > > active
> > > > > 2.x releases and removal of not-ready features).
> > > > >
> > > > > Given the above, in the absence of us specifically pushing to roll
> > > > through
> > > > > major version numbers for some reason, I think a reasonably
> > > conservative
> > > > > estimate is for HBase 3 to arrive in late 2022 or early 2023 and
> then
> > > > HBase
> > > > > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems
> unlikely.
> > > > >
> > > > > That all said, our current reference guide section on java versions
> > > does
> > > > > not sound very confident about JDK11 support.
> > > > >
> > > > >> A Note on JDK11 *
> > > > >> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> > > > > support is limited to
> > > > >> compilation and running the full test suite. There are open
> > questions
> > > > > regarding the runtime
> > > > >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> > > > > (HADOOP-15338).
> > > > >> Significantly, neither project has yet released a version with
> > > explicit
> > > > > runtime support for
> > > > >> JDK11. The remaining known issues in HBase are catalogued in
> > > > HBASE-22972.
> > > > >>
> > > > >
> > > > > Since that blurb was written, Hadoop has added JDK11 support [1] as
> > has
> > > > > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could
> > > update
> > > > > our minimum supported versions of these projects to match that
> > support.
> > > > >
> > > > > What do folks think?
> > > > >
> > > > > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> > > > > [2]:
> > > > >
> > >
> https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

Unfortunately if we want to keep jdk8 working we can't rely on building
with the jdk release target option on newer jdks.

We ran into this recently in HBASE-25465 where a JDK bug came up.

On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang)  wrote:

> +1 on moving the minimum required java version for compiling HBase to Java
> 11. But we could still use --release=8 to still support jdk8 at runtime.
>
> Java 8 is still widely used, and I believe this will last for even more
> years, as lots of big companies such as RedHat, Microsoft and Amazon are
> still shipping new jdk8 releases, I do not think it is time to fully drop
> the support of jdk8.
>
> Thanks.
>
> Sean Busbey  于2022年2月16日周三 09:57写道：
>
> > If we set the minJDK to 11 I believe that will effectively remove jdk8
> > support, rather than "just" deprecate it.
> >
> > As we build out more robust testing I would be in favor of running less
> of
> > it on deprecated JDKs.
> >
> >
> > On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
> >
> > > Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems reasonable
> to
> > > me.
> > >
> > > Gotta start pushing the issue somehow.
> > >
> > > On 2/15/22 1:47 PM, Sean Busbey wrote:
> > > > Hi folks!
> > > >
> > > > It's been some time since we decided to stick to LTS JDK releases as
> a
> > > way
> > > > of getting a handle on the JDK treadmill.
> > > >
> > > > What do folks think about deprecating JDK8? The openjdk8u project is
> > > still
> > > > going and there are commercial support options at least through 2030.
> > > >
> > > > Deprecating it in HBase 3 would mean we could remove it in HBase 4,
> not
> > > > that we would _have_ to remove it. The way I think about likely
> timing
> > of
> > > > these events goes like this:
> > > >
> > > > * HBase 2 started alphas in June 2017, betas in January 2018, and
> came
> > > out
> > > > in April 2018
> > > > * HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
> > > > discussed how close we are to our stated beta goals (upgrades from
> > active
> > > > 2.x releases and removal of not-ready features).
> > > >
> > > > Given the above, in the absence of us specifically pushing to roll
> > > through
> > > > major version numbers for some reason, I think a reasonably
> > conservative
> > > > estimate is for HBase 3 to arrive in late 2022 or early 2023 and then
> > > HBase
> > > > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.
> > > >
> > > > That all said, our current reference guide section on java versions
> > does
> > > > not sound very confident about JDK11 support.
> > > >
> > > >> A Note on JDK11 *
> > > >> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> > > > support is limited to
> > > >> compilation and running the full test suite. There are open
> questions
> > > > regarding the runtime
> > > >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> > > > (HADOOP-15338).
> > > >> Significantly, neither project has yet released a version with
> > explicit
> > > > runtime support for
> > > >> JDK11. The remaining known issues in HBase are catalogued in
> > > HBASE-22972.
> > > >>
> > > >
> > > > Since that blurb was written, Hadoop has added JDK11 support [1] as
> has
> > > > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could
> > update
> > > > our minimum supported versions of these projects to match that
> support.
> > > >
> > > > What do folks think?
> > > >
> > > > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> > > > [2]:
> > > >
> > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
> > > >
> > >
> >
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

> Regarding the original question, I would be in favor of the proposal. Time
> marches on. I assume just to state the obvious that our destination of
> minimum LTS would shift from 8 to 11.


Yes, sorry I should have expressly stated JDK11 would become the minimum
with some release after HBase 3.

I got here because I wanted to start working on qualifying JDK17 as a
runtime environment but then realized we were putting more caveats on JDK11
than I expected.

Hadoop 2 isn’t exactly dead, at least the source branch is still receiving
> occasional update, but is not releasing. We should probably consider it
> effectively EOL.


IIRC we've already dropped Hadoop 2 support for HBase 3.

The Hadoop minimum could become 3.3. The primary consideration to my mind
> is the state of S3A: in what version it can be said to be stable and
> feature complete. I think 3.3 is the appropriate code line for that
> criteria but perhaps 3.2 could serve as well.


I really like this as a criteria. Anyone else have an idea on this?



On Tue, Feb 15, 2022, 16:28 Andrew Purtell  wrote:

> The section in our docs on JDK11 seems out of date. In my experience using
> Java 11 to run ZooKeeper (3.5), Hadoop HDFS/YARN/MR (2.10 and 3.3), and
> HBase (2.4) is a nonissue. These software versions are stable under load.
> Most recently I tested scale ingest with 11.0.15+1, hot off the presses so
> to speak. No issues to report — although, I cannot claim this is the
> production configuration where I work yet. Perhaps near the end of 2022.
> Anyway, I think mine is likely not the only experience like this so the
> hedging language can be removed from the text, especially if someone else
> writes in with positive testimonial.
>
> Regarding the original question, I would be in favor of the proposal. Time
> matches on. I assume just to state the obvious that our destination of
> minimum LTS would shift from 8 to 11.
>
> I also agree it would be good to move up from EOL or soon to be EOL
> versions of our dependencies. Looks like ZooKeeper 3.6 will be the lowest
> live code line by the end of this year. Hadoop 2 isn’t exactly dead, at
> least the source branch is still receiving occasional update, but is not
> releasing. We should probably consider it effectively EOL. The Hadoop
> minimum could become 3.3. The primary consideration to my mind is the state
> of S3A: in what version it can be said to be stable and feature complete. I
> think 3.3 is the appropriate code line for that criteria but perhaps 3.2
> could serve as well.
>
> > On Feb 15, 2022, at 10:48 AM, Sean Busbey  wrote:
> >
> > Hi folks!
> >
> > It's been some time since we decided to stick to LTS JDK releases as a
> way
> > of getting a handle on the JDK treadmill.
> >
> > What do folks think about deprecating JDK8? The openjdk8u project is
> still
> > going and there are commercial support options at least through 2030.
> >
> > Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
> > that we would _have_ to remove it. The way I think about likely timing of
> > these events goes like this:
> >
> > * HBase 2 started alphas in June 2017, betas in January 2018, and came
> out
> > in April 2018
> > * HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
> > discussed how close we are to our stated beta goals (upgrades from active
> > 2.x releases and removal of not-ready features).
> >
> > Given the above, in the absence of us specifically pushing to roll
> through
> > major version numbers for some reason, I think a reasonably conservative
> > estimate is for HBase 3 to arrive in late 2022 or early 2023 and then
> HBase
> > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.
> >
> > That all said, our current reference guide section on java versions does
> > not sound very confident about JDK11 support.
> >
> >> A Note on JDK11 *
> >> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> > support is limited to
> >> compilation and running the full test suite. There are open questions
> > regarding the runtime
> >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> > (HADOOP-15338).
> >> Significantly, neither project has yet released a version with explicit
> > runtime support for
> >> JDK11. The remaining known issues in HBase are catalogued in
> HBASE-22972.
> >>
> >
> > Since that blurb was written, Hadoop has added JDK11 support [1] as has
> > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could update
> > our minimum supported versions of these projects to match that support.
> >
> > What do folks think?
> >
> > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> > [2]:
> > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Duo Zhang]

+1 on moving the minimum required java version for compiling HBase to Java
11. But we could still use --release=8 to still support jdk8 at runtime.

Java 8 is still widely used, and I believe this will last for even more
years, as lots of big companies such as RedHat, Microsoft and Amazon are
still shipping new jdk8 releases, I do not think it is time to fully drop
the support of jdk8.

Thanks.

Sean Busbey  于2022年2月16日周三 09:57写道：

> If we set the minJDK to 11 I believe that will effectively remove jdk8
> support, rather than "just" deprecate it.
>
> As we build out more robust testing I would be in favor of running less of
> it on deprecated JDKs.
>
>
> On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
>
> > Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems reasonable to
> > me.
> >
> > Gotta start pushing the issue somehow.
> >
> > On 2/15/22 1:47 PM, Sean Busbey wrote:
> > > Hi folks!
> > >
> > > It's been some time since we decided to stick to LTS JDK releases as a
> > way
> > > of getting a handle on the JDK treadmill.
> > >
> > > What do folks think about deprecating JDK8? The openjdk8u project is
> > still
> > > going and there are commercial support options at least through 2030.
> > >
> > > Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
> > > that we would _have_ to remove it. The way I think about likely timing
> of
> > > these events goes like this:
> > >
> > > * HBase 2 started alphas in June 2017, betas in January 2018, and came
> > out
> > > in April 2018
> > > * HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
> > > discussed how close we are to our stated beta goals (upgrades from
> active
> > > 2.x releases and removal of not-ready features).
> > >
> > > Given the above, in the absence of us specifically pushing to roll
> > through
> > > major version numbers for some reason, I think a reasonably
> conservative
> > > estimate is for HBase 3 to arrive in late 2022 or early 2023 and then
> > HBase
> > > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.
> > >
> > > That all said, our current reference guide section on java versions
> does
> > > not sound very confident about JDK11 support.
> > >
> > >> A Note on JDK11 *
> > >> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> > > support is limited to
> > >> compilation and running the full test suite. There are open questions
> > > regarding the runtime
> > >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> > > (HADOOP-15338).
> > >> Significantly, neither project has yet released a version with
> explicit
> > > runtime support for
> > >> JDK11. The remaining known issues in HBase are catalogued in
> > HBASE-22972.
> > >>
> > >
> > > Since that blurb was written, Hadoop has added JDK11 support [1] as has
> > > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could
> update
> > > our minimum supported versions of these projects to match that support.
> > >
> > > What do folks think?
> > >
> > > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> > > [2]:
> > >
> https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
> > >
> >
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

If we set the minJDK to 11 I believe that will effectively remove jdk8
support, rather than "just" deprecate it.

As we build out more robust testing I would be in favor of running less of
it on deprecated JDKs.


On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:

> Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems reasonable to
> me.
>
> Gotta start pushing the issue somehow.
>
> On 2/15/22 1:47 PM, Sean Busbey wrote:
> > Hi folks!
> >
> > It's been some time since we decided to stick to LTS JDK releases as a
> way
> > of getting a handle on the JDK treadmill.
> >
> > What do folks think about deprecating JDK8? The openjdk8u project is
> still
> > going and there are commercial support options at least through 2030.
> >
> > Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
> > that we would _have_ to remove it. The way I think about likely timing of
> > these events goes like this:
> >
> > * HBase 2 started alphas in June 2017, betas in January 2018, and came
> out
> > in April 2018
> > * HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
> > discussed how close we are to our stated beta goals (upgrades from active
> > 2.x releases and removal of not-ready features).
> >
> > Given the above, in the absence of us specifically pushing to roll
> through
> > major version numbers for some reason, I think a reasonably conservative
> > estimate is for HBase 3 to arrive in late 2022 or early 2023 and then
> HBase
> > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.
> >
> > That all said, our current reference guide section on java versions does
> > not sound very confident about JDK11 support.
> >
> >> A Note on JDK11 *
> >> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> > support is limited to
> >> compilation and running the full test suite. There are open questions
> > regarding the runtime
> >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> > (HADOOP-15338).
> >> Significantly, neither project has yet released a version with explicit
> > runtime support for
> >> JDK11. The remaining known issues in HBase are catalogued in
> HBASE-22972.
> >>
> >
> > Since that blurb was written, Hadoop has added JDK11 support [1] as has
> > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could update
> > our minimum supported versions of these projects to match that support.
> >
> > What do folks think?
> >
> > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> > [2]:
> > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
> >
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Andrew Purtell]

The section in our docs on JDK11 seems out of date. In my experience using Java 
11 to run ZooKeeper (3.5), Hadoop HDFS/YARN/MR (2.10 and 3.3), and HBase (2.4) 
is a nonissue. These software versions are stable under load. Most recently I 
tested scale ingest with 11.0.15+1, hot off the presses so to speak. No issues 
to report — although, I cannot claim this is the production configuration where 
I work yet. Perhaps near the end of 2022. Anyway, I think mine is likely not 
the only experience like this so the hedging language can be removed from the 
text, especially if someone else writes in with positive testimonial. 

Regarding the original question, I would be in favor of the proposal. Time 
matches on. I assume just to state the obvious that our destination of minimum 
LTS would shift from 8 to 11. 

I also agree it would be good to move up from EOL or soon to be EOL versions of 
our dependencies. Looks like ZooKeeper 3.6 will be the lowest live code line by 
the end of this year. Hadoop 2 isn’t exactly dead, at least the source branch 
is still receiving occasional update, but is not releasing. We should probably 
consider it effectively EOL. The Hadoop minimum could become 3.3. The primary 
consideration to my mind is the state of S3A: in what version it can be said to 
be stable and feature complete. I think 3.3 is the appropriate code line for 
that criteria but perhaps 3.2 could serve as well. 

> On Feb 15, 2022, at 10:48 AM, Sean Busbey  wrote:
> 
> Hi folks!
> 
> It's been some time since we decided to stick to LTS JDK releases as a way
> of getting a handle on the JDK treadmill.
> 
> What do folks think about deprecating JDK8? The openjdk8u project is still
> going and there are commercial support options at least through 2030.
> 
> Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
> that we would _have_ to remove it. The way I think about likely timing of
> these events goes like this:
> 
> * HBase 2 started alphas in June 2017, betas in January 2018, and came out
> in April 2018
> * HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
> discussed how close we are to our stated beta goals (upgrades from active
> 2.x releases and removal of not-ready features).
> 
> Given the above, in the absence of us specifically pushing to roll through
> major version numbers for some reason, I think a reasonably conservative
> estimate is for HBase 3 to arrive in late 2022 or early 2023 and then HBase
> 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.
> 
> That all said, our current reference guide section on java versions does
> not sound very confident about JDK11 support.
> 
>> A Note on JDK11 *
>> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> support is limited to
>> compilation and running the full test suite. There are open questions
> regarding the runtime
>> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> (HADOOP-15338).
>> Significantly, neither project has yet released a version with explicit
> runtime support for
>> JDK11. The remaining known issues in HBase are catalogued in HBASE-22972.
>> 
> 
> Since that blurb was written, Hadoop has added JDK11 support [1] as has
> ZooKeeper[2]. As a part of buttoning up our JDK11 support we could update
> our minimum supported versions of these projects to match that support.
> 
> What do folks think?
> 
> [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> [2]:
> https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Josh Elser]

Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems reasonable to me.

Gotta start pushing the issue somehow.

On 2/15/22 1:47 PM, Sean Busbey wrote:

Hi folks!

It's been some time since we decided to stick to LTS JDK releases as a way
of getting a handle on the JDK treadmill.

What do folks think about deprecating JDK8? The openjdk8u project is still
going and there are commercial support options at least through 2030.

Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
that we would _have_ to remove it. The way I think about likely timing of
these events goes like this:

* HBase 2 started alphas in June 2017, betas in January 2018, and came out
in April 2018
* HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
discussed how close we are to our stated beta goals (upgrades from active
2.x releases and removal of not-ready features).

Given the above, in the absence of us specifically pushing to roll through
major version numbers for some reason, I think a reasonably conservative
estimate is for HBase 3 to arrive in late 2022 or early 2023 and then HBase
4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.

That all said, our current reference guide section on java versions does
not sound very confident about JDK11 support.


A Note on JDK11 *
Preliminary support for JDK11 is introduced with HBase 2.3.0. This

support is limited to

compilation and running the full test suite. There are open questions

regarding the runtime

compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop

(HADOOP-15338).

Significantly, neither project has yet released a version with explicit

runtime support for

JDK11. The remaining known issues in HBase are catalogued in HBASE-22972.



Since that blurb was written, Hadoop has added JDK11 support [1] as has
ZooKeeper[2]. As a part of buttoning up our JDK11 support we could update
our minimum supported versions of these projects to match that support.

What do folks think?

[1]: https://hadoop.apache.org/docs/r3.3.0/index.html
[2]:
https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq

[DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

Hi folks!

It's been some time since we decided to stick to LTS JDK releases as a way
of getting a handle on the JDK treadmill.

What do folks think about deprecating JDK8? The openjdk8u project is still
going and there are commercial support options at least through 2030.

Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
that we would _have_ to remove it. The way I think about likely timing of
these events goes like this:

* HBase 2 started alphas in June 2017, betas in January 2018, and came out
in April 2018
* HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
discussed how close we are to our stated beta goals (upgrades from active
2.x releases and removal of not-ready features).

Given the above, in the absence of us specifically pushing to roll through
major version numbers for some reason, I think a reasonably conservative
estimate is for HBase 3 to arrive in late 2022 or early 2023 and then HBase
4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.

That all said, our current reference guide section on java versions does
not sound very confident about JDK11 support.

> A Note on JDK11 *
> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
support is limited to
> compilation and running the full test suite. There are open questions
regarding the runtime
> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
(HADOOP-15338).
> Significantly, neither project has yet released a version with explicit
runtime support for
> JDK11. The remaining known issues in HBase are catalogued in HBASE-22972.
>

Since that blurb was written, Hadoop has added JDK11 support [1] as has
ZooKeeper[2]. As a part of buttoning up our JDK11 support we could update
our minimum supported versions of these projects to match that support.

What do folks think?

[1]: https://hadoop.apache.org/docs/r3.3.0/index.html
[2]:
https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq