Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
> > > > > > > planned to require JDK17. > > > > > > > > > > > > > > Aside: > > > > > > > Andrew, there are Phoenix PRs waiting for reviews which add the > > > > > > > bulk of > > > > > > > changes required for HBase 3.0 support. (Mostly just getting rid > > > > > > > of > > > > > HBase > > > > > > > 1.0 APIs) > > > > > > > I'd appreciate it if someone from your team could review them. See > > > > > > > PHOENIX-7164. I'd like to add support for HBase 3.0 > > > > > > > to Phoenix promptly after 3.0 is released. > > > > > > > > > > > > > > Istvan > > > > > > > > > > > > > > > > > > > > >> On Fri, Mar 7, 2025 at 2:56 AM Andrew Purtell > > > > > > >> > > > > > wrote: > > > > > > >> > > > > > > >> I'm not sure what other user appetites for HBase 3 in production > > > > > > >> are, > > > > > but > > > > > > >> at the $dayjob it's got to be 18 months to 2 years out. First, > > > > > > >> we are > > > > > a > > > > > > >> Phoenix shop, so Phoenix must adapt their code for HBase 3 and > > > > > > >> prepare > > > > > > >> releases based on HBase 3 that will be production ready, and as > > > > > > >> far > > > > > as I > > > > > > >> know this isn't on the radar for them yet. (It's quite possible > > > > > > >> our > > > > > shop > > > > > > >> would do the work, but not until next year at the earliest.) > > > > > > >> Then, we > > > > > must > > > > > > >> update all of our internal code for API differences, and > > > > > > >> rigorously > > > > > test > > > > > > >> incremental upgrade and rollback, until we can move up and back > > > > > seamlessly, > > > > > > >> and that will take time. I think we might consider allocating > > > > > resources for > > > > > > >> it beginning next year. I don't think we are unusual in that > > > > > > >> regard. > > > > > > >> > > > > > > >> All this to say, maintenance changes like a Jetty uplift should > > > > > > >> not be > > > > > > >> restricted to 3.x only, because 2.x users are going to remain on > > > > > > >> 2.x > > > > > for a > > > > > > >> while, and 2.x users have just as much exposure to security > > > > > > >> blockers > > > > > > >> related to end of support versions of Jetty. If the project > > > > > > >> doesn't > > > > > land > > > > > > >> things like Jetty uplift back to 2.x, that's just more work for > > > > > > >> us to > > > > > do > > > > > > >> it, and less bandwidth to pursue more forward looking activities. > > > > > > >> > > > > > > >> For your consideration. > > > > > > >> > > > > > > >> On Thu, Mar 6, 2025 at 5:45 PM 张铎(Duo Zhang) > > > > > > >> > > > > > > >> wrote: > > > > > > >> > > > > > > >>> If we can speed up the 3.x release, since it has already moved > > > > > > >>> up the > > > > > > >>> JDK17+, it will be easier to land these changes in 3.x only. > > > > > > >>> > > > > > > >>> Istvan Toth 于2025年3月7日周五 03:42写道： > > > > > > >>>> > > > > > > >>>> I'd split that into two tickets. > > > > > > >>>> > > > > > > >>>> First move to Jetty 12 while keeping the old javax package > > > > > > >>>> names, > > > > > and > > > > > > >>> once > > > > > > >>>> that's done and tested
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
incremental upgrade and rollback, until we can move up and back > > > > seamlessly, > > > > > >> and that will take time. I think we might consider allocating > > > > resources for > > > > > >> it beginning next year. I don't think we are unusual in that > > > > > >> regard. > > > > > >> > > > > > >> All this to say, maintenance changes like a Jetty uplift should > > > > > >> not be > > > > > >> restricted to 3.x only, because 2.x users are going to remain on > > > > > >> 2.x > > > > for a > > > > > >> while, and 2.x users have just as much exposure to security > > > > > >> blockers > > > > > >> related to end of support versions of Jetty. If the project doesn't > > > > land > > > > > >> things like Jetty uplift back to 2.x, that's just more work for us > > > > > >> to > > > > do > > > > > >> it, and less bandwidth to pursue more forward looking activities. > > > > > >> > > > > > >> For your consideration. > > > > > >> > > > > > >> On Thu, Mar 6, 2025 at 5:45 PM 张铎(Duo Zhang) > > > > > >> > > > > > >> wrote: > > > > > >> > > > > > >>> If we can speed up the 3.x release, since it has already moved up > > > > > >>> the > > > > > >>> JDK17+, it will be easier to land these changes in 3.x only. > > > > > >>> > > > > > >>> Istvan Toth 于2025年3月7日周五 03:42写道： > > > > > >>>> > > > > > >>>> I'd split that into two tickets. > > > > > >>>> > > > > > >>>> First move to Jetty 12 while keeping the old javax package names, > > > > and > > > > > >>> once > > > > > >>>> that's done and tested, we can look into moving to jakarta. > > > > > >>>> > > > > > >>>> Istvan > > > > > >>>> > > > > > >>>> On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell > > > > > >>>> > > > > > >>> wrote: > > > > > >>>> > > > > > >>>>>> Therefore, I plan to use the EE8 environment on Jetty 12. > > > > > >>>>> > > > > > >>>>> Ah, thanks for mentioning EE. > > > > > >>>>> > > > > > >>>>> Do you have a requirement to stay with EE 8 ? > > > > > >>>>> > > > > > >>>>> If we could target EE 9 instead of EE 8 that would align with > > > > > >>>>> some > > > > > >>> internal > > > > > >>>>> requirements at $dayjob. For what it's worth. Mostly the point > > > > > >>>>> of > > > > EE > > > > > >> 9 > > > > > >>> is a > > > > > >>>>> move of the EE stuff into a new namespace, allowing for further > > > > > >>> development > > > > > >>>>> under the Eclipse Foundation without trademark conflicts with > > > > > >> Oracle's > > > > > >>>>> "Java" branding. The trademark issue and follow on Java > > > > > >>>>> licensing > > > > > >>> concerns > > > > > >>>>> are going to be relevant for anyone who might be visited by an > > > > Oracle > > > > > >>>>> lawyer-shark. > > > > > >>>>> > > > > > >>>>> On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain > > > > > >>> wrote: > > > > > >>>>> > > > > > >>>>>>> I think that the best course of action would be adding a > > > > > >>>>>>> separate > > > > > >>>>> Jetty11 > > > > > >>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be > > > > > >>>>>> both > > > > > >>>>&g
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
an look into moving to jakarta. > > > > >>>> > > > > >>>> Istvan > > > > >>>> > > > > >>>> On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell > > > > >>> wrote: > > > > >>>> > > > > >>>>>> Therefore, I plan to use the EE8 environment on Jetty 12. > > > > >>>>> > > > > >>>>> Ah, thanks for mentioning EE. > > > > >>>>> > > > > >>>>> Do you have a requirement to stay with EE 8 ? > > > > >>>>> > > > > >>>>> If we could target EE 9 instead of EE 8 that would align with some > > > > >>> internal > > > > >>>>> requirements at $dayjob. For what it's worth. Mostly the point of > > > EE > > > > >> 9 > > > > >>> is a > > > > >>>>> move of the EE stuff into a new namespace, allowing for further > > > > >>> development > > > > >>>>> under the Eclipse Foundation without trademark conflicts with > > > > >> Oracle's > > > > >>>>> "Java" branding. The trademark issue and follow on Java licensing > > > > >>> concerns > > > > >>>>> are going to be relevant for anyone who might be visited by an > > > Oracle > > > > >>>>> lawyer-shark. > > > > >>>>> > > > > >>>>> On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain > > > > >>> wrote: > > > > >>>>> > > > > >>>>>>> I think that the best course of action would be adding a > > > > >>>>>>> separate > > > > >>>>> Jetty11 > > > > >>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > > > >>>>>> used/updated by the 2.x / 3.x branches. > > > > >>>>>> > > > > >>>>>> +1, I would like to volunteer for this work. > > > > >>>>>> > > > > >>>>>> Maintaining Jetty 9.4.x could be quite challenging from a CVE > > > > >>> perspective > > > > >>>>>> since it has been EOCS (End of Community Support) since June > > > > >>>>>> 2022, > > > > >>> and > > > > >>>>> new > > > > >>>>>> releases may not be forthcoming for a long time. I faced similar > > > > >>> issues > > > > >>>>>> during our last hbase-thirdparty release. See this issue comment: > > > > >>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > > > > >>>>>> > > > > >>>>>> Regarding the version and CVEs, I agree with @Andrew and suggest > > > > >>> that we > > > > >>>>>> jump directly to Jetty 12, bypassing Jetty 11, to support > > > > >>> javax.servlet > > > > >>>>> for > > > > >>>>>> JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty > > > > >> 12. > > > > >>> See > > > > >>>>>> Jetty version history: > > > > >>> https://jetty.org/download.html#version-history > > > > >>>>>> > > > > >>>>>> @Istvan, is there any particular reason you recommend moving to > > > > >>> Jetty 11, > > > > >>>>>> which is also EOCS? > > > > >>>>>> > > > > >>>>>> If others are fine with me taking this up, I can create the > > > > >> necessary > > > > >>>>>> JIRAs for the Jetty migration project and start the work soon. > > > > >>>>>> > > > > >>>>>> Regards, > > > > >>>>>> Nihal > > > > >>>>>> > > > > >>>>>> On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote: > > > > >>>>>>> FYI I think Hadoo
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
f Jetty. If the project doesn't > > > land > > > > >> things like Jetty uplift back to 2.x, that's just more work for us to > > > do > > > > >> it, and less bandwidth to pursue more forward looking activities. > > > > >> > > > > >> For your consideration. > > > > >> > > > > >> On Thu, Mar 6, 2025 at 5:45 PM 张铎(Duo Zhang) > > > > >> wrote: > > > > >> > > > > >>> If we can speed up the 3.x release, since it has already moved up > > > > >>> the > > > > >>> JDK17+, it will be easier to land these changes in 3.x only. > > > > >>> > > > > >>> Istvan Toth 于2025年3月7日周五 03:42写道： > > > > >>>> > > > > >>>> I'd split that into two tickets. > > > > >>>> > > > > >>>> First move to Jetty 12 while keeping the old javax package names, > > > and > > > > >>> once > > > > >>>> that's done and tested, we can look into moving to jakarta. > > > > >>>> > > > > >>>> Istvan > > > > >>>> > > > > >>>> On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell > > > > >>> wrote: > > > > >>>> > > > > >>>>>> Therefore, I plan to use the EE8 environment on Jetty 12. > > > > >>>>> > > > > >>>>> Ah, thanks for mentioning EE. > > > > >>>>> > > > > >>>>> Do you have a requirement to stay with EE 8 ? > > > > >>>>> > > > > >>>>> If we could target EE 9 instead of EE 8 that would align with some > > > > >>> internal > > > > >>>>> requirements at $dayjob. For what it's worth. Mostly the point of > > > EE > > > > >> 9 > > > > >>> is a > > > > >>>>> move of the EE stuff into a new namespace, allowing for further > > > > >>> development > > > > >>>>> under the Eclipse Foundation without trademark conflicts with > > > > >> Oracle's > > > > >>>>> "Java" branding. The trademark issue and follow on Java licensing > > > > >>> concerns > > > > >>>>> are going to be relevant for anyone who might be visited by an > > > Oracle > > > > >>>>> lawyer-shark. > > > > >>>>> > > > > >>>>> On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain > > > > >>> wrote: > > > > >>>>> > > > > >>>>>>> I think that the best course of action would be adding a > > > > >>>>>>> separate > > > > >>>>> Jetty11 > > > > >>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > > > >>>>>> used/updated by the 2.x / 3.x branches. > > > > >>>>>> > > > > >>>>>> +1, I would like to volunteer for this work. > > > > >>>>>> > > > > >>>>>> Maintaining Jetty 9.4.x could be quite challenging from a CVE > > > > >>> perspective > > > > >>>>>> since it has been EOCS (End of Community Support) since June > > > > >>>>>> 2022, > > > > >>> and > > > > >>>>> new > > > > >>>>>> releases may not be forthcoming for a long time. I faced similar > > > > >>> issues > > > > >>>>>> during our last hbase-thirdparty release. See this issue comment: > > > > >>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > > > > >>>>>> > > > > >>>>>> Regarding the version and CVEs, I agree with @Andrew and suggest > > > > >>> that we > > > > >>>>>> jump directly to Jetty 12, bypassing Jetty 11, to support > > > > >>> javax.servlet > > > > >>>
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
t;> Do you have a requirement to stay with EE 8 ? > > > >>>>> > > > >>>>> If we could target EE 9 instead of EE 8 that would align with some > > > >>> internal > > > >>>>> requirements at $dayjob. For what it's worth. Mostly the point of > > EE > > > >> 9 > > > >>> is a > > > >>>>> move of the EE stuff into a new namespace, allowing for further > > > >>> development > > > >>>>> under the Eclipse Foundation without trademark conflicts with > > > >> Oracle's > > > >>>>> "Java" branding. The trademark issue and follow on Java licensing > > > >>> concerns > > > >>>>> are going to be relevant for anyone who might be visited by an > > Oracle > > > >>>>> lawyer-shark. > > > >>>>> > > > >>>>> On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain > > > >>> wrote: > > > >>>>> > > > >>>>>>> I think that the best course of action would be adding a separate > > > >>>>> Jetty11 > > > >>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > > >>>>>> used/updated by the 2.x / 3.x branches. > > > >>>>>> > > > >>>>>> +1, I would like to volunteer for this work. > > > >>>>>> > > > >>>>>> Maintaining Jetty 9.4.x could be quite challenging from a CVE > > > >>> perspective > > > >>>>>> since it has been EOCS (End of Community Support) since June 2022, > > > >>> and > > > >>>>> new > > > >>>>>> releases may not be forthcoming for a long time. I faced similar > > > >>> issues > > > >>>>>> during our last hbase-thirdparty release. See this issue comment: > > > >>>>>> > > > >>>>> > > > >>> > > > >> > > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > > > >>>>>> > > > >>>>>> Regarding the version and CVEs, I agree with @Andrew and suggest > > > >>> that we > > > >>>>>> jump directly to Jetty 12, bypassing Jetty 11, to support > > > >>> javax.servlet > > > >>>>> for > > > >>>>>> JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty > > > >> 12. > > > >>> See > > > >>>>>> Jetty version history: > > > >>> https://jetty.org/download.html#version-history > > > >>>>>> > > > >>>>>> @Istvan, is there any particular reason you recommend moving to > > > >>> Jetty 11, > > > >>>>>> which is also EOCS? > > > >>>>>> > > > >>>>>> If others are fine with me taking this up, I can create the > > > >> necessary > > > >>>>>> JIRAs for the Jetty migration project and start the work soon. > > > >>>>>> > > > >>>>>> Regards, > > > >>>>>> Nihal > > > >>>>>> > > > >>>>>> On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote: > > > >>>>>>> FYI I think Hadoop will need to make similar moves too. > > > >>>>>>> Sounds like Hadoop should start the work to drop JDK8 as well. I > > > >>> know > > > >>>>>> Steve > > > >>>>>>> has a patch that requires JDK17+ due to Iceberg > > > >>>>>>> https://github.com/apache/hadoop/pull/7316 > > > >>>>>>> > > > >>>>>>> -- Forwarded message - > > > >>>>>>> From: Istvan Toth > > > >>>>>>> Date: Sun, Mar 2, 2025 at 9:41 PM > > > >>>>>>> Subject: [DISCUSS] Plans for JDK22 / SecurityManager removal > > > >>>>>>> To: HBase Dev List > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> Hi! > > > >>>>>>>
Re: Fwd: [DISCUSS] Plans for JDK22 / SecurityManager removal
> Therefore, I plan to use the EE8 environment on Jetty 12. Ah, thanks for mentioning EE. Do you have a requirement to stay with EE 8 ? If we could target EE 9 instead of EE 8 that would align with some internal requirements at $dayjob. For what it's worth. Mostly the point of EE 9 is a move of the EE stuff into a new namespace, allowing for further development under the Eclipse Foundation without trademark conflicts with Oracle's "Java" branding. The trademark issue and follow on Java licensing concerns are going to be relevant for anyone who might be visited by an Oracle lawyer-shark. On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain wrote: > > I think that the best course of action would be adding a separate Jetty11 > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > used/updated by the 2.x / 3.x branches. > > +1, I would like to volunteer for this work. > > Maintaining Jetty 9.4.x could be quite challenging from a CVE perspective > since it has been EOCS (End of Community Support) since June 2022, and new > releases may not be forthcoming for a long time. I faced similar issues > during our last hbase-thirdparty release. See this issue comment: > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > > Regarding the version and CVEs, I agree with @Andrew and suggest that we > jump directly to Jetty 12, bypassing Jetty 11, to support javax.servlet for > JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty 12. See > Jetty version history: https://jetty.org/download.html#version-history > > @Istvan, is there any particular reason you recommend moving to Jetty 11, > which is also EOCS? > > If others are fine with me taking this up, I can create the necessary > JIRAs for the Jetty migration project and start the work soon. > > Regards, > Nihal > > On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote: > > FYI I think Hadoop will need to make similar moves too. > > Sounds like Hadoop should start the work to drop JDK8 as well. I know > Steve > > has a patch that requires JDK17+ due to Iceberg > > https://github.com/apache/hadoop/pull/7316 > > > > -- Forwarded message - > > From: Istvan Toth > > Date: Sun, Mar 2, 2025 at 9:41 PM > > Subject: [DISCUSS] Plans for JDK22 / SecurityManager removal > > To: HBase Dev List > > > > > > Hi! > > > > The last big incompatible JDK change is the JEP411/JEP486 SecurityManager > > removal process, which has gotten serious with JDK22, which disables > > Subject.doAs*() Subject.getSubject() by default (while providing the kind > > of equivalent callAs() and current() methods), and kills automatic > Subject > > propagation to new threads. > > > > To add insult to injury the new methods are only available from JDK18, so > > it is not possible to just move to the new API without breaking JDK17. > > > > Some of this can only be solved in Hadoop (hopefully 3.5) , but HBase can > > also take steps to move towards JDK22 compatibility in the meantime. > > > > Here's the plan I have in mind: > > > > - Review and the dependencies WRT SecurityManage usage (just search for > API > > calls) > > - Update the dependencies as needed > > > > - Add a reflection-based compatibility Utility class based on Jetty's > > SecurityUtils, (or one of its derivatives) > > - Replace the deprecated method calls with the compatibility versions > > - Wrap Runnables/Callables with Subject current()/callAs() methods. > > > > One large dependency I have identified is Jetty. > > While technically we MAY be able to keep using 9.4 by overriding its > > default ThreadFactory, I think that considering that even semi-formal > Jetty > > 9.4 support is on its very last legs (the latest update was never even > > announced and is not listed in the downloads page), the advantages of > > updating to Jetty 11 on branch-3+ now far outweighs the added maintenance > > burden of the branch divergence. > > > > I think that the best course of action would be adding a separate Jetty11 > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > used/updated by the 2.x / 3.x branches. > > > > What do you think ? > > > > - Do you agree that JEP411/486 should be treated as a priority ? > > - Would you volunteer for the dependency audit ? > > - Do you agree with the Jetty update, and the plan outlined above ? > > > > Istvan > > > > ps: Sorry, no links because GMail seems to react very bady to any. Please > > use search. > > > > - > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Best regards, Andrew Unrest, ignorance distilled, nihilistic imbeciles - It's what we’ve earned Welcome, apocalypse, what’s taken you so long? Bring us the fitting end that we’ve been counting on - A23, Welcome, Apocalypse
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
> > > > > > >> On Fri, Mar 7, 2025 at 2:56 AM Andrew Purtell > wrote: > > >> > > >> I'm not sure what other user appetites for HBase 3 in production are, > but > > >> at the $dayjob it's got to be 18 months to 2 years out. First, we are > a > > >> Phoenix shop, so Phoenix must adapt their code for HBase 3 and prepare > > >> releases based on HBase 3 that will be production ready, and as far > as I > > >> know this isn't on the radar for them yet. (It's quite possible our > shop > > >> would do the work, but not until next year at the earliest.) Then, we > must > > >> update all of our internal code for API differences, and rigorously > test > > >> incremental upgrade and rollback, until we can move up and back > seamlessly, > > >> and that will take time. I think we might consider allocating > resources for > > >> it beginning next year. I don't think we are unusual in that regard. > > >> > > >> All this to say, maintenance changes like a Jetty uplift should not be > > >> restricted to 3.x only, because 2.x users are going to remain on 2.x > for a > > >> while, and 2.x users have just as much exposure to security blockers > > >> related to end of support versions of Jetty. If the project doesn't > land > > >> things like Jetty uplift back to 2.x, that's just more work for us to > do > > >> it, and less bandwidth to pursue more forward looking activities. > > >> > > >> For your consideration. > > >> > > >> On Thu, Mar 6, 2025 at 5:45 PM 张铎(Duo Zhang) > > >> wrote: > > >> > > >>> If we can speed up the 3.x release, since it has already moved up the > > >>> JDK17+, it will be easier to land these changes in 3.x only. > > >>> > > >>> Istvan Toth 于2025年3月7日周五 03:42写道： > > >>>> > > >>>> I'd split that into two tickets. > > >>>> > > >>>> First move to Jetty 12 while keeping the old javax package names, > and > > >>> once > > >>>> that's done and tested, we can look into moving to jakarta. > > >>>> > > >>>> Istvan > > >>>> > > >>>> On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell > > >>> wrote: > > >>>> > > >>>>>> Therefore, I plan to use the EE8 environment on Jetty 12. > > >>>>> > > >>>>> Ah, thanks for mentioning EE. > > >>>>> > > >>>>> Do you have a requirement to stay with EE 8 ? > > >>>>> > > >>>>> If we could target EE 9 instead of EE 8 that would align with some > > >>> internal > > >>>>> requirements at $dayjob. For what it's worth. Mostly the point of > EE > > >> 9 > > >>> is a > > >>>>> move of the EE stuff into a new namespace, allowing for further > > >>> development > > >>>>> under the Eclipse Foundation without trademark conflicts with > > >> Oracle's > > >>>>> "Java" branding. The trademark issue and follow on Java licensing > > >>> concerns > > >>>>> are going to be relevant for anyone who might be visited by an > Oracle > > >>>>> lawyer-shark. > > >>>>> > > >>>>> On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain > > >>> wrote: > > >>>>> > > >>>>>>> I think that the best course of action would be adding a separate > > >>>>> Jetty11 > > >>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > >>>>>> used/updated by the 2.x / 3.x branches. > > >>>>>> > > >>>>>> +1, I would like to volunteer for this work. > > >>>>>> > > >>>>>> Maintaining Jetty 9.4.x could be quite challenging from a CVE > > >>> perspective > > >>>>>> since it has been EOCS (End of Community Support) since June 2022, > > >>> and > > >>>>> new > > >>>>>> releases may not be forthcoming for a long time. I faced similar > > >>> issues > > >>>>>> during our last hbase-thirdparty release. See this i
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
>> incremental upgrade and rollback, until we can move up and back seamlessly, > >> and that will take time. I think we might consider allocating resources for > >> it beginning next year. I don't think we are unusual in that regard. > >> > >> All this to say, maintenance changes like a Jetty uplift should not be > >> restricted to 3.x only, because 2.x users are going to remain on 2.x for a > >> while, and 2.x users have just as much exposure to security blockers > >> related to end of support versions of Jetty. If the project doesn't land > >> things like Jetty uplift back to 2.x, that's just more work for us to do > >> it, and less bandwidth to pursue more forward looking activities. > >> > >> For your consideration. > >> > >> On Thu, Mar 6, 2025 at 5:45 PM 张铎(Duo Zhang) > >> wrote: > >> > >>> If we can speed up the 3.x release, since it has already moved up the > >>> JDK17+, it will be easier to land these changes in 3.x only. > >>> > >>> Istvan Toth 于2025年3月7日周五 03:42写道： > >>>> > >>>> I'd split that into two tickets. > >>>> > >>>> First move to Jetty 12 while keeping the old javax package names, and > >>> once > >>>> that's done and tested, we can look into moving to jakarta. > >>>> > >>>> Istvan > >>>> > >>>> On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell > >>> wrote: > >>>> > >>>>>> Therefore, I plan to use the EE8 environment on Jetty 12. > >>>>> > >>>>> Ah, thanks for mentioning EE. > >>>>> > >>>>> Do you have a requirement to stay with EE 8 ? > >>>>> > >>>>> If we could target EE 9 instead of EE 8 that would align with some > >>> internal > >>>>> requirements at $dayjob. For what it's worth. Mostly the point of EE > >> 9 > >>> is a > >>>>> move of the EE stuff into a new namespace, allowing for further > >>> development > >>>>> under the Eclipse Foundation without trademark conflicts with > >> Oracle's > >>>>> "Java" branding. The trademark issue and follow on Java licensing > >>> concerns > >>>>> are going to be relevant for anyone who might be visited by an Oracle > >>>>> lawyer-shark. > >>>>> > >>>>> On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain > >>> wrote: > >>>>> > >>>>>>> I think that the best course of action would be adding a separate > >>>>> Jetty11 > >>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > >>>>>> used/updated by the 2.x / 3.x branches. > >>>>>> > >>>>>> +1, I would like to volunteer for this work. > >>>>>> > >>>>>> Maintaining Jetty 9.4.x could be quite challenging from a CVE > >>> perspective > >>>>>> since it has been EOCS (End of Community Support) since June 2022, > >>> and > >>>>> new > >>>>>> releases may not be forthcoming for a long time. I faced similar > >>> issues > >>>>>> during our last hbase-thirdparty release. See this issue comment: > >>>>>> > >>>>> > >>> > >> https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > >>>>>> > >>>>>> Regarding the version and CVEs, I agree with @Andrew and suggest > >>> that we > >>>>>> jump directly to Jetty 12, bypassing Jetty 11, to support > >>> javax.servlet > >>>>> for > >>>>>> JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty > >> 12. > >>> See > >>>>>> Jetty version history: > >>> https://jetty.org/download.html#version-history > >>>>>> > >>>>>> @Istvan, is there any particular reason you recommend moving to > >>> Jetty 11, > >>>>>> which is also EOCS? > >>>>>> > >>>>>> If others are fine with me taking this up, I can create the > >> necessary > >>>>>> JIRAs for the Jetty migration project and start the work soon.
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
;>> >>>>>> Maintaining Jetty 9.4.x could be quite challenging from a CVE >>> perspective >>>>>> since it has been EOCS (End of Community Support) since June 2022, >>> and >>>>> new >>>>>> releases may not be forthcoming for a long time. I faced similar >>> issues >>>>>> during our last hbase-thirdparty release. See this issue comment: >>>>>> >>>>> >>> >> https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 >>>>>> >>>>>> Regarding the version and CVEs, I agree with @Andrew and suggest >>> that we >>>>>> jump directly to Jetty 12, bypassing Jetty 11, to support >>> javax.servlet >>>>> for >>>>>> JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty >> 12. >>> See >>>>>> Jetty version history: >>> https://jetty.org/download.html#version-history >>>>>> >>>>>> @Istvan, is there any particular reason you recommend moving to >>> Jetty 11, >>>>>> which is also EOCS? >>>>>> >>>>>> If others are fine with me taking this up, I can create the >> necessary >>>>>> JIRAs for the Jetty migration project and start the work soon. >>>>>> >>>>>> Regards, >>>>>> Nihal >>>>>> >>>>>> On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote: >>>>>>> FYI I think Hadoop will need to make similar moves too. >>>>>>> Sounds like Hadoop should start the work to drop JDK8 as well. I >>> know >>>>>> Steve >>>>>>> has a patch that requires JDK17+ due to Iceberg >>>>>>> https://github.com/apache/hadoop/pull/7316 >>>>>>> >>>>>>> -- Forwarded message - >>>>>>> From: Istvan Toth >>>>>>> Date: Sun, Mar 2, 2025 at 9:41 PM >>>>>>> Subject: [DISCUSS] Plans for JDK22 / SecurityManager removal >>>>>>> To: HBase Dev List >>>>>>> >>>>>>> >>>>>>> Hi! >>>>>>> >>>>>>> The last big incompatible JDK change is the JEP411/JEP486 >>>>> SecurityManager >>>>>>> removal process, which has gotten serious with JDK22, which >>> disables >>>>>>> Subject.doAs*() Subject.getSubject() by default (while providing >>> the >>>>> kind >>>>>>> of equivalent callAs() and current() methods), and kills >> automatic >>>>>> Subject >>>>>>> propagation to new threads. >>>>>>> >>>>>>> To add insult to injury the new methods are only available from >>> JDK18, >>>>> so >>>>>>> it is not possible to just move to the new API without breaking >>> JDK17. >>>>>>> >>>>>>> Some of this can only be solved in Hadoop (hopefully 3.5) , but >>> HBase >>>>> can >>>>>>> also take steps to move towards JDK22 compatibility in the >>> meantime. >>>>>>> >>>>>>> Here's the plan I have in mind: >>>>>>> >>>>>>> - Review and the dependencies WRT SecurityManage usage (just >>> search for >>>>>> API >>>>>>> calls) >>>>>>> - Update the dependencies as needed >>>>>>> >>>>>>> - Add a reflection-based compatibility Utility class based on >>> Jetty's >>>>>>> SecurityUtils, (or one of its derivatives) >>>>>>> - Replace the deprecated method calls with the compatibility >>> versions >>>>>>> - Wrap Runnables/Callables with Subject current()/callAs() >> methods. >>>>>>> >>>>>>> One large dependency I have identified is Jetty. >>>>>>> While technically we MAY be able to keep using 9.4 by overriding >>> its >>>>>>> default ThreadFactory, I think that considering that even >>> semi-formal >>>>>> Jetty >>>>>>> 9.4 support is on its very last legs (the latest update was never >>> even >>>>>>> announced and is not listed in the downloads page), the >>> advantages of >>>>>>> updating to Jetty 11 on branch-3+ now far outweighs the added >>>>> maintenance >>>>>>> burden of the branch divergence. >>>>>>> >>>>>>> I think that the best course of action would be adding a separate >>>>> Jetty11 >>>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both >>>>>>> used/updated by the 2.x / 3.x branches. >>>>>>> >>>>>>> What do you think ? >>>>>>> >>>>>>> - Do you agree that JEP411/486 should be treated as a priority ? >>>>>>> - Would you volunteer for the dependency audit ? >>>>>>> - Do you agree with the Jetty update, and the plan outlined >> above ? >>>>>>> >>>>>>> Istvan >>>>>>> >>>>>>> ps: Sorry, no links because GMail seems to react very bady to >> any. >>>>> Please >>>>>>> use search. >>>>>>> >>>>>> >>>>>> >> - >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> Andrew >>>>> >>>>> Unrest, ignorance distilled, nihilistic imbeciles - >>>>>It's what we’ve earned >>>>> Welcome, apocalypse, what’s taken you so long? >>>>> Bring us the fitting end that we’ve been counting on >>>>> - A23, Welcome, Apocalypse >>>>> >>>> >>>> >>>> -- >>>> *István Tóth* | Sr. Staff Software Engineer >>>> *Email*: [email protected] >>>> cloudera.com <https://www.cloudera.com> >>>> [image: Cloudera] <https://www.cloudera.com/> >>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: >>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: >>> Cloudera >>>> on LinkedIn] <https://www.linkedin.com/company/cloudera> >>>> -- >>>> -- >>> >> >> >> -- >> Best regards, >> Andrew >> >> Unrest, ignorance distilled, nihilistic imbeciles - >>It's what we’ve earned >> Welcome, apocalypse, what’s taken you so long? >> Bring us the fitting end that we’ve been counting on >> - A23, Welcome, Apocalypse >> > > > -- > *István Tóth* | Sr. Staff Software Engineer > *Email*: [email protected] > cloudera.com <https://www.cloudera.com> > [image: Cloudera] <https://www.cloudera.com/> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera > on LinkedIn] <https://www.linkedin.com/company/cloudera> > -- > --
Re: Fwd: [DISCUSS] Plans for JDK22 / SecurityManager removal
> > jump directly to Jetty 12, bypassing Jetty 11, to support > > javax.servlet > > > > for > > > > > JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty > 12. > > See > > > > > Jetty version history: > > https://jetty.org/download.html#version-history > > > > > > > > > > @Istvan, is there any particular reason you recommend moving to > > Jetty 11, > > > > > which is also EOCS? > > > > > > > > > > If others are fine with me taking this up, I can create the > necessary > > > > > JIRAs for the Jetty migration project and start the work soon. > > > > > > > > > > Regards, > > > > > Nihal > > > > > > > > > > On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote: > > > > > > FYI I think Hadoop will need to make similar moves too. > > > > > > Sounds like Hadoop should start the work to drop JDK8 as well. I > > know > > > > > Steve > > > > > > has a patch that requires JDK17+ due to Iceberg > > > > > > https://github.com/apache/hadoop/pull/7316 > > > > > > > > > > > > -- Forwarded message - > > > > > > From: Istvan Toth > > > > > > Date: Sun, Mar 2, 2025 at 9:41 PM > > > > > > Subject: [DISCUSS] Plans for JDK22 / SecurityManager removal > > > > > > To: HBase Dev List > > > > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > The last big incompatible JDK change is the JEP411/JEP486 > > > > SecurityManager > > > > > > removal process, which has gotten serious with JDK22, which > > disables > > > > > > Subject.doAs*() Subject.getSubject() by default (while providing > > the > > > > kind > > > > > > of equivalent callAs() and current() methods), and kills > automatic > > > > > Subject > > > > > > propagation to new threads. > > > > > > > > > > > > To add insult to injury the new methods are only available from > > JDK18, > > > > so > > > > > > it is not possible to just move to the new API without breaking > > JDK17. > > > > > > > > > > > > Some of this can only be solved in Hadoop (hopefully 3.5) , but > > HBase > > > > can > > > > > > also take steps to move towards JDK22 compatibility in the > > meantime. > > > > > > > > > > > > Here's the plan I have in mind: > > > > > > > > > > > > - Review and the dependencies WRT SecurityManage usage (just > > search for > > > > > API > > > > > > calls) > > > > > > - Update the dependencies as needed > > > > > > > > > > > > - Add a reflection-based compatibility Utility class based on > > Jetty's > > > > > > SecurityUtils, (or one of its derivatives) > > > > > > - Replace the deprecated method calls with the compatibility > > versions > > > > > > - Wrap Runnables/Callables with Subject current()/callAs() > methods. > > > > > > > > > > > > One large dependency I have identified is Jetty. > > > > > > While technically we MAY be able to keep using 9.4 by overriding > > its > > > > > > default ThreadFactory, I think that considering that even > > semi-formal > > > > > Jetty > > > > > > 9.4 support is on its very last legs (the latest update was never > > even > > > > > > announced and is not listed in the downloads page), the > > advantages of > > > > > > updating to Jetty 11 on branch-3+ now far outweighs the added > > > > maintenance > > > > > > burden of the branch divergence. > > > > > > > > > > > > I think that the best course of action would be adding a separate > > > > Jetty11 > > > > > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > > > > > used/updated by the 2.x / 3.x branches. > > > > > > > > > > > > What do you think ? > > > > > > > > > > > > - Do you agree that JEP411/486 should be treated as a priority ? > > > > > > - Would you volunteer for the dependency audit ? > > > > > > - Do you agree with the Jetty update, and the plan outlined > above ? > > > > > > > > > > > > Istvan > > > > > > > > > > > > ps: Sorry, no links because GMail seems to react very bady to > any. > > > > Please > > > > > > use search. > > > > > > > > > > > > > > > > > - > > > > > To unsubscribe, e-mail: [email protected] > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Andrew > > > > > > > > Unrest, ignorance distilled, nihilistic imbeciles - > > > > It's what we’ve earned > > > > Welcome, apocalypse, what’s taken you so long? > > > > Bring us the fitting end that we’ve been counting on > > > >- A23, Welcome, Apocalypse > > > > > > > > > > > > > -- > > > *István Tóth* | Sr. Staff Software Engineer > > > *Email*: [email protected] > > > cloudera.com <https://www.cloudera.com> > > > [image: Cloudera] <https://www.cloudera.com/> > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > > Cloudera > > > on LinkedIn] <https://www.linkedin.com/company/cloudera> > > > -- > > > -- > > > > > -- > Best regards, > Andrew > > Unrest, ignorance distilled, nihilistic imbeciles - > It's what we’ve earned > Welcome, apocalypse, what’s taken you so long? > Bring us the fitting end that we’ve been counting on >- A23, Welcome, Apocalypse > -- *István Tóth* | Sr. Staff Software Engineer *Email*: [email protected] cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> -- --
Re: Fwd: [DISCUSS] Plans for JDK22 / SecurityManager removal
If we can speed up the 3.x release, since it has already moved up the JDK17+, it will be easier to land these changes in 3.x only. Istvan Toth 于2025年3月7日周五 03:42写道： > > I'd split that into two tickets. > > First move to Jetty 12 while keeping the old javax package names, and once > that's done and tested, we can look into moving to jakarta. > > Istvan > > On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell wrote: > > > > Therefore, I plan to use the EE8 environment on Jetty 12. > > > > Ah, thanks for mentioning EE. > > > > Do you have a requirement to stay with EE 8 ? > > > > If we could target EE 9 instead of EE 8 that would align with some internal > > requirements at $dayjob. For what it's worth. Mostly the point of EE 9 is a > > move of the EE stuff into a new namespace, allowing for further development > > under the Eclipse Foundation without trademark conflicts with Oracle's > > "Java" branding. The trademark issue and follow on Java licensing concerns > > are going to be relevant for anyone who might be visited by an Oracle > > lawyer-shark. > > > > On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain wrote: > > > > > > I think that the best course of action would be adding a separate > > Jetty11 > > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > > used/updated by the 2.x / 3.x branches. > > > > > > +1, I would like to volunteer for this work. > > > > > > Maintaining Jetty 9.4.x could be quite challenging from a CVE perspective > > > since it has been EOCS (End of Community Support) since June 2022, and > > new > > > releases may not be forthcoming for a long time. I faced similar issues > > > during our last hbase-thirdparty release. See this issue comment: > > > > > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > > > > > > Regarding the version and CVEs, I agree with @Andrew and suggest that we > > > jump directly to Jetty 12, bypassing Jetty 11, to support javax.servlet > > for > > > JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty 12. See > > > Jetty version history: https://jetty.org/download.html#version-history > > > > > > @Istvan, is there any particular reason you recommend moving to Jetty 11, > > > which is also EOCS? > > > > > > If others are fine with me taking this up, I can create the necessary > > > JIRAs for the Jetty migration project and start the work soon. > > > > > > Regards, > > > Nihal > > > > > > On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote: > > > > FYI I think Hadoop will need to make similar moves too. > > > > Sounds like Hadoop should start the work to drop JDK8 as well. I know > > > Steve > > > > has a patch that requires JDK17+ due to Iceberg > > > > https://github.com/apache/hadoop/pull/7316 > > > > > > > > -- Forwarded message - > > > > From: Istvan Toth > > > > Date: Sun, Mar 2, 2025 at 9:41 PM > > > > Subject: [DISCUSS] Plans for JDK22 / SecurityManager removal > > > > To: HBase Dev List > > > > > > > > > > > > Hi! > > > > > > > > The last big incompatible JDK change is the JEP411/JEP486 > > SecurityManager > > > > removal process, which has gotten serious with JDK22, which disables > > > > Subject.doAs*() Subject.getSubject() by default (while providing the > > kind > > > > of equivalent callAs() and current() methods), and kills automatic > > > Subject > > > > propagation to new threads. > > > > > > > > To add insult to injury the new methods are only available from JDK18, > > so > > > > it is not possible to just move to the new API without breaking JDK17. > > > > > > > > Some of this can only be solved in Hadoop (hopefully 3.5) , but HBase > > can > > > > also take steps to move towards JDK22 compatibility in the meantime. > > > > > > > > Here's the plan I have in mind: > > > > > > > > - Review and the dependencies WRT SecurityManage usage (just search for > > > API > > > > calls) > > > > - Update the dependencies as needed > > > > > > > > - Add a reflection-based compatibility Utility class based on Jetty's > > > > SecurityUtils, (or one of its derivatives) > > > > - Replace the deprecated meth
Re: Fwd: [DISCUSS] Plans for JDK22 / SecurityManager removal
I'm not sure what other user appetites for HBase 3 in production are, but at the $dayjob it's got to be 18 months to 2 years out. First, we are a Phoenix shop, so Phoenix must adapt their code for HBase 3 and prepare releases based on HBase 3 that will be production ready, and as far as I know this isn't on the radar for them yet. (It's quite possible our shop would do the work, but not until next year at the earliest.) Then, we must update all of our internal code for API differences, and rigorously test incremental upgrade and rollback, until we can move up and back seamlessly, and that will take time. I think we might consider allocating resources for it beginning next year. I don't think we are unusual in that regard. All this to say, maintenance changes like a Jetty uplift should not be restricted to 3.x only, because 2.x users are going to remain on 2.x for a while, and 2.x users have just as much exposure to security blockers related to end of support versions of Jetty. If the project doesn't land things like Jetty uplift back to 2.x, that's just more work for us to do it, and less bandwidth to pursue more forward looking activities. For your consideration. On Thu, Mar 6, 2025 at 5:45 PM 张铎(Duo Zhang) wrote: > If we can speed up the 3.x release, since it has already moved up the > JDK17+, it will be easier to land these changes in 3.x only. > > Istvan Toth 于2025年3月7日周五 03:42写道： > > > > I'd split that into two tickets. > > > > First move to Jetty 12 while keeping the old javax package names, and > once > > that's done and tested, we can look into moving to jakarta. > > > > Istvan > > > > On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell > wrote: > > > > > > Therefore, I plan to use the EE8 environment on Jetty 12. > > > > > > Ah, thanks for mentioning EE. > > > > > > Do you have a requirement to stay with EE 8 ? > > > > > > If we could target EE 9 instead of EE 8 that would align with some > internal > > > requirements at $dayjob. For what it's worth. Mostly the point of EE 9 > is a > > > move of the EE stuff into a new namespace, allowing for further > development > > > under the Eclipse Foundation without trademark conflicts with Oracle's > > > "Java" branding. The trademark issue and follow on Java licensing > concerns > > > are going to be relevant for anyone who might be visited by an Oracle > > > lawyer-shark. > > > > > > On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain > wrote: > > > > > > > > I think that the best course of action would be adding a separate > > > Jetty11 > > > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > > > used/updated by the 2.x / 3.x branches. > > > > > > > > +1, I would like to volunteer for this work. > > > > > > > > Maintaining Jetty 9.4.x could be quite challenging from a CVE > perspective > > > > since it has been EOCS (End of Community Support) since June 2022, > and > > > new > > > > releases may not be forthcoming for a long time. I faced similar > issues > > > > during our last hbase-thirdparty release. See this issue comment: > > > > > > > > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > > > > > > > > Regarding the version and CVEs, I agree with @Andrew and suggest > that we > > > > jump directly to Jetty 12, bypassing Jetty 11, to support > javax.servlet > > > for > > > > JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty 12. > See > > > > Jetty version history: > https://jetty.org/download.html#version-history > > > > > > > > @Istvan, is there any particular reason you recommend moving to > Jetty 11, > > > > which is also EOCS? > > > > > > > > If others are fine with me taking this up, I can create the necessary > > > > JIRAs for the Jetty migration project and start the work soon. > > > > > > > > Regards, > > > > Nihal > > > > > > > > On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote: > > > > > FYI I think Hadoop will need to make similar moves too. > > > > > Sounds like Hadoop should start the work to drop JDK8 as well. I > know > > > > Steve > > > > > has a patch that requires JDK17+ due to Iceberg > > > > > https://github.com/apache/hadoop/pull/7316 > > > > > > > > > > -- Forwarded message -
Re: Fwd: [DISCUSS] Plans for JDK22 / SecurityManager removal
I'd split that into two tickets. First move to Jetty 12 while keeping the old javax package names, and once that's done and tested, we can look into moving to jakarta. Istvan On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell wrote: > > Therefore, I plan to use the EE8 environment on Jetty 12. > > Ah, thanks for mentioning EE. > > Do you have a requirement to stay with EE 8 ? > > If we could target EE 9 instead of EE 8 that would align with some internal > requirements at $dayjob. For what it's worth. Mostly the point of EE 9 is a > move of the EE stuff into a new namespace, allowing for further development > under the Eclipse Foundation without trademark conflicts with Oracle's > "Java" branding. The trademark issue and follow on Java licensing concerns > are going to be relevant for anyone who might be visited by an Oracle > lawyer-shark. > > On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain wrote: > > > > I think that the best course of action would be adding a separate > Jetty11 > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > used/updated by the 2.x / 3.x branches. > > > > +1, I would like to volunteer for this work. > > > > Maintaining Jetty 9.4.x could be quite challenging from a CVE perspective > > since it has been EOCS (End of Community Support) since June 2022, and > new > > releases may not be forthcoming for a long time. I faced similar issues > > during our last hbase-thirdparty release. See this issue comment: > > > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > > > > Regarding the version and CVEs, I agree with @Andrew and suggest that we > > jump directly to Jetty 12, bypassing Jetty 11, to support javax.servlet > for > > JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty 12. See > > Jetty version history: https://jetty.org/download.html#version-history > > > > @Istvan, is there any particular reason you recommend moving to Jetty 11, > > which is also EOCS? > > > > If others are fine with me taking this up, I can create the necessary > > JIRAs for the Jetty migration project and start the work soon. > > > > Regards, > > Nihal > > > > On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote: > > > FYI I think Hadoop will need to make similar moves too. > > > Sounds like Hadoop should start the work to drop JDK8 as well. I know > > Steve > > > has a patch that requires JDK17+ due to Iceberg > > > https://github.com/apache/hadoop/pull/7316 > > > > > > -- Forwarded message - > > > From: Istvan Toth > > > Date: Sun, Mar 2, 2025 at 9:41 PM > > > Subject: [DISCUSS] Plans for JDK22 / SecurityManager removal > > > To: HBase Dev List > > > > > > > > > Hi! > > > > > > The last big incompatible JDK change is the JEP411/JEP486 > SecurityManager > > > removal process, which has gotten serious with JDK22, which disables > > > Subject.doAs*() Subject.getSubject() by default (while providing the > kind > > > of equivalent callAs() and current() methods), and kills automatic > > Subject > > > propagation to new threads. > > > > > > To add insult to injury the new methods are only available from JDK18, > so > > > it is not possible to just move to the new API without breaking JDK17. > > > > > > Some of this can only be solved in Hadoop (hopefully 3.5) , but HBase > can > > > also take steps to move towards JDK22 compatibility in the meantime. > > > > > > Here's the plan I have in mind: > > > > > > - Review and the dependencies WRT SecurityManage usage (just search for > > API > > > calls) > > > - Update the dependencies as needed > > > > > > - Add a reflection-based compatibility Utility class based on Jetty's > > > SecurityUtils, (or one of its derivatives) > > > - Replace the deprecated method calls with the compatibility versions > > > - Wrap Runnables/Callables with Subject current()/callAs() methods. > > > > > > One large dependency I have identified is Jetty. > > > While technically we MAY be able to keep using 9.4 by overriding its > > > default ThreadFactory, I think that considering that even semi-formal > > Jetty > > > 9.4 support is on its very last legs (the latest update was never even > > > announced and is not listed in the downloads page), the advantages of > > > updating to Jetty 11 on branch-3+ now far outweighs the added > maintenance >
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
Thanks Nihal, and thanks a lot for volunteering. I don't recommend Jetty 11, It was a mistake in my original mail, which I corrected in a response. (BTW I wasn't even aware that 11 does not support the old API) The only risk with 12 is that while it is supported today, we have no commitment on the support timeline for 12. 10 and 11 are also discontinued, which left 9.4 as a quasi-official semi-supported LTS release. There is some risk that at some point Jetty 12 will be hard EOLed and the successor will not support JDK17, while 9.4.x would still receive ad-hoc security fixes . I guess that it all depends on whether there would be sponsors for continued security releases, we just cannot tell, even the Jetty people may not have an answer now. I still think that moving to 12 is the right thing to do, just pointing out that at some point we're likely to run into Jetty problems again, as our release and JDK support strategy and Jetty's do not match up. If we get into that situation, we could also patch our thirdparty Jetty for CVEs, but that's an additional burden, and that would also mean getting into a losing fight with dumb static scanners and their users, or some interested party could sponsor security releases. Istvan On Tue, Mar 4, 2025 at 7:28 PM Nihal Jain wrote: > > I think that the best course of action would be adding a separate Jetty11 > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > used/updated by the 2.x / 3.x branches. > > +1, I would like to volunteer for this work. > > Sticking to Jetty 9.4.x could be quite challenging from a CVE perspective > since it has been EOCS (End of Community Support) since June 2022, and new > releases may not be forthcoming for a long time. I faced similar issues > during our last hbase-thirdparty release. See this issue comment > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 > > I agree with Andrew and suggest that we jump directly to Jetty 12, > bypassing Jetty 11, to support javax.servlet for JSP 2.3.1. Therefore, I > suggest we use the EE8 environment on Jetty 12. See > https://jetty.org/download.html#version-history > > Istvan, is there any particular reason you recommend moving to Jetty 11, > which is also EOCS? > > If others are fine with me taking this up, I can create the necessary > JIRAs for the Jetty migration project and start the work soon. > > Regards, > Nihal > > > On Tue, 4 Mar 2025 at 2:27 AM, Istvan Toth wrote: > >> Thanks for your reply, Andrew. >> >> If no-one else picks it up, I will try to start working on moving to Jetty >> 12 in a few weeks. >> >> On the importance of recent Java support: (which doesn't change the fact >> that we're at the mercy of Hadoop releases) >> >> I also used to think that it's not important to support the latest Java >> versions, but recent discussions with the Trino team have changed my mind. >> My new understanding is that there are an increasing number of users and >> projects who are aggressively moving to the latest Java versions, both >> applications and libraries (for example Trino), >> and if we don't keep up, then HBase/Phoenix becomes a non-starter by >> simply >> not running in their JVMs. >> >> Yes, probably 80% of the current users and 90+% percent of nodes are >> enterprises and other very risk-averse entities that wouldn't even >> consider a non-LTS java version, but by lagging with Java support, we are >> driving off new adapters that are vital for the long-term health of the >> project. >> >> By keeping up with non-LTS versions, we also get a head-start, and are >> much >> more likely to support new LTS releases on day one. >> >> Istvan >> >> On Mon, Mar 3, 2025 at 6:07 PM Andrew Purtell >> wrote: >> >> > I don't think compatibility with the very most recent Java version is a >> > primary requirement for this project. Identifying the issues and >> proposing >> > a plan as has been done here is great. Beyond that, if someone has the >> > burning need to move to Java 22 or later we should here about it then by >> > user filed JIRAs or PRs or mailing list posts asking about breaks >> caused by >> > SecurityManager removal. Should Hadoop take steps we could also leverage >> > that, after the change lands in a released version we can consume. >> > >> > The dependency on the old version of Jetty that seems about to fall out >> of >> > maintenance seems like a more immediate issue, given that Jetty is a >> > frequent flyer for security issues and has a number of (solved) CVEs >> posted >> > over the past few years. >> > >> > > I think that the best course of action would be adding a separate >> Jetty >> > 12 >> > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both >> > > used/updated by the 2.x / 3.x branches. >> > >> > +1 >> > Presumably JIRAs/PRs to move up to Jetty 12 would follow. I think we can >> > find helpers for a Jetty 12 uplift at the $dayjob here also. >> > >> > >> > On Mon, Mar 3, 2025 at 5:50 AM Istvan Toth wro
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
> I think that the best course of action would be adding a separate Jetty11 > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > used/updated by the 2.x / 3.x branches. +1, I would like to volunteer for this work. Sticking to Jetty 9.4.x could be quite challenging from a CVE perspective since it has been EOCS (End of Community Support) since June 2022, and new releases may not be forthcoming for a long time. I faced similar issues during our last hbase-thirdparty release. See this issue comment https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092 I agree with Andrew and suggest that we jump directly to Jetty 12, bypassing Jetty 11, to support javax.servlet for JSP 2.3.1. Therefore, I suggest we use the EE8 environment on Jetty 12. See https://jetty.org/download.html#version-history Istvan, is there any particular reason you recommend moving to Jetty 11, which is also EOCS? If others are fine with me taking this up, I can create the necessary JIRAs for the Jetty migration project and start the work soon. Regards, Nihal On Tue, 4 Mar 2025 at 2:27 AM, Istvan Toth wrote: > Thanks for your reply, Andrew. > > If no-one else picks it up, I will try to start working on moving to Jetty > 12 in a few weeks. > > On the importance of recent Java support: (which doesn't change the fact > that we're at the mercy of Hadoop releases) > > I also used to think that it's not important to support the latest Java > versions, but recent discussions with the Trino team have changed my mind. > My new understanding is that there are an increasing number of users and > projects who are aggressively moving to the latest Java versions, both > applications and libraries (for example Trino), > and if we don't keep up, then HBase/Phoenix becomes a non-starter by simply > not running in their JVMs. > > Yes, probably 80% of the current users and 90+% percent of nodes are > enterprises and other very risk-averse entities that wouldn't even > consider a non-LTS java version, but by lagging with Java support, we are > driving off new adapters that are vital for the long-term health of the > project. > > By keeping up with non-LTS versions, we also get a head-start, and are much > more likely to support new LTS releases on day one. > > Istvan > > On Mon, Mar 3, 2025 at 6:07 PM Andrew Purtell wrote: > > > I don't think compatibility with the very most recent Java version is a > > primary requirement for this project. Identifying the issues and > proposing > > a plan as has been done here is great. Beyond that, if someone has the > > burning need to move to Java 22 or later we should here about it then by > > user filed JIRAs or PRs or mailing list posts asking about breaks caused > by > > SecurityManager removal. Should Hadoop take steps we could also leverage > > that, after the change lands in a released version we can consume. > > > > The dependency on the old version of Jetty that seems about to fall out > of > > maintenance seems like a more immediate issue, given that Jetty is a > > frequent flyer for security issues and has a number of (solved) CVEs > posted > > over the past few years. > > > > > I think that the best course of action would be adding a separate Jetty > > 12 > > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > > used/updated by the 2.x / 3.x branches. > > > > +1 > > Presumably JIRAs/PRs to move up to Jetty 12 would follow. I think we can > > find helpers for a Jetty 12 uplift at the $dayjob here also. > > > > > > On Mon, Mar 3, 2025 at 5:50 AM Istvan Toth wrote: > > > >> Correction: I meant Jetty 12 > >> > >> On Mon, Mar 3, 2025 at 6:41 AM Istvan Toth wrote: > >> > >> > Hi! > >> > > >> > The last big incompatible JDK change is the JEP411/JEP486 > >> SecurityManager > >> > removal process, which has gotten serious with JDK22, which disables > >> > Subject.doAs*() Subject.getSubject() by default (while providing the > >> kind > >> > of equivalent callAs() and current() methods), and kills automatic > >> Subject > >> > propagation to new threads. > >> > > >> > To add insult to injury the new methods are only available from JDK18, > >> so > >> > it is not possible to just move to the new API without breaking JDK17. > >> > > >> > Some of this can only be solved in Hadoop (hopefully 3.5) , but HBase > >> can > >> > also take steps to move towards JDK22 compatibility in the meantime. > >> > > >> > Here's the plan I have in mind: > >> > > >> > - Review and the dependencies WRT SecurityManage usage (just search > for > >> > API calls) > >> > - Update the dependencies as needed > >> > > >> > - Add a reflection-based compatibility Utility class based on Jetty's > >> > SecurityUtils, (or one of its derivatives) > >> > - Replace the deprecated method calls with the compatibility versions > >> > - Wrap Runnables/Callables with Subject current()/callAs() methods. > >> > > >> > One large dependency I have identified is Jetty. > >> > While technically we MAY be a
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
Thanks for your reply, Andrew. If no-one else picks it up, I will try to start working on moving to Jetty 12 in a few weeks. On the importance of recent Java support: (which doesn't change the fact that we're at the mercy of Hadoop releases) I also used to think that it's not important to support the latest Java versions, but recent discussions with the Trino team have changed my mind. My new understanding is that there are an increasing number of users and projects who are aggressively moving to the latest Java versions, both applications and libraries (for example Trino), and if we don't keep up, then HBase/Phoenix becomes a non-starter by simply not running in their JVMs. Yes, probably 80% of the current users and 90+% percent of nodes are enterprises and other very risk-averse entities that wouldn't even consider a non-LTS java version, but by lagging with Java support, we are driving off new adapters that are vital for the long-term health of the project. By keeping up with non-LTS versions, we also get a head-start, and are much more likely to support new LTS releases on day one. Istvan On Mon, Mar 3, 2025 at 6:07 PM Andrew Purtell wrote: > I don't think compatibility with the very most recent Java version is a > primary requirement for this project. Identifying the issues and proposing > a plan as has been done here is great. Beyond that, if someone has the > burning need to move to Java 22 or later we should here about it then by > user filed JIRAs or PRs or mailing list posts asking about breaks caused by > SecurityManager removal. Should Hadoop take steps we could also leverage > that, after the change lands in a released version we can consume. > > The dependency on the old version of Jetty that seems about to fall out of > maintenance seems like a more immediate issue, given that Jetty is a > frequent flyer for security issues and has a number of (solved) CVEs posted > over the past few years. > > > I think that the best course of action would be adding a separate Jetty > 12 > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > used/updated by the 2.x / 3.x branches. > > +1 > Presumably JIRAs/PRs to move up to Jetty 12 would follow. I think we can > find helpers for a Jetty 12 uplift at the $dayjob here also. > > > On Mon, Mar 3, 2025 at 5:50 AM Istvan Toth wrote: > >> Correction: I meant Jetty 12 >> >> On Mon, Mar 3, 2025 at 6:41 AM Istvan Toth wrote: >> >> > Hi! >> > >> > The last big incompatible JDK change is the JEP411/JEP486 >> SecurityManager >> > removal process, which has gotten serious with JDK22, which disables >> > Subject.doAs*() Subject.getSubject() by default (while providing the >> kind >> > of equivalent callAs() and current() methods), and kills automatic >> Subject >> > propagation to new threads. >> > >> > To add insult to injury the new methods are only available from JDK18, >> so >> > it is not possible to just move to the new API without breaking JDK17. >> > >> > Some of this can only be solved in Hadoop (hopefully 3.5) , but HBase >> can >> > also take steps to move towards JDK22 compatibility in the meantime. >> > >> > Here's the plan I have in mind: >> > >> > - Review and the dependencies WRT SecurityManage usage (just search for >> > API calls) >> > - Update the dependencies as needed >> > >> > - Add a reflection-based compatibility Utility class based on Jetty's >> > SecurityUtils, (or one of its derivatives) >> > - Replace the deprecated method calls with the compatibility versions >> > - Wrap Runnables/Callables with Subject current()/callAs() methods. >> > >> > One large dependency I have identified is Jetty. >> > While technically we MAY be able to keep using 9.4 by overriding its >> > default ThreadFactory, I think that considering that even semi-formal >> Jetty >> > 9.4 support is on its very last legs (the latest update was never even >> > announced and is not listed in the downloads page), the advantages of >> > updating to Jetty 11 on branch-3+ now far outweighs the added >> maintenance >> > burden of the branch divergence. >> > >> > I think that the best course of action would be adding a separate >> Jetty11 >> > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both >> > used/updated by the 2.x / 3.x branches. >> > >> > What do you think ? >> > >> > - Do you agree that JEP411/486 should be treated as a priority ? >> > - Would you volunteer for the dependency audit ? >> > - Do you agree with the Jetty update, and the plan outlined above ? >> > >> > Istvan >> > >> > ps: Sorry, no links because GMail seems to react very bady to any. >> Please >> > use search. >> > >> > >> > > > -- > Best regards, > Andrew > > Unrest, ignorance distilled, nihilistic imbeciles - > It's what we’ve earned > Welcome, apocalypse, what’s taken you so long? > Bring us the fitting end that we’ve been counting on >- A23, Welcome, Apocalypse >
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
I don't think compatibility with the very most recent Java version is a primary requirement for this project. Identifying the issues and proposing a plan as has been done here is great. Beyond that, if someone has the burning need to move to Java 22 or later we should here about it then by user filed JIRAs or PRs or mailing list posts asking about breaks caused by SecurityManager removal. Should Hadoop take steps we could also leverage that, after the change lands in a released version we can consume. The dependency on the old version of Jetty that seems about to fall out of maintenance seems like a more immediate issue, given that Jetty is a frequent flyer for security issues and has a number of (solved) CVEs posted over the past few years. > I think that the best course of action would be adding a separate Jetty 12 > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > used/updated by the 2.x / 3.x branches. +1 Presumably JIRAs/PRs to move up to Jetty 12 would follow. I think we can find helpers for a Jetty 12 uplift at the $dayjob here also. On Mon, Mar 3, 2025 at 5:50 AM Istvan Toth wrote: > Correction: I meant Jetty 12 > > On Mon, Mar 3, 2025 at 6:41 AM Istvan Toth wrote: > > > Hi! > > > > The last big incompatible JDK change is the JEP411/JEP486 SecurityManager > > removal process, which has gotten serious with JDK22, which disables > > Subject.doAs*() Subject.getSubject() by default (while providing the kind > > of equivalent callAs() and current() methods), and kills automatic > Subject > > propagation to new threads. > > > > To add insult to injury the new methods are only available from JDK18, so > > it is not possible to just move to the new API without breaking JDK17. > > > > Some of this can only be solved in Hadoop (hopefully 3.5) , but HBase can > > also take steps to move towards JDK22 compatibility in the meantime. > > > > Here's the plan I have in mind: > > > > - Review and the dependencies WRT SecurityManage usage (just search for > > API calls) > > - Update the dependencies as needed > > > > - Add a reflection-based compatibility Utility class based on Jetty's > > SecurityUtils, (or one of its derivatives) > > - Replace the deprecated method calls with the compatibility versions > > - Wrap Runnables/Callables with Subject current()/callAs() methods. > > > > One large dependency I have identified is Jetty. > > While technically we MAY be able to keep using 9.4 by overriding its > > default ThreadFactory, I think that considering that even semi-formal > Jetty > > 9.4 support is on its very last legs (the latest update was never even > > announced and is not listed in the downloads page), the advantages of > > updating to Jetty 11 on branch-3+ now far outweighs the added maintenance > > burden of the branch divergence. > > > > I think that the best course of action would be adding a separate Jetty11 > > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > > used/updated by the 2.x / 3.x branches. > > > > What do you think ? > > > > - Do you agree that JEP411/486 should be treated as a priority ? > > - Would you volunteer for the dependency audit ? > > - Do you agree with the Jetty update, and the plan outlined above ? > > > > Istvan > > > > ps: Sorry, no links because GMail seems to react very bady to any. Please > > use search. > > > > > -- Best regards, Andrew Unrest, ignorance distilled, nihilistic imbeciles - It's what we’ve earned Welcome, apocalypse, what’s taken you so long? Bring us the fitting end that we’ve been counting on - A23, Welcome, Apocalypse
Re: [DISCUSS] Plans for JDK22 / SecurityManager removal
Correction: I meant Jetty 12 On Mon, Mar 3, 2025 at 6:41 AM Istvan Toth wrote: > Hi! > > The last big incompatible JDK change is the JEP411/JEP486 SecurityManager > removal process, which has gotten serious with JDK22, which disables > Subject.doAs*() Subject.getSubject() by default (while providing the kind > of equivalent callAs() and current() methods), and kills automatic Subject > propagation to new threads. > > To add insult to injury the new methods are only available from JDK18, so > it is not possible to just move to the new API without breaking JDK17. > > Some of this can only be solved in Hadoop (hopefully 3.5) , but HBase can > also take steps to move towards JDK22 compatibility in the meantime. > > Here's the plan I have in mind: > > - Review and the dependencies WRT SecurityManage usage (just search for > API calls) > - Update the dependencies as needed > > - Add a reflection-based compatibility Utility class based on Jetty's > SecurityUtils, (or one of its derivatives) > - Replace the deprecated method calls with the compatibility versions > - Wrap Runnables/Callables with Subject current()/callAs() methods. > > One large dependency I have identified is Jetty. > While technically we MAY be able to keep using 9.4 by overriding its > default ThreadFactory, I think that considering that even semi-formal Jetty > 9.4 support is on its very last legs (the latest update was never even > announced and is not listed in the downloads page), the advantages of > updating to Jetty 11 on branch-3+ now far outweighs the added maintenance > burden of the branch divergence. > > I think that the best course of action would be adding a separate Jetty11 > module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both > used/updated by the 2.x / 3.x branches. > > What do you think ? > > - Do you agree that JEP411/486 should be treated as a priority ? > - Would you volunteer for the dependency audit ? > - Do you agree with the Jetty update, and the plan outlined above ? > > Istvan > > ps: Sorry, no links because GMail seems to react very bady to any. Please > use search. > >
[DISCUSS] Plans for JDK22 / SecurityManager removal
Hi! The last big incompatible JDK change is the JEP411/JEP486 SecurityManager removal process, which has gotten serious with JDK22, which disables Subject.doAs*() Subject.getSubject() by default (while providing the kind of equivalent callAs() and current() methods), and kills automatic Subject propagation to new threads. To add insult to injury the new methods are only available from JDK18, so it is not possible to just move to the new API without breaking JDK17. Some of this can only be solved in Hadoop (hopefully 3.5) , but HBase can also take steps to move towards JDK22 compatibility in the meantime. Here's the plan I have in mind: - Review and the dependencies WRT SecurityManage usage (just search for API calls) - Update the dependencies as needed - Add a reflection-based compatibility Utility class based on Jetty's SecurityUtils, (or one of its derivatives) - Replace the deprecated method calls with the compatibility versions - Wrap Runnables/Callables with Subject current()/callAs() methods. One large dependency I have identified is Jetty. While technically we MAY be able to keep using 9.4 by overriding its default ThreadFactory, I think that considering that even semi-formal Jetty 9.4 support is on its very last legs (the latest update was never even announced and is not listed in the downloads page), the advantages of updating to Jetty 11 on branch-3+ now far outweighs the added maintenance burden of the branch divergence. I think that the best course of action would be adding a separate Jetty11 module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both used/updated by the 2.x / 3.x branches. What do you think ? - Do you agree that JEP411/486 should be treated as a priority ? - Would you volunteer for the dependency audit ? - Do you agree with the Jetty update, and the plan outlined above ? Istvan ps: Sorry, no links because GMail seems to react very bady to any. Please use search.
