Hi, Great discussion, I agree with all that you have said. And that we will have to deal with the intricacies of the MR-JAR regardless of the outcome here, which is doable.
I would very much like to avoid supporting Java 17 (released in Sep 2021) in 2025. So far we have two possible approaches: 1. Release Lucene 10.0.0 now with Java 17 minimum. Bump _main_ to Java 21. 2. Release Lucene 9.9.0 soon(ish) with Java 11 minimum. Bump _main_ to Java 21, and release 10.0.0 in the first quarter of 2024. Have I captured this correctly? Are there other alternatives that should be considered? My issue with no.1 is that the 10.x train will likely live on for ~2yrs? In which case we’ll be supporting Java 17 until some time in late 2025, when Java 25 is released. This could be mitigated by releasing Lucene 11.0.0 earlier than 2yrs, say 1yr after 10.0.0. The only concern I have with no.2 is that it could be considered an “aggressive” adoption of Java 21 - adoption sooner than the ecosystem can handle, e.g. are environments in which Lucene is deployed, and their transitive dependencies, ready to run on Java 21? By the time we’re ready to release 10.0.0, say March 2023, then I expect no issue with this. I’m only putting dates on things to make the conversation more concrete, clearly these are all just illustrative. -Chris. P.S. This thread is starting to touch on the general release cadence of Lucene, which is a bigger topic than the subject line of this mail, but I think is great to think about these bumps in this context. > On 5 Nov 2023, at 09:46, Uwe Schindler <u...@thetaphi.de> wrote: > > Hi, > > thanks Chris. This is why I suggested the idea, to have the discussion here. > We are already close to Lucene 9.9. Do we want 9.10? We had that long series > of minor releases only int the 4.x branch (which ended in 4.10). > > I have some comments inline: > > >>> On 3 Nov 2023, at 13:11, Uwe Schindler <u...@thetaphi.de> >>> <mailto:u...@thetaphi.de> wrote: >>> >>> Hi, >>> >>> I had another idea: Why not release main as 10.0.0 *NOW* and create >>> branch_10x (with Java 17) minimum, stop working on 9.x, and move main >>> branch to 21? >> I see now that 9.x has a minimum Java version of 11, and that _main_ has a >> minimum version of 17. I previously overlooked this ( I thought that 9.x was >> on 17, but it is not ). Ok, so your idea is actually quite inline with how >> things have happened in the past. >> >> For ease of reference, here are the dates of the last 4 major releases. >> 9.0.0 Dec 2021 >> 8.0.0 Mar 2019 >> 7.0.0 Sep 2017 >> 6.0.0 Apr 2016 >> >> If we release 10.0.0 now (with a minimum of 17) that drops the need to >> support Java 11 (since work in 9.x will mostly stop). I’m ok with this, and >> we get the benefits of dropping < Java 17. But can we be more ambitious in >> our approach here? >> >> I’ll defer to others about what is in _main_ to justify a major release or >> not - the driver for a release should be more than just the minimum Java >> version. >> >> Alternatively, what if we were to not release 10.0.0 for another while, say >> 3 - 6 months, and at the same time bump it to Java 21. In the meantime we >> can keep the 9.x updates coming. My motivation for suggesting this is that >> it appears that major Lucene versions seem to be around every 2 years or so, >> and if we release 10 with Java 17, the we’ll still be reluctant to use Java >> APIs and features between 17 and 21 for the next, likely, 2 years. An >> alternative to that is to release Lucene 11.0.0 sometime before the 2 year >> mark. >> >>> I would be happy to remove the MmapByteBuffer directory in Java 18. >> We can only do this when we move to a minimum Java > 17, so in your proposal >> that would be in _main_ some time post the fork for branch_10x. That seems >> ok. > Sorry this was a typo with version number. I meant Java 21 would no longer > require (Mapped-)ByteBufferIndexInput. >>> Unfortunately in Java 21 we still need a hack top compile the MemorySegment >>> classes because of the preview flag. And for the incubator we also need the >>> APIJAR files. But we can do this then without MR-JAR unless we need a new >>> version for Java 22, 23 of vectors. My idea would be to patch in the api >>> JAR during compile of "main" sourceset classes. >> Yeah, regardless of the minimum version bump some work is needed here :-( >> Where possible we should try to minimise it, but I agree we’ll likely need >> updates for the vector stuff in 22+. > I figured out it is not so easy, we need additional maintenance and possibly > a MR JAR also with Java 21: > > In Java 21, panama-foreign is still preview. So when compiling we need the > APIJAR. > In the MR-JAR compilation we patch the APIJAR into the java.base module > (which we also need for incubating). The problem is: YOu cannot patch the > "java.base" module and at same time pass "--release 21". So In that code part > we need to compile against actual class library (I have no idea why patching > is disallowed with --release). It prints a cryptic error message, but makes > no sense to me. > Because of the inability to use "--release" we still need to compile the > Panama classes in a separate gradle sourceSet. But we can copy the separate > sourceSet output for 21 directly into the main JAR part (but we can also let > it live in versions/21. > > This should not stop us from moving to 21, the details with how to build the > JRA/MR-JAR can be solved separately. You PR looks fine, I would keep away > from the MR-JAR sourceSets for now. We can clean the up later. > > Keeping parts of the MR-JAR logic as suggested before helps with backporting. > > Uwe > >>> Am 03.11.2023 um 13:20 schrieb Chris Hegarty: >>>> Hi, >>>> >>>> I would like to start the discussion and gather feedback on bumping the >>>> minimum Java version requirement to 21. >>>> >>>> I have no particular timeline in mind, but these kinda bumps often >>>> require dependency updates [*], small code refactorings, etc, and can >>>> take some time to plan and execute. It's best to at least have a plan >>>> for when, rather than if! Any bump would of course be limited to the >>>> _main_ branch, and therefore targeting a major Lucene release (no >>>> changes to branches targeting minor patch releases). >>>> >>>> I'm sure subscribers to this list are already familiar with the various >>>> goodies that have been added between Java 17 and 21, so I'll not >>>> enumerate them here, but rather callout just two particular benefits >>>> that I think are significant to the Lucene project. >>>> >>>> 1) Put a lower bound on the number of memory segment mmap and Panama >>>> Vector similarity implementations that we need to carry. This not only >>>> reduces maintenance cost, but avoids additional consideration and >>>> experimentation for performance improvements. >>>> >>>> 2) Support for half float, Float::float16ToFloat and Float::floatToFloat16, >>>> which will likely be beneficial in several places. >>>> >>>> More concretely, and somewhat orthogonal to the discussion of when, I >>>> would like to create a meta-issue capturing the prerequisites to a >>>> version bump. >>>> >>>> Your thoughts, comments, and feedback are very much welcome. >>>> >>>> -Chris. >>>> >>>> [*] we need at least an ECJ JDT dependency update, that supports >>>> Java 21, https://www.eclipse.org/lists/eclipse-dev/msg12203.html >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>> <mailto:dev-unsubscr...@lucene.apache.org> >>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>> <mailto:dev-h...@lucene.apache.org> >>>> >>> -- >>> Uwe Schindler >>> Achterdiek 19, D-28357 Bremen >>> https://www.thetaphi.de <https://www.thetaphi.de/> >>> eMail: u...@thetaphi.de <mailto:u...@thetaphi.de> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> <mailto:dev-unsubscr...@lucene.apache.org> >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> <mailto:dev-h...@lucene.apache.org> >>> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> <mailto:dev-unsubscr...@lucene.apache.org> >> For additional commands, e-mail: dev-h...@lucene.apache.org >> <mailto:dev-h...@lucene.apache.org> >> > -- > Uwe Schindler > Achterdiek 19, D-28357 Bremen > https://www.thetaphi.de <https://www.thetaphi.de/> > eMail: u...@thetaphi.de <mailto:u...@thetaphi.de>
