Re: New branch and feature freeze for Lucene/Solr 8.10.0

[Timothy Potter]

Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
headed out on vacation tomorrow, back at keys on Monday, Sept 20
unless someone else wants to pick up the RM duties before then?

After failing the test suite at various places and other weirdness
like .asc files not getting created, I finally got to the smoke test
part, which is now failing with:

  File 
"/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
line 176, in checkJARMetaData
raise RuntimeError('%s is missing "%s" inside its
META-INF/MANIFEST.MF (wrong git revision?)' % \
RuntimeError: JAR file
"/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
is missing "Implementation-Version: 8.10.0
ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
META-INF/MANIFEST.MF (wrong git revision?)

FWIW, I verified that the other Lucene JAR files have this line in
them, such as core:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.9.15
Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
Extension-Name: org.apache.lucene
Specification-Title: Lucene Search Engine: core
Specification-Version: 8.10.0
Specification-Vendor: The Apache Software Foundation
Implementation-Title: org.apache.lucene
Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
 7 - tjp - 2021-09-14 19:08:42
Implementation-Vendor: The Apache Software Foundation
X-Compile-Source-JDK: 8
X-Compile-Target-JDK: 8
Multi-Release: true

On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
 wrote:
>
> All the best, this is the worst step.
>
> On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter,  wrote:
>>
>> Building RC1 now ... stay tuned.
>>
>> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter  wrote:
>> >
>> > Thanks for the update Mike!
>> >
>> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
>> > SOLR-15621, which looks like an easy win for the issue Cassandra
>> > reported on Slack earlier today.
>> >
>> > Cheers,
>> > Tim
>> >
>> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob  wrote:
>> > >
>> > > Hi Tim, I'm still working on SOLR-1, the code and benchmarking
>> > > both look pretty good, but I've got a few last unit tests that I need
>> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
>> > > sure to keep you updated though.
>> > >
>> > >
>> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter  
>> > > wrote:
>> > > >
>> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
>> > > > the schema designer. I haven't built the RC yet, so going to see if I
>> > > > can get this in today.
>> > > >
>> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter  
>> > > > wrote:
>> > > > >
>> > > > > NOTICE:
>> > > > >
>> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on 
>> > > > > stable branch.
>> > > > >
>> > > > > Please observe the normal rules:
>> > > > >
>> > > > > * No new features may be committed to the branch.
>> > > > >
>> > > > > * Documentation patches, build patches and serious bug fixes may be
>> > > > >   committed to the branch. However, you should submit all patches you
>> > > > >   want to commit to Jira first to give others the chance to review
>> > > > >   and possibly vote against the patch. Keep in mind that it is our
>> > > > >   main intention to keep the branch as stable as possible.
>> > > > >
>> > > > > * All patches that are intended for the branch should first be 
>> > > > > committed
>> > > > >   to the unstable branch, merged into the stable branch, and then 
>> > > > > into
>> > > > >   the current release branch.
>> > > > >
>> > > > > * Normal unstable and stable branch development may continue as 
>> > > > > usual.
>> > > > >   However, if you plan to commit a big change to the unstable branch
>> > > > >   while the branch feature freeze is in effect, think twice: can't 
>> > > > > the
>> > > > >   addition wait a couple more days? Merges of bug fixes into the 
>> > > > > branch
>> > > > >   may become more difficult.
>> > > > >
>> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will 
>> > > > > delay
>> > > > >   a release candidate build.
>> > > > > 
>> > > >
>> > > > -
>> > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > > > For additional commands, e-mail: dev-h...@lucene.apache.org
>> > > >
>> > >
>> > > -
>> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > > For additional commands, e-mail: dev-h...@lucene.apache.org
>> > >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> For additional commands, e-mail: dev-h...@solr.apache.org
>>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional 

Re: Question - LUCENE-9077

[Dawid Weiss]

> I see on the issue it mentions "See notes below on why this respin is
> needed."
>

I can't remember what I meant by that, to be honest...


> I'm not able to find the notes mentioned that talk about the reasons for
> the changes which would be nice to be able to look over... but I might just
> be looking in the wrong place, so if someone could point me to them that
> would be awesome!
>
> Also since this is a significant change, is anyone aware of some article,
> blog post, or internal discussion that talks about what was gained out of
> the switch over to Gradle (performance, ease of development, etc., or
> perhaps this would all be in the notes mentioned above).


The big reason was that the ant built had become slow and a nightmare to
maintain (macros, external tools, dependencies). The entire toolchain had
years of monkey patching all over the place. Maven would be another option
but Lucene does tricky stuff during its build and Maven doesn't yield
easily to customizations (in my opinion).

I think the best way to get a feeling for the reasons why the build had to
be modernized is if you compare the existing branch_8x and main -- look at
how ant build files are written and organized and try to build a few things
here and there... Then try to do the same things on main and you'll see the
difference, hopefully for the better.

Dawid

P.S. I happen to like ant very much - this has nothing to do with the tool
itself. Still, working with manual dependencies and writing complex stuff
in seems to be a thing of the past.

Re: Java 11/17 Version Matrix

[Dawid Weiss]

Let's wait for this functionality and see what happens. If the gain is
significant then this provides an incentive to upgrade for everyone.
MR-JARs will be a pain to keep consistent...

Dawid

On Tue, Sep 14, 2021 at 10:19 AM Adrien Grand  wrote:

> I think we should discuss options when Project Panama is released. Doing
> frequent major releases forces users to reindex more often. If Project
> Panama was released shortly and we decided to release Lucene 10
> immediately, this would force users to reindex their 8.x data to be able to
> upgrade, I know of many Elasticsearch users for whom it would cause
> headaches and I suspect that it's no different for Solr users and many
> direct users of Lucene. I'd rather look into publishing a MR JAR of bumping
> the minimum required Java version in a minor release than releasing Lucene
> 10.0 less than 1 year after 9.0.
>
> On Mon, Sep 13, 2021 at 9:19 PM Uwe Schindler  wrote:
>
>> In addition: we test for compatibility with Java 17 (both Lucene and
>> Solr), so consumer is still able to use any version and has enough
>> flexibility.
>>
>> Uwe
>>
>> Am 13. September 2021 18:52:53 UTC schrieb Dawid Weiss <
>> dawid.we...@gmail.com>:
>>>
>>>
>>> I agree with Uwe and Robert. JDK11, then the min bar should move if
>>> there is something that brings value (be it performance, LTS or some other
>>> attractive option).
>>>
>>> Dawid
>>>
>>> On Mon, Sep 13, 2021 at 8:15 PM Robert Muir  wrote:
>>>
 +1, I think the main thing to watch out for is project panama. If we
 get a 18.x/19.x release with non-incubating APIs, I think it makes
 sense to create a new major Lucene version. Even if it isn't an
 OpenJDK LTS release. It could really change a lot, especially
 regarding hotspots in the code such as postings/dv compression. So it
 would be great to allow a lot of folks to make a "take two" on these
 algorithms with vectorization in mind. This is just my opinion.

 On Mon, Sep 13, 2021 at 2:10 PM Uwe Schindler  wrote:
 >
 > There are no good reasons to do Java 17 and it is way too early.
 >
 >
 >
 > Reagrding real optimizations, Lucene 17 is unfortunately not
 containing Project Panama or Vector API, so it looks more like Java 18/19
 is a good candidate as a new minimum at a later stage.
 >
 >
 >
 > I’d release Lucene 9 with Java 11 (which is LTS) and then decide
 later if we update to some post-17 version to get the new vector and panama
 APIs (vector search, SIMD and also MMapDirectory v2). If we do this, we
 should simply release Lucene 10.
 >
 >
 >
 > Uwe
 >
 >
 >
 > -
 >
 > Uwe Schindler
 >
 > Achterdiek 19, D-28357 Bremen
 >
 > https://www.thetaphi.de
 >
 > eMail: u...@thetaphi.de
 >
 >
 >
 > From: Mike Drob 
 > Sent: Monday, September 13, 2021 8:00 PM
 > To: Solr/Lucene Dev 
 > Subject: Java 11/17 Version Matrix
 >
 >
 >
 > Hi Devs,
 >
 >
 >
 > What are our thoughts on Java 11 and 17 version compatibility going
 forward for Lucene 9? Will we support both? If so, would Java 11 support
 likely continue for the entire 9.x release line?
 >
 >
 >
 > Is there a JIRA tracking this?
 >
 >
 >
 > Thanks,
 >
 > Mike

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

 --
>> Uwe Schindler
>> Achterdiek 19, 28357 Bremen
>> https://www.thetaphi.de
>>
>
>
> --
> Adrien
>

Re: New branch and feature freeze for Lucene/Solr 8.10.0

[Ishan Chattopadhyaya]

All the best, this is the worst step.

On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter,  wrote:

> Building RC1 now ... stay tuned.
>
> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter 
> wrote:
> >
> > Thanks for the update Mike!
> >
> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> > SOLR-15621, which looks like an easy win for the issue Cassandra
> > reported on Slack earlier today.
> >
> > Cheers,
> > Tim
> >
> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob  wrote:
> > >
> > > Hi Tim, I'm still working on SOLR-1, the code and benchmarking
> > > both look pretty good, but I've got a few last unit tests that I need
> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> > > sure to keep you updated though.
> > >
> > >
> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter 
> wrote:
> > > >
> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while
> testing
> > > > the schema designer. I haven't built the RC yet, so going to see if I
> > > > can get this in today.
> > > >
> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
> thelabd...@apache.org> wrote:
> > > > >
> > > > > NOTICE:
> > > > >
> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on
> stable branch.
> > > > >
> > > > > Please observe the normal rules:
> > > > >
> > > > > * No new features may be committed to the branch.
> > > > >
> > > > > * Documentation patches, build patches and serious bug fixes may be
> > > > >   committed to the branch. However, you should submit all patches
> you
> > > > >   want to commit to Jira first to give others the chance to review
> > > > >   and possibly vote against the patch. Keep in mind that it is our
> > > > >   main intention to keep the branch as stable as possible.
> > > > >
> > > > > * All patches that are intended for the branch should first be
> committed
> > > > >   to the unstable branch, merged into the stable branch, and then
> into
> > > > >   the current release branch.
> > > > >
> > > > > * Normal unstable and stable branch development may continue as
> usual.
> > > > >   However, if you plan to commit a big change to the unstable
> branch
> > > > >   while the branch feature freeze is in effect, think twice: can't
> the
> > > > >   addition wait a couple more days? Merges of bug fixes into the
> branch
> > > > >   may become more difficult.
> > > > >
> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker"
> will delay
> > > > >   a release candidate build.
> > > > > 
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: New branch and feature freeze for Lucene/Solr 8.10.0

[Timothy Potter]

Building RC1 now ... stay tuned.

On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter  wrote:
>
> Thanks for the update Mike!
>
> I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> SOLR-15621, which looks like an easy win for the issue Cassandra
> reported on Slack earlier today.
>
> Cheers,
> Tim
>
> On Thu, Sep 9, 2021 at 11:32 AM Mike Drob  wrote:
> >
> > Hi Tim, I'm still working on SOLR-1, the code and benchmarking
> > both look pretty good, but I've got a few last unit tests that I need
> > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> > sure to keep you updated though.
> >
> >
> > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter  wrote:
> > >
> > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> > > the schema designer. I haven't built the RC yet, so going to see if I
> > > can get this in today.
> > >
> > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter  
> > > wrote:
> > > >
> > > > NOTICE:
> > > >
> > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable 
> > > > branch.
> > > >
> > > > Please observe the normal rules:
> > > >
> > > > * No new features may be committed to the branch.
> > > >
> > > > * Documentation patches, build patches and serious bug fixes may be
> > > >   committed to the branch. However, you should submit all patches you
> > > >   want to commit to Jira first to give others the chance to review
> > > >   and possibly vote against the patch. Keep in mind that it is our
> > > >   main intention to keep the branch as stable as possible.
> > > >
> > > > * All patches that are intended for the branch should first be committed
> > > >   to the unstable branch, merged into the stable branch, and then into
> > > >   the current release branch.
> > > >
> > > > * Normal unstable and stable branch development may continue as usual.
> > > >   However, if you plan to commit a big change to the unstable branch
> > > >   while the branch feature freeze is in effect, think twice: can't the
> > > >   addition wait a couple more days? Merges of bug fixes into the branch
> > > >   may become more difficult.
> > > >
> > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will 
> > > > delay
> > > >   a release candidate build.
> > > > 
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Question - LUCENE-9077

[Drew Baugher]

Hello, had a few questions regarding the work done on: 
https://issues.apache.org/jira/browse/LUCENE-9077

I see on the issue it mentions "See notes below on why this respin is needed."
I'm not able to find the notes mentioned that talk about the reasons for the 
changes which would be nice to be able to look over... but I might just be 
looking in the wrong place, so if someone could point me to them that would be 
awesome!

Also since this is a significant change, is anyone aware of some article, blog 
post, or internal discussion that talks about what was gained out of the switch 
over to Gradle (performance, ease of development, etc., or perhaps this would 
all be in the notes mentioned above).

Thanks!

Drew
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Release Announcement: General Availability of Java 17 / JDK 17

[Rory O'Donnell]

Hi Uwe & Dawid,

*Release Announcement: General Availability of Java 17 / JDK 17 *

**

 * JDK 17, the reference implementation of Java 17, is now Generally
   Available. [1]
 * GPL-licensed OpenJDK builds from Oracle are available here:
   https://jdk.java.net/17/ 
 * JDK 17 Release notes
   
 * Inside Java: The Arrival of Java 17!
   

*JDK 17 includes the following features [2]:*

 * JEP 306: Restore Always-Strict Floating-Point Semantics
   
 * JEP 356: Enhanced Pseudo-Random Number Generators
   
 * JEP 382: New macOS Rendering Pipeline
   
 * JEP 391: macOS/AArch64 Port 
 * JEP 398: Deprecate the Applet API for Removal
   
 * JEP 403: Strongly Encapsulate JDK Internals
   
 * JEP 406: Pattern Matching for switch (Preview)
   
 * JEP 407: Remove RMI Activation 
 * JEP 409: Sealed Classes 
 * JEP 410: Remove the Experimental AOT and JIT Compiler
   
 * JEP 411: Deprecate the Security Manager for Removal
   
 * JEP 412: Foreign Function & Memory API (Incubator)
   
 * JEP 414: Vector API (Second Incubator)
   
 * JEP 415: Context-Specific Deserialization Filters
   

*JDK 17 will be a long-term-support (LTS) release* from most 
vendors,including Oracle. If you’re upgrading from the previous LTS 
release,JDK 11, then you have many more JEPs to look forward to, 
summarized here:


https://openjdk.java.net/jdk/17/jeps-since-jdk-11 




Thanks to everyone who contributed to JDK 17, whether by creating 
features or enhancements, logging bugs, or


downloading and testing the early-access builds.


*OpenJDK 18 Early Access build 14 is now available at 
https://jdk.java.net/18/ 

*

 * These early access, open source builds are provided under the GNU
   General Public License, version 2, with the Classpath Exception
   .
 * JEPs targeted to JDK 18, so far:
 o JEP 400: UTF-8 by Default 
 o JEP 413: Code Snippets in Java API Documentation
   

 * Release Notes are available at https://jdk.java.net/18/release-notes
   

 * Significant changes since the last availability email:
 o JDK-8271745: Fix Issues With the KW and KWP Modes of SunJCE Provider
 o JDK-8262186: Call X509KeyManager.chooseClientAlias once for all
   key types
 o JDK-8225083: Remove Google certificate that is expiring in
   December 2021
 o JDK-8251329: Zip File System Provider Throws ZipException when
   entry name element contains "." or ".."
 o JDK-8225082: Remove IdenTrust certificate that is expiring in
   September 2021
 o

*Project Loom Early-Access Builds*

 * Build 18-loom+2-74 (2021/8/7) based on jdk-18+9
    is
   available - https://jdk.java.net/loom/ 
 * These early access, open source builds are provided under the GNU
   General Public License, version 2, with the Classpath Exception
   .
 * Please send feedback via e-mail to loom-...@openjdk.java.net
   . To send e-mail to this address
   you must first subscribe to the mailing list
   .

Rgds,Rory


[1] 
https://mail.openjdk.java.net/pipermail/jdk-dev/2021-September/006037.html


[2] https://openjdk.java.net/projects/jdk/17/ 

Re: Java 11/17 Version Matrix

[Robert Muir]

Sorry, this is a bogus argument. Nobody is holding a gun to anyone's
head and forcing them to upgrade.

When we have new major functionality, we should be able to issue new
major releases, to hell with elasticsearch users.

On Tue, Sep 14, 2021 at 4:19 AM Adrien Grand  wrote:
>
> I think we should discuss options when Project Panama is released. Doing 
> frequent major releases forces users to reindex more often. If Project Panama 
> was released shortly and we decided to release Lucene 10 immediately, this 
> would force users to reindex their 8.x data to be able to upgrade, I know of 
> many Elasticsearch users for whom it would cause headaches and I suspect that 
> it's no different for Solr users and many direct users of Lucene. I'd rather 
> look into publishing a MR JAR of bumping the minimum required Java version in 
> a minor release than releasing Lucene 10.0 less than 1 year after 9.0.
>
> On Mon, Sep 13, 2021 at 9:19 PM Uwe Schindler  wrote:
>>
>> In addition: we test for compatibility with Java 17 (both Lucene and Solr), 
>> so consumer is still able to use any version and has enough flexibility.
>>
>> Uwe
>>
>> Am 13. September 2021 18:52:53 UTC schrieb Dawid Weiss 
>> :
>>>
>>>
>>> I agree with Uwe and Robert. JDK11, then the min bar should move if there 
>>> is something that brings value (be it performance, LTS or some other 
>>> attractive option).
>>>
>>> Dawid
>>>
>>> On Mon, Sep 13, 2021 at 8:15 PM Robert Muir  wrote:

 +1, I think the main thing to watch out for is project panama. If we
 get a 18.x/19.x release with non-incubating APIs, I think it makes
 sense to create a new major Lucene version. Even if it isn't an
 OpenJDK LTS release. It could really change a lot, especially
 regarding hotspots in the code such as postings/dv compression. So it
 would be great to allow a lot of folks to make a "take two" on these
 algorithms with vectorization in mind. This is just my opinion.

 On Mon, Sep 13, 2021 at 2:10 PM Uwe Schindler  wrote:
 >
 > There are no good reasons to do Java 17 and it is way too early.
 >
 >
 >
 > Reagrding real optimizations, Lucene 17 is unfortunately not containing 
 > Project Panama or Vector API, so it looks more like Java 18/19 is a good 
 > candidate as a new minimum at a later stage.
 >
 >
 >
 > I’d release Lucene 9 with Java 11 (which is LTS) and then decide later 
 > if we update to some post-17 version to get the new vector and panama 
 > APIs (vector search, SIMD and also MMapDirectory v2). If we do this, we 
 > should simply release Lucene 10.
 >
 >
 >
 > Uwe
 >
 >
 >
 > -
 >
 > Uwe Schindler
 >
 > Achterdiek 19, D-28357 Bremen
 >
 > https://www.thetaphi.de
 >
 > eMail: u...@thetaphi.de
 >
 >
 >
 > From: Mike Drob 
 > Sent: Monday, September 13, 2021 8:00 PM
 > To: Solr/Lucene Dev 
 > Subject: Java 11/17 Version Matrix
 >
 >
 >
 > Hi Devs,
 >
 >
 >
 > What are our thoughts on Java 11 and 17 version compatibility going 
 > forward for Lucene 9? Will we support both? If so, would Java 11 support 
 > likely continue for the entire 9.x release line?
 >
 >
 >
 > Is there a JIRA tracking this?
 >
 >
 >
 > Thanks,
 >
 > Mike

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

>> --
>> Uwe Schindler
>> Achterdiek 19, 28357 Bremen
>> https://www.thetaphi.de
>
>
>
> --
> Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: getField vs getDeclaredField in analysis SPI

[Alan Woodward]

Thanks for opening the issue Uwe! I agree that option #1 is the best long term 
fix (although for the moment I’ve wrapped the relevant ES code in a 
doPrivileged block which solves the immediate problem).

> Is Elasticsearch now using the Analysis Factory framework instead of their 
> own factories?

No, we still have our own factory implementations.  In fact, looking more 
closely at this, I don’t think we need to call these reload SPI methods at all 
and I can probably remove it entirely :) Still, it found a possible bug!


> On 13 Sep 2021, at 12:48, Uwe Schindler  wrote:
> 
> See https://issues.apache.org/jira/browse/LUCENE-10101
> 
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
> 
>> -Original Message-
>> From: Uwe Schindler 
>> Sent: Monday, September 13, 2021 9:59 AM
>> To: dev@lucene.apache.org
>> Subject: RE: getField vs getDeclaredField in analysis SPI
>> 
>> Hi Alan,
>> 
>> I will open an issue about this!
>> 
>> Uwe
>> 
>> -
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> https://www.thetaphi.de
>> eMail: u...@thetaphi.de
>> 
>>> -Original Message-
>>> From: Uwe Schindler 
>>> Sent: Monday, September 6, 2021 4:57 PM
>>> To: dev@lucene.apache.org
>>> Subject: RE: getField vs getDeclaredField in analysis SPI
>>> 
>>> Hi Alan,
>>> 
>>> Would you open issue, I will take it!?
>>> 
>>> Maybe also post your opinion about think fix #1 or fix #2 is better. I tend 
>>> to
>> go
>>> for fix #1. getDeclaredField() should theoretically be faster, but that 
>>> won't
>>> matter here: If it goes the slow path (going up to superclass) it will fail
>> anyways
>>> and that's the exceptional case. A correct factory should have a NAME field
>> and
>>> its lookup is fast and the additional check introduced for the class is 
>>> cheap.
>>> 
>>> Uwe
>>> 
>>> -
>>> Uwe Schindler
>>> Achterdiek 19, D-28357 Bremen
>>> https://www.thetaphi.de
>>> eMail: u...@thetaphi.de
>>> 
 -Original Message-
 From: Alan Woodward 
 Sent: Monday, September 6, 2021 3:48 PM
 To: dev@lucene.apache.org
 Subject: Re: getField vs getDeclaredField in analysis SPI
 
 Thanks Uwe!
 
> On 6 Sep 2021, at 13:11, Uwe Schindler  wrote:
> 
> Hi Alan,
> 
>> LUCENE-9281 moved the `lookupSPIName` method from
>> AbstractAnalysisFactory to AnalysisSPILoader; the method is mostly the
 same,
>> but one line has been changed from Class.getField() to
 Class.getDeclaredField().
>> This can fall foul of the Security Manager, which wants a higher level of
>> permission for getDeclaredField.  Was this an intentional change? As I
> 
> This was intentional because the previous code wasn't fully correct,
>> because
>>> I
 had some safety check in mind: The main reason for the getDeclaredField()
>> is
>>> to
 lookup the field only in this class; while getField() also looks into
>> superclasses.
 E.g. if the superclass has a NAME field because of a programming error it
>>> would
 pick that up, which would be wrong. When investigating other
 implementations using "named" lookups out there (even in JDK), they used
 getDeclaredField() when accessing a static member.
> 
> There are 2 solutions:
> - Change to getField(), but in the if statement below check the actual
>> class:
 (field.getDeclaringClass()==service) (see https://github.com/apache/lucene-
 solr/pull/1360/files#diff-
 
>>> 
>> 6a65d91199a18bc4ee2d00a1e9dc283aedc4134846e0d7aafdc484f8263e250bR
 159-R162)
> - Wrap with doPrivileged in Lucene code. As far as I remember Lucene
>> needs
 the permission anyways. With doPrivileged you would delegate
>> responsibility.
> 
> I'd open a JIRA issue, I can fix this. It only affects Lucene 9.0.
> 
>> understand it it’s looking for a NAME static field on the class in 
>> question,
 which
>> should always be public. I’m in the process of upgrading elasticsearch to
>>> use
 a
>> lucene 9 snapshot, and this change means that I need to wrap SPI
>>> reloading
>> code in doPrivileged() blocks, which is a bit of a pain.
> 
> Thansk for doing this. Is Elasticsearch now using the Analysis Factory
 framework instead of their own factories?
> 
>> Thanks, Alan
> 
> No problem,
> Uwe
> 
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 
 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org
>>> 
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For 

Re: Java 11/17 Version Matrix

[Adrien Grand]

I think we should discuss options when Project Panama is released. Doing
frequent major releases forces users to reindex more often. If Project
Panama was released shortly and we decided to release Lucene 10
immediately, this would force users to reindex their 8.x data to be able to
upgrade, I know of many Elasticsearch users for whom it would cause
headaches and I suspect that it's no different for Solr users and many
direct users of Lucene. I'd rather look into publishing a MR JAR of bumping
the minimum required Java version in a minor release than releasing Lucene
10.0 less than 1 year after 9.0.

On Mon, Sep 13, 2021 at 9:19 PM Uwe Schindler  wrote:

> In addition: we test for compatibility with Java 17 (both Lucene and
> Solr), so consumer is still able to use any version and has enough
> flexibility.
>
> Uwe
>
> Am 13. September 2021 18:52:53 UTC schrieb Dawid Weiss <
> dawid.we...@gmail.com>:
>>
>>
>> I agree with Uwe and Robert. JDK11, then the min bar should move if there
>> is something that brings value (be it performance, LTS or some other
>> attractive option).
>>
>> Dawid
>>
>> On Mon, Sep 13, 2021 at 8:15 PM Robert Muir  wrote:
>>
>>> +1, I think the main thing to watch out for is project panama. If we
>>> get a 18.x/19.x release with non-incubating APIs, I think it makes
>>> sense to create a new major Lucene version. Even if it isn't an
>>> OpenJDK LTS release. It could really change a lot, especially
>>> regarding hotspots in the code such as postings/dv compression. So it
>>> would be great to allow a lot of folks to make a "take two" on these
>>> algorithms with vectorization in mind. This is just my opinion.
>>>
>>> On Mon, Sep 13, 2021 at 2:10 PM Uwe Schindler  wrote:
>>> >
>>> > There are no good reasons to do Java 17 and it is way too early.
>>> >
>>> >
>>> >
>>> > Reagrding real optimizations, Lucene 17 is unfortunately not
>>> containing Project Panama or Vector API, so it looks more like Java 18/19
>>> is a good candidate as a new minimum at a later stage.
>>> >
>>> >
>>> >
>>> > I’d release Lucene 9 with Java 11 (which is LTS) and then decide later
>>> if we update to some post-17 version to get the new vector and panama APIs
>>> (vector search, SIMD and also MMapDirectory v2). If we do this, we should
>>> simply release Lucene 10.
>>> >
>>> >
>>> >
>>> > Uwe
>>> >
>>> >
>>> >
>>> > -
>>> >
>>> > Uwe Schindler
>>> >
>>> > Achterdiek 19, D-28357 Bremen
>>> >
>>> > https://www.thetaphi.de
>>> >
>>> > eMail: u...@thetaphi.de
>>> >
>>> >
>>> >
>>> > From: Mike Drob 
>>> > Sent: Monday, September 13, 2021 8:00 PM
>>> > To: Solr/Lucene Dev 
>>> > Subject: Java 11/17 Version Matrix
>>> >
>>> >
>>> >
>>> > Hi Devs,
>>> >
>>> >
>>> >
>>> > What are our thoughts on Java 11 and 17 version compatibility going
>>> forward for Lucene 9? Will we support both? If so, would Java 11 support
>>> likely continue for the entire 9.x release line?
>>> >
>>> >
>>> >
>>> > Is there a JIRA tracking this?
>>> >
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > Mike
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de
>


-- 
Adrien