> Having that explicitly called out would have been SUPER helpful.
Blaming Java in an exception thrown by Lucene is a ridiculous idea.
On Wed, 17 May, 2023, 3:33 am Gus Heck, <gus.h...@gmail.com> wrote:
> Found it.
>
> It's a solr thing made worse by the interaction of lucene testutils and
> jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
> when checking a URL
> /*
> * Checks whether the resource URL should be returned.
> * Returns null on security check failure.
> * Called by java.net.URLClassLoader.
> */
> public static URL checkURL(URL url) {
> if (url != null) {
> try {
> check(url);
> } catch (Exception e) {
> return null;
> }
> }
> return url;
> }
>
> Yay. Fun. JDK classes swallowing exceptions silently.
>
> At the start of this it only took me a little while to discover that there
> was a security manager in play via debugging. Remembering that I saw emails
> about that, I went to jira, found the ticket enabling it by default in 9.x
> and eventually tracked down the name of the security policy file by reading
> solr.in.sh and /bin/solr... The key issue that tripped me up is that the
> tests have a *separate* security policy file, and there was pretty much no
> way to know this without extensive reading of the build. Thus I got thrown
> off track when
>
> permission java.io.FilePermission
> "${user.home}${/}.m2${/}repository${/}-", "read";
>
> To solr/server/etc/security.policy had no effect. That and the fact that
> no security exception was reported, led me to start chasing increasingly
> improbable hypotheses. Many hours later when I went back to debugging
> deeply into class loading, I found that the code was actually reading the
> jar files in question, and then I finally caught it throwing a security
> exception during my debugging.
>
> It turns out that adding the above permission to
> gradle/testing/randomization/policies/solr-tests.policy allows the test to
> pass. [1]
>
> I think we need to document this somewhere (or someone needs to point me to
> the doc I missed, FWIW I hit this basically following the process in
> dev-docs/dependency-upgrades.adoc treating lucene like a dependency, and
> unaware that there is a "shortcut" mode for lucene specifically in
> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading that
> file none-to clear anyway)
>
> That's the solr part, the lucene part is that the security exception is hit
> when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> when org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
> does
>
> savedCodec = Codec.getDefault();
>
> The error message "An SPI class of type org.apache.lucene.codecs.Codec with
> name 'Lucene95' does not exist." was moderately misleading because the file
> and the services files in the jar definitely did exist. This message should
> vary if there is an installed security manager, maybe saying something
> like:
>
> "An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95'
> does not exist. We have detected that a security manager is installed so it
> is also possible that the jar containing the codec is inaccessible under
> the current security policy. (Java does not throw SecurityException if this
> is the case, it just ignores the jar!)" [2]
>
> Having that explicitly called out would have been SUPER helpful.
>
> -Gus
>
> [1]: https://issues.apache.org/jira/browse/SOLR-16804
> [2]: https://github.com/apache/lucene/issues/12300
>
>
> On Mon, May 15, 2023 at 3:17 PM Michael Sokolov <msoko...@gmail.com>
> wrote:
>
> > random guess - does it have something to do with modules?
> >
> > On Mon, May 15, 2023 at 11:14 AM Gus Heck <gus.h...@gmail.com> wrote:
> > >
> > > I hadn't seen that one. Thanks, I'll look at it. It already looks a bit
> > confusing though since it seems to have options for pointing to a repo,
> but
> > I appear to be pulling the jars successfully from .m2/repository
> already...
> > (except then they don't work, so successful means I see them in the
> > classpath of the relevant classloader). And if we can't deploy a valid
> jar
> > to mavenLocal for some reason (tweaked the solr build so it sees
> > mavenLocal()), (or solr can't consume such a jar) that seems like an
> issue
> > for whichever one is breaking that.
> > >
> > > Debugging: The JDK appears to be attempting to load the services file
> > from modules, but not seeing the lucene module. (just the jdk ones) Also
> it
> > passes through a block that says:
> > >
> > > // not in a package of a module defined to this loader
> > > for (URL url : findMiscResource(name)) {
> > >
> > > (but then iterates
> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load
> things
> > anyway)
> > >
> > > -Gus
> > >
> > > On Mon, May 15, 2023 at 10:54 AM Houston Putman <
> houstonput...@gmail.com>
> > wrote:
> > >>
> > >> Gus, I haven't done this myself, but are you using the instructions
> > provided in Solr's "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
> > >>
> > >> It looks like you need to specify the development lucene version
> > differently than other dependencies...
> > >>
> > >> - Houston
> > >>
> > >> On Sat, May 13, 2023 at 10:14 AM Michael Sokolov <msoko...@gmail.com>
> > wrote:
> > >>>
> > >>> doh I actually read your email and you said you already checked that
> -
> > >>> I'm going to send out one of those "sokolov would like to retract the
> > >>> previous email" emails. Does GMail even pretend to do that? I don't
> > >>> know what's going on there! sorry
> > >>>
> > >>> On Sat, May 13, 2023 at 10:13 AM Michael Sokolov <msoko...@gmail.com
> >
> > wrote:
> > >>> >
> > >>> > sorry - META-INF not WEB-INF
> > >>> >
> > >>> > On Sat, May 13, 2023 at 10:12 AM Michael Sokolov <
> msoko...@gmail.com>
> > wrote:
> > >>> > >
> > >>> > > You are probably missing the contents of WEB-INF in your custom
> > jar?
> > >>> > > Roughly speaking the files in there define run-time-bound
> > "services"
> > >>> > > that are looked up by name by the JDK's service-loader API.
> > >>> > >
> > >>> > > On Sat, May 13, 2023 at 9:33 AM Gus Heck <gus.h...@gmail.com>
> > wrote:
> > >>> > > >
> > >>> > > > Cross posting to lucene on the possibility that folks here are
> > more likely to add customized lucene to Solr and recognize what I'm
> > stumbling on? (zero responses on solr list)
> > >>> > > >
> > >>> > > > Note that the specific test that I happened to copy is not the
> > issue, all tests are doing this (or at least so many tests are failing I
> > can't see the ones that are passing easily).
> > >>> > > >
> > >>> > > > ---------- Forwarded message ---------
> > >>> > > > From: Gus Heck <gus.h...@gmail.com>
> > >>> > > > Date: Wed, May 10, 2023 at 6:50 PM
> > >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
> > >>> > > > To: <dev@solr.apache.org>
> > >>> > > >
> > >>> > > >
> > >>> > > > Lucene:
> > >>> > > >
> > >>> > > > I made a tweak to lucene for something I'm investigating, gave
> > it a new version, deployed to mavenLocal()
> > >>> > > > I have verified that the jars are built with correct
> > META-INF/services files
> > >>> > > >
> > >>> > > > Solr:
> > >>> > > >
> > >>> > > > I added mavenLocal() in gradle/globals.gradle
> > >>> > > > I removed the license file sha1 sigs for the default lucene &
> > creates signatures for my test version
> > >>> > > > I updated versions.props
> > >>> > > > I updated versions.lock
> > >>> > > >
> > >>> > > > Now when I run individual solr tests via my ide they seem to
> > pass, but virtually every test run via gradle fails with something like:
> > >>> > > >
> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
> FAILED
> > >>> > > > java.lang.ExceptionInInitializerError
> > >>> > > > at
> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > >>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >>> > > >
> > >>> > > > Caused by:
> > >>> > > > java.lang.IllegalArgumentException: An SPI class of
> type
> > org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You
> > need to add the corresponding JAR file supporting this SPI to your
> > classpath. The current classpath supports the following names: []
> > >>> > > > at
> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > >>> > > > at
> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > >>> > > > ... 19 more
> > >>> > > >
> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
> FAILED
> > >>> > > > java.lang.NullPointerException
> > >>> > > > at
> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > >>> > > > at
> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > >>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >>> > > >
> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's
> > output saved to
> >
> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
> > copied below:
> > >>> > > > > java.lang.ExceptionInInitializerError
> > >>> > > > > at
> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > >>> > > > > at
> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > >>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >>> > > > >
> > >>> > > > > Caused by:
> > >>> > > > > java.lang.IllegalArgumentException: An SPI class
> of
> > type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
> > You need to add the corresponding JAR file supporting this SPI to your
> > classpath. The current classpath supports the following names: []
> > >>> > > > > at
> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > >>> > > > > at
> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > >>> > > > > ... 19 more
> > >>> > > > > java.lang.NullPointerException
> > >>> > > > > at
> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > >>> > > > > at
> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > >>> > > > > at
> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > >>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
> > (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
> > >>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
> > >>> > > > 2> NOTE: reproduce with: gradlew test --tests
> > TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
> > -Dtests.file.encoding=UTF-8
> > >>> > > >
> > >>> > > > The only difference I've been able to determine is that the IDE
> > does not run the test with a security manager, but that feels like a red
> > herring since nothing is throwing security exception (or if it is, it's
> > getting swallowed!).
> > >>> > > >
> > >>> > > > I have debugged and I can see that the classloaders have the
> new
> > lucene jars I built on the classpath (in /Users/gus/.m2/repository as
> they
> > should), but I haven't yet figuree out why the services are not loading.
> > >>> > > >
> > >>> > > > -Gus
> > >>> > > >
> > >>> > > >
> > >>> > > > --
> > >>> > > > http://www.needhamsoftware.com (work)
> > >>> > > > http://www.the111shift.com (play)
> > >>> > > >
> > >>> > > >
> > >>> > > > --
> > >>> > > > http://www.needhamsoftware.com (work)
> > >>> > > > http://www.the111shift.com (play)
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > >>> For additional commands, e-mail: dev-h...@lucene.apache.org
> > >>>
> > >
> > >
> > > --
> > > http://www.needhamsoftware.com (work)
> > > http://www.the111shift.com (play)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
> >
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
