On Tue, Jan 26, 2016 at 9:35 AM, Uwe Schindler <[email protected]> wrote: > Hi, > >> Down at the bottom level of the SPI mechanism, the API accepts >> ClassLoader objects. However, the Directory API does not have methods >> that take a class loader, so, in practical terms, the SPI mechanism >> always uses the Thread Context Class Loader. > > Not entirely true. We use context classloader as per ServiceLoader spec, but > we also inspect Lucene's class loader. > >> I just spent some time sorting out a muddle in an OSGi application >> using Lucene, eventually resolved by setting the TCCL correctly. >> >> I wonder what people think of the following possible activities: >> >> 1: add an explicit ClassLoader the Directory classes (and whatever >> below them) to allow applications to take advantage of the SPI's API. > > Directory has nothing to do with SPI. It is only used by Analyzers and > DirectoryReader/IndexWriter.
The stacktrace in question enters through Directory. You open a directory, it opens a segment, it tries to find a codec. It fails to find the codec if the TCCL does not contain the right copy of the lucene-codecs jar. So, I submit, it makes sense for Directory.open to have doc to have a note about this; it could possibly make sense to have an explicit class loader, but I'm not motivated to push it. I accept that the thing to note is more complex than I wrote quickly in this this. I can fill in more details if you are interested. > > But the problem is Codec.forName() and others are static methods, so it is > not easy to change this to other classloaders. This is similar to other stuff > like Java's Locales, Charsets, Image in Java, but also TIKA stuff. They all > depend on static SPI behaviour, not compatible with differen class loaders. > > To tell Lucene to load its stuff from other classloaders, we have methods in > Codec, PostingsFormats,... to revisit classloaders and collect new SPIs from > there (and you can pass one). > >> 2: add a bit of javadoc to the Directory classes that reminds people >> that these APIs consume the TCCL. > > As above, it does, but not only. Not sure what your problem is...But, I think > the issue is that OSGI tries to load every JAR file into its own classloader > and this breaks it. So lucene-core.jar will not see any other Jar files, so > it falls back to context class loader. > > It is well known that ServiceLoader and other Java stuff around that does not > play nicely with all OSGI complexity (see below). > >> 3: nothing. >> >> Meanwhile, I've been thinking about a proposal to make Lucene build >> OSGi bundles itself, rather than leaving the job to Apache Servicemix. >> >> The hard part here is the OSGi principle that every Java _package_ has >> a single home Jar file. I don't think I have the stomach lining to try >> to talk anyone around here into some sort of major refactoring to that >> end. The servicemix jars work around this adequately. One thing the >> servicemix jars don't deal with is lucene-codecs. >> >> I guess I'd like to start by asking how the dev community feels about >> the whole idea of getting native support for OSGi into Lucene. If >> there's a strong reaction of _ugh_, I won't push it. > > _uuuuuuuuuuuuGGGGGHHHhhhhhhh_ > > OSGI is incompatible with Lucene's pacakging, sorry, that is hard to fix. And > in my personal opinion, people who want to make OSGI modules out of Lucene > may do this separately, but it will result in major havoc, sorry. > > This is the same discussion like people requesting "serialization". To make > Lucene work with you own OSGI stuff, you have to repackage your Lucene > dependencies into "one big" single JAR file, otherwise you would also get > huge slowdowns because of classloader separation (OSGI will add proxy classes > everywhere...). It's not nearly as bad as that, it turns out. Servicemix builds three OSGi bundles for Lucene: core, analyzers, and queryparsers. It's not one big jar. https://github.com/apache/servicemix-bundles/blob/master/lucene-5.4.0/pom.xml or https://github.com/apache/servicemix-bundles/blob/master/lucene-analyzers-common-5.4.0/pom.xml. So, ugh registered, but I ask you to look at those and see if they reduce the size of the ugh at all in contemplating them as part of Lucene itself. > > Uwe > > >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
