Hi Hoss, Thanks for the detailed explanation. This all makes sense now including the specific error message and my multiple errors .
I put the correct org.apache.lucene.codecs.PostingsFormat in the jar, indexed and searched some documents and everything is working fine. I'll push the code/configuration to our test machines, index a few hundred GB of book OCR, and see if this will now enable us to run our normal indexing and searching with significantly less memory. Tom Tom On Fri, Mar 13, 2015 at 12:26 PM, Chris Hostetter <[email protected]> wrote: > > : I don't really understand SPI and class loaders, but you are right this > : class is a subclass of PostingsFormat not Codecs. So is there an issue > : with the whole idea, or is there just some subtlety of class loading and > : the SPILoader I'm not understanding? > > SPI is just a mechanism Java gives you to dynamicly load Class > implementations of an Abstraction using a symbolic name -- it's a > factory pattern, where the Factory (provided by the JVM) doesn't have > to know all of the implementations (which can be written by users and > made available on the classpath in arbitrary Jar files. > > the files you put in your META-INF/services directory should match the > class name that Lucene is expecing and that you are extending -- so if you > are extending PostingFormat, then you register that as a PostingFormat > implementation in a > META-INF/services/org.apache.lucene.codecs.PostingFormat file inside your > jar. > > you don't need/want to specify any other classes in those service files. > In the service file you mentioned trying ot use... > > : Contents of META-INF/services/org.apache.lucene.codecs.Codec in the jar > : file: > : org.apache.lucene.codecs.lucene49.Lucene49Codec > : org.apache.lucene.codecs.lucene410.Lucene410Codec > : # tbw adds custom wrapper here per Hoss e-mail > : org.apache.lucene.codecs.HTPostingsFormatWrapper > > ...you're telling the JVM 3 things that are incorrect... > > 1) that your jar contains a Lucene49Codec class (it does not) > 2) that your jar contains a Lucene410Codec class (also no) > 3) that your jar contains an HTPostingsFormatWrapper (true but) which > extends Codec (it does not) > > > ...and at no point does your services file tell SPI that your > HTPostingsFormatWrapper is/can-be a PostingsFormat. > > Your speciic error seems to be coming from Lucene trying to scan the list > of SPI loadable *Codec* implementations, and being confused because you've > said HTPostingsFormatWrapper is an implementation of "Codec" but it can't > be cast as a Codec. > > > > -Hoss > http://www.lucidworks.com/ >
