RE: Cannot instantiate SPI class: org.apache.lucene.codecs.lucene42.Lucene42Codec

2013-04-09 Thread Uwe Schindler
Hi Manivannan, > thanks for the reply. > Im not using any custom class loaders, I tried using the 4.2.1, i got the same > issue, facet42 is found, but not lucene42. > i have the facet.jar in my final jar. > > location="../libs/luceneV4.2/lucene-codecs-4.2.0.jar"/>* > > ** > > *

Re: Cannot instantiate SPI class: org.apache.lucene.codecs.lucene42.Lucene42Codec

2013-04-09 Thread Manivannan Selvadurai
thanks for the reply. Im not using any custom class loaders, I tried using the 4.2.1, i got the same issue, facet42 is found, but not lucene42. i have the facet.jar in my final jar. location="../libs/luceneV4.2/lucene-codecs-4.2.0.jar"/>* > ** > *** One more information.

Re: Indexing Term Frequency Vectors

2013-04-09 Thread Adrien Grand
Hi, On Tue, Apr 9, 2013 at 5:24 PM, Sharon Tam wrote: > I tried following following this payloads tutorial to attach the term > frequencies as payloads: > http://searchhub.org/2009/08/05/getting-started-with-payloads/ > > But I'm confused as to where I need to override the term frequency counter

Re: Token Stream with Offsets (Token Sources class)

2013-04-09 Thread vempap
well, I found out the issue - it is because the maxDocCharsToAnalyze is 0 in the weightedSpanTermsExtractor by default. Works fine if I change there or use the QueryScorer which has a default limit of 51200. Thanks. - -- Phani -- View this message in context: http://lucene.472066.n3.nabble

Re: DocValues space usage

2013-04-09 Thread Wei Wang
Adrien and Rober, thanks a lot for the hints. Will try a few options and see how it goes. On Tue, Apr 9, 2013 at 9:25 AM, Robert Muir wrote: > On Tue, Apr 9, 2013 at 9:11 AM, Adrien Grand wrote: > > > The default codec stores numeric doc values by blocks of 4096 values > > that have independent

RE: Detecting when an index was not closed properly

2013-04-09 Thread Becker, Thomas
To answer my own question here, I switched from the default NativeFSLockFactory to the SimpleFSLockFactory. This allows me to detect when the IndexWriter was not properly closed since an attempt to open a new one will throw a LockObtainFailedException, which I can key on to know to rebuild the

Re: FacetedSearch and MultiReader

2013-04-09 Thread Shai Erera
Hello Nicola, I think it would be good if you start a new thread to discuss this problem, as I don't think it's related to the issue in this thread. Also, I did not understand what's the problem you're running into. What used to work before 4.2 and doesn't work now? Shai On Tue, Apr 9, 2013 at

Re: DocValues space usage

2013-04-09 Thread Robert Muir
On Tue, Apr 9, 2013 at 9:11 AM, Adrien Grand wrote: > The default codec stores numeric doc values by blocks of 4096 values > that have independent numbers of bits per values. If you end up having > most of these blocks empty, doc values will require little space but > in a worst-case scenario whe

Re: DocValues space usage

2013-04-09 Thread Robert Muir
On Tue, Apr 9, 2013 at 9:06 AM, Wei Wang wrote: > Thanks for the hint. Could you point to some Codec that might do this for > some types, even just as an side effect as you mentioned? It will be > helpful to have something to start with. > Have a look at diskdv/ codec in the codecs/ module. Its

Re: DocValues space usage

2013-04-09 Thread Adrien Grand
Hi, On Tue, Apr 9, 2013 at 5:22 PM, Wei Wang wrote: > DocValues makes fast per doc value lookup possible, which is nice. But it > brings other interesting issues. > > Assume there are 100M docs and 200 NumericDocValuesFields, this ends up > with huge number of disk and memory usage, even if there

Re: DocValues space usage

2013-04-09 Thread Wei Wang
Thanks for the hint. Could you point to some Codec that might do this for some types, even just as an side effect as you mentioned? It will be helpful to have something to start with. And could you elaborate a bit more for "the facet on tons of sparse fields"? I just got a vague idea from the comm

Re: DocValues space usage

2013-04-09 Thread Robert Muir
On Tue, Apr 9, 2013 at 8:22 AM, Wei Wang wrote: > DocValues makes fast per doc value lookup possible, which is nice. But it > brings other interesting issues. > > Assume there are 100M docs and 200 NumericDocValuesFields, this ends up > with huge number of disk and memory usage, even if there are

Re: FacetedSearch and MultiReader

2013-04-09 Thread Nicola Buso
Hi, I'm trying to use Lucene 4.2, but this merge of more taxonomy indexes seam is no more working. Do you have any idea why it has not to work in Lucene 4.2? Normal faceted search on a single index is working correctly. Nicola. On Thu, 2013-01-24 at 16:53 +, Nicola Buso wrote: > Hi Shai, >

DocValues space usage

2013-04-09 Thread Wei Wang
DocValues makes fast per doc value lookup possible, which is nice. But it brings other interesting issues. Assume there are 100M docs and 200 NumericDocValuesFields, this ends up with huge number of disk and memory usage, even if there are just thousands of values for each field. I guess this is b

RE: How to improve retrieval time when searching for a date range

2013-04-09 Thread Uwe Schindler
Hi, Precision step=1 is not necessarily the fastest (see javadocs of Lucene, should be similar in Lucene.NET). Try the default, 4, first. In general, those range queries will always be slower than text-only queries, as there is much more work to do (more terms, more documents,...) This questio

How to improve retrieval time when searching for a date range

2013-04-09 Thread Avi Levy
Hello, I have a Lucene.NET index created with version 2.9.4.1. The size of the index is about 25 Million entries (In the production environment I will have 50 Million or more). The Index size is 5.75GB. The index is used for searching by text. I need to add a new functionality that allows performi

RE: Cannot instantiate SPI class: org.apache.lucene.codecs.lucene42.Lucene42Codec

2013-04-09 Thread Uwe Schindler
There may be 2 problems, to fix: a) use 4.2.1 (if you are using custom class loaders in your application) b) The stack trace says, that it can only find the codec "Facet42". This codec is only in lucene-facet.jar, but this JAR file is not in your classpath. So you seem to have some classpath prob

Re: Cannot instantiate SPI class: org.apache.lucene.codecs.lucene42.Lucene42Codec

2013-04-09 Thread Manivannan Selvadurai
i don't get what u mean by setup. im not using solr. Could u please be more specific. we dont use maven for building the jar. Its just few build.xmls. It is just plain Lucene 4.2. I have to create an index using some tokenizers and filters. with thanks, Manivannan On Tue, Apr 9, 2013 at 4:07 PM,

RE: Cannot instantiate SPI class: org.apache.lucene.codecs.lucene42.Lucene42Codec

2013-04-09 Thread Uwe Schindler
As mentioned in the original answer, what is the setup you are using? - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Manivannan Selvadurai [mailto:manivan...@unmetric.com] > Sent: Tuesday, April 09, 201

Re: Cannot instantiate SPI class: org.apache.lucene.codecs.lucene42.Lucene42Codec

2013-04-09 Thread Manivannan Selvadurai
I dint understand what u mean by directly inside ant or just for compiling, but this is the process we do Im using the Lucene jars in a build.xml and we use ant to build the jar and i invoke the program on that resulting jar. So will using lucene 4.2.1 fix the issue for me? On Tue, Apr 9, 2013

RE: Cannot instantiate SPI class: org.apache.lucene.codecs.lucene42.Lucene42Codec

2013-04-09 Thread Uwe Schindler
This may be fixed in 4.2.1, but this depends on your setup. Are you using Lucene directly inside ANT or is it just for compiling code? Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Manivannan Sel

Cannot instantiate SPI class: org.apache.lucene.codecs.lucene42.Lucene42Codec

2013-04-09 Thread Manivannan Selvadurai
Hi All, I'm facing this problem. When i build a jar out of my project and try to run the program that invokes lucene part of the code, it gives me this exception Exception in thread "main" java.util.ServiceConfigurationError: Cannot instantiate SPI class: org.apache.lucene.codecs.