Re: SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-28 Thread Toke Eskildsen
Wittenberg, Lucas wrote: > As suggested I switched to using DocValues and SortedDocValues. > Now QTime is down to an average of 1100, which is much, much better > but still far from the 30 I had with SOLR 4. > I suppose it is due to the block-oriented compression you mentioned. I apologize for

RE: SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-28 Thread Wittenberg, Lucas
Ok, thank you Erick and Toke. As suggested I switched to using DocValues and SortedDocValues. Now QTime is down to an average of 1100, which is much, much better but still far from the 30 I had with SOLR 4. I suppose it is due to the block-oriented compression you mentioned. Not sure if it is

Re: SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-27 Thread Erick Erickson
> I don't know the precedence rules for stored vs. dovValues in Solr DocValues are used if (and only if) all the fields being returned have docValues=“true” _and_ are single-valued, or if you’ve explicitly set useDocValuesAsStored. single-valued docValues are they only situation where the

Re: SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-27 Thread Toke Eskildsen
On Tue, 2019-08-27 at 09:05 +, Wittenberg, Lucas wrote: > But the "customid" field is already set as docValues="true" actually. > Well, I guess so as it is a type="string" which by default has > docValues="true". > > required="true" multiValued="false" /> > docValues="true" /> Yeah, it's a

Re: SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-27 Thread Toke Eskildsen
On Mon, 2019-08-26 at 16:01 +, Wittenberg, Lucas wrote: > @Override > public void collect(int docNumber) throws IOException { > if (null != this.reader && > isValid(this.reader.document(docNumber).get("customid"))) > { >

Re: SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-27 Thread Erick Erickson
Well, the question is then whether you’re getting the value from the docValues structure or the stored structure. My bet is the latter. Simple test would be to comment out the line and return some random value just to see how long it takes. > On Aug 27, 2019, at 5:05 AM, Wittenberg, Lucas >

RE: SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-27 Thread Wittenberg, Lucas
Thanks for the suggestion. But the "customid" field is already set as docValues="true" actually. Well, I guess so as it is a type="string" which by default has docValues="true". -Message d'origine- De : Wittenberg, Lucas Envoyé : lundi 26 août 2019 18:01 À :

Re: SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-26 Thread Erick Erickson
Is “customid” a docValues=true field? I suspect not, in which case I think this is the problem (but do be warned, I don’t spend much time in Lucene code). this.reader.document(docNumber).get("customid”) document(docNumber) goes out to do a disk read I think. If it were docValues=true, it could

SOLR 7+ / Lucene 7+ and performance issues with DelegatingCollector and PostFilter

2019-08-26 Thread Wittenberg, Lucas
Hello all, Here is the situation I am facing. I am migrating from SOLR 4 to SOLR 7. SOLR 4 is running on Tomcat 8, SOLR 7 runs with built in Jetty 9. The largest core contains about 1,800,000 documents (about 3 GB). The migration went through smoothly. But something's bothering me. I have a