Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-23 Thread Ignacio Vera
gt; > > > Even retrieving single document 8.7.0 is more than x2 slower > > > > > > > > On Fri, Jan 22, 2021 at 2:28 PM Diego Ceccarelli (BLOOMBERG/ LONDON) > < > > > > dceccarel...@bloomberg.net> wrote: > > > > > > > > > > I t

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Rob Audenaerde
i (BLOOMBERG/ LONDON) < > > > dceccarel...@bloomberg.net> wrote: > > > > > > > > I think it will be similar ratio retrieving any number of > documents. > > > > > > > > I'm not sure this is true, if you retrieve a huge amount

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Diego Ceccarelli (BLOOMBERG/ LONDON)
content from Lucene you are also dealing with how the operating system decides to load the content in memory. Cheers, Diego From: java-user@lucene.apache.org At: 01/22/21 15:22:32To: java-user@lucene.apache.org Subject: Re: Slower document retrieval in 8.7.0 comparing to 7.5.0 Just played

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Martynas L
gt; wrote: > > > > > > I think it will be similar ratio retrieving any number of documents. > > > > > > I'm not sure this is true, if you retrieve a huge amount of documents > you > > > might cause troubles to the GC. > > > > > > From: ja

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Rob Audenaerde
he GC. > > > > From: java-user@lucene.apache.org At: 01/22/21 12:11:19To: > > java-user@lucene.apache.org > > Subject: Re: Slower document retrieval in 8.7.0 comparing to 7.5.0 > > > > The accent should not be on retrieved documents number, but on the > durati

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Martynas L
ou retrieve a huge amount of documents you > might cause troubles to the GC. > > From: java-user@lucene.apache.org At: 01/22/21 12:11:19To: > java-user@lucene.apache.org > Subject: Re: Slower document retrieval in 8.7.0 comparing to 7.5.0 > > The accent should not be on retrieve

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Diego Ceccarelli (BLOOMBERG/ LONDON)
wer document retrieval in 8.7.0 comparing to 7.5.0 The accent should not be on retrieved documents number, but on the duration ratio - 8.7.0 is 3 times slower. I think it will be similar ratio retrieving any number of documents. On Fri, Jan 22, 2021 at 1:39 PM Rob Audenaerde wrote: > Hi Mar

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Martynas L
The accent should not be on retrieved documents number, but on the duration ratio - 8.7.0 is 3 times slower. I think it will be similar ratio retrieving any number of documents. On Fri, Jan 22, 2021 at 1:39 PM Rob Audenaerde wrote: > Hi Martrynas, > > In your sample code you are retrieving all

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Rob Audenaerde
Hi Martrynas, In your sample code you are retrieving all (1 million!) documents from the index, that surely is not a good match for lucene :) Is that a good reflection of your use-case? On Fri, Jan 22, 2021 at 9:52 AM Martynas L wrote: > Please see the sample at >

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-22 Thread Martynas L
Please see the sample at https://drive.google.com/drive/folders/1ufVZXzkugBAFnuy8HLAY6mbPWzjknrfE IndexGenerator - creates a dummy index. IndexReader - retrieves documents - duration time with 7.5.0 version is ~2s, while ~6s with 8.7.0 Regards, Martynas On Thu, Jan 21, 2021 at 8:21 PM Rob

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-21 Thread Rob Audenaerde
There is no attachment in the previous email that I can see? Maybe you can post it online? On Thu, Jan 21, 2021 at 4:54 PM Martynas L wrote: > Hello, > > Are there any comments on this issue? > If there is no workaround, we will be forced to rollback to the 7.5.0 > version. > > Best regards, >

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-21 Thread Martynas L
Hello, Are there any comments on this issue? If there is no workaround, we will be forced to rollback to the 7.5.0 version. Best regards, Martynas On Tue, Jan 12, 2021 at 12:27 PM Martynas L wrote: > Hi, > > Please see attached sample. > IndexGenerator - creates a dummy index. > IndexReader -

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2021-01-12 Thread Martynas L
Hi, Please see attached sample. IndexGenerator - creates a dummy index. IndexReader - retrieves documents - duration time with 7.5.0 version is ~2s, while ~6s with 8.7.0 Regards, Martynas On Tue, Dec 22, 2020 at 3:23 PM Vincenzo D'Amore wrote: > I think it would be useful to have an example

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2020-12-22 Thread Martynas L
Query is fast, but document retrieval is "slow". We call: 1) IndexSearcher#search(Query, Collector) to collect docIDs, and then 2) retrieve documents with IndexSearcher#doc(int). In our case (1) takes less than 0.5s, while (2) almost 1.5 min (4 times slower than 7.5.0) On Tue, Dec 22, 2020 at

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2020-12-22 Thread Vincenzo D'Amore
I think it would be useful to have an example of a document and, if possible, an example of query that takes too long. On Mon, Dec 21, 2020 at 1:47 PM Martynas L wrote: > Hello, > > I am sorry for the delay. > > Not sure what you mean by "workload". We have a performance tests, which > started

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2020-12-21 Thread Martynas L
Hello, I am sorry for the delay. Not sure what you mean by "workload". We have a performance tests, which started failing after upgrading to 8.7.0. So I just tried to query the index (built form the same source) to get all documents and compare the performance with 7.5.0. Document "size" is a

Re: Slower document retrieval in 8.7.0 comparing to 7.5.0

2020-12-03 Thread Adrien Grand
Hello Martynas, There have indeed been changes related to stored fields in 8.7. What does your workload look like and how large are your documents on average? On Thu, Dec 3, 2020 at 3:04 PM Martynas L wrote: > Hi, > We've migrated from 7.5.0 to 8.7.0 and find out that the index "searching" >