Yes already read this stuff but was not sure 1000 docs was considered as "deep"
or not.
De : Vincenzo D'Amore
Envoyé : mardi 14 janvier 2020 22:22
À : solr-user@lucene.apache.org
Objet : Re: Querying multiple pages for same keyword at same time
Indeed, with a max of 1K doc to be manipulated, I don't expect issues.
We are looking at other avenues to understand our issues.
Regards,
Gaël
Had you already seen Solr deep paging?
https://lucidworks.com/post/coming-soon-to-solr-efficient-cursor-based-iteration-of-large-result-sets/
> On Tue, 14 Jan 2020 at 20:41, Erick Erickson wrote:
> Conceptually asking for cods 900-1000 works something like this. Solr (well,
> Lucene
Conceptually asking for cods 900-1000 works something like this. Solr (well,
Lucene actually) has to keep a sorted list 1,000 items long of scores and doc
IDs because you can’t know whether doc N+1 will be in the list, or where. So
the list manipulation is what takes the extra time. For even
Ok I understand better.
Solr does not "read" the 1 to 900 docs to retrieve 901 to 1000 but it still
needs to compute some stuff (docset intersection or something like that,
right?) and sort, which is costly, and then "read" the docs.
> Are those 10 requests happening simultaneously, or
On 1/13/2020 11:53 AM, Gael Jourdan-Weil wrote:
Just to clarify something, we are not returning 1000 docs per request, we are
only returning 100.
We get 10 requests to Solr querying for docs 1 to 100, then 101 to 200, ...
until 901 to 1000.
But all that in the exact same second.
But I
De : Erick Erickson
Envoyé : lundi 13 janvier 2020 14:44
À : solr-user@lucene.apache.org
Objet : Re: Querying multiple pages for same keyword at same time
To return stored values, Lucene must
1> read the stored values from disk
2> decompress a minimum 16K block
3>
To return stored values, Lucene must
1> read the stored values from disk
2> decompress a minimum 16K block
3> assemble the return packet.
So you’re returning 500-1,000 documents per request, it may just be the above
set of steps. Solr was never designed to _return_ large result sets. Search
