> On 5 Sep 2023, at 16:17, Michael Gibney <mich...@michaelgibney.net> wrote: > >> Note: for a test search that retrieves only 10 documents, qtime is very low >> (2 msec) but the full request time to get javabin or json data is very slow >> (several seconds). > > Reading between the lines here: does "full request" return a larger > number of documents? How many? Are you attempting to return values for > the fields (either implicitly or by glob)? Are you certain that the > fields are all `stored=true`? If you're calculating stats on them, > they are presumably already `docValues=true`. Do all of the docs > contain all of the fields (i.e., you have 5k fields -- does each > individual doc have all 5k fields, or a only a subset of the total > number of fields)? > > By any chance, is "QTime" (as reported in solr logs and response body) > appear substantially less than the actual amount of time it takes to > receive the response in the client?
Hello and thanks for the reply. I think I found the issue is that having such many fields and returning all of them (fl=*) causes a lot of data to be generated and so it becomes very slow. So it’s exactly as you guessed :) We’re doing two fixes to our process : 1. we will not store zero values to store less data 2. we will use fl to get the fields we need in each part of the process to consume only the minimum data required. Thanks — Ing. Andrea Vettori Sistemi Informativi B2BIres s.r.l.
