On Sat, Aug 8, 2015 at 7:48 AM, <[email protected]> wrote: >> First, I'd try to find out which call into Lucy takes so long, > > As far as I can see it is the loop that reads the result with $hits->next
All that Hits#next does is read a document from disk. At that point, the search has been completed and we have a list of document IDs; each call to next() takes one of those IDs, finds the location of the data and reads from mmap'd memory to deserialize it into a hash. Performance of next() scales with the size of the document fetched, not with the size of the index (except insofar as index size affects OS cache performance) or the complexity of the search query. Text data in the document store is not compressed. How large are these documents? If they are small, yet each iteration of the $hits->next loop is unacceptably slow, it it hard to understand how the code Lucy is executing could be the source of the slowdown. > It's a production system (it not happens in the test system) !!! > 2.) The second issue is, that from time to time the search time increases > drastically, so it goes up to serveral seconds and more You had mentioned before that restarting the process fixes this issue. How much memory is the process consuming when you restart it? > P.S. The system is a vm with 16GB ram, 6 cores and is running on a SSD and > most times 98% idle, so it should not be a performance issues of the > host itself. What do graphs of memory and IO on this box look like? Marvin Humphrey
