On Thu, 2015-08-06 at 14:32 +0200, Bernd Fehling wrote: > It "feels" like this behavior showed up after changing to docValues for > sorting, > because before the 99 percentile for qtime was at 550ms average and 1.4 > seconds at max.
DocValues have faster startup but comes with a constant performance cost as they have to be fetched through the memory map. Judged from what people are observing & sharing, that cost is "small". Field faceting (docValues or not) takes more time for larger result sets. If your queries are anywhere like ours, the vast majority of them will hit a fairly small subset of your whole corpus. Your 99% might contain only queries with hit count < 10% of your total document count (hand waving a lot here). When we are talking faceting, I find generating a plot from the production log, with hit count on x and response time on y, very illuminating. > So my assumption is that the inverted index on the sort fields (when _not_ > using > docValues) helps here with the disadvantage of using a lot of heap :-( Yes. But I highly doubt that it explains a factor 30 in speed. More like a factor 2. > Do you think an all_parameters -> complete_response cache is possible? > It could be initialized right before or during warmup and would not take to > much memory. Sorry, I don't know much of the mechanics of handlers in Solr and cannot say how the in-theory-simple caching would fit. - Toke Eskildsen, State and University Library, Denmark
