On 6/30/2015 6:23 AM, Erick Erickson wrote: > I've actually seen this happen right in front of my eyes "in the > field". However, that was a very high-performance environment. My > assumption was that fragmented index files were causing more disk > seeks especially for the first-pass query response in distributed > mode. So, if the problem is similar, it should go away if you test > requesting fewer docs. Note: This is not a cure for your problem, but > would be useful for identifying if it's similar to what I saw. > > NOTE: the symptom was a significant disparity between the QTime (which > does not measure assembling the document) and the response time. _If_ > that's the case and _if_ my theory that disk access is the culprit, > then SOLR-5478 and SOLR-6810 should be a big help as they remove the > first-pass decompression for distributed searches. > > If that hypothesis has any validity, I'd expect you're running on > spinning-disks rather than SSDs, is that so?
If the index is small enough that all or most of it will fit into the OS disk cache, then the problem of disk seeks would largely disappear, because all or most data would be read from RAM, not the disk. If the index is WAY too big to fit into available RAM (think about terabyte scale indexes), then I think SSD is probably the only way to get performance levels that could be called good. Thanks, Shawn
