Hey, I've recently noticed that there is a very large spike in the QTime for nodes serving queries, immediately after snappulling and snapinstalling. The numbers i'm seeing there are obviously some kind of lock-contention/concurrency issue, as I've monitored iostat/sar and its not a disk IO issue ( all the index is still mostly in the os caches, as is also noticeable in the snappuller.log which runs very fast - the incremental update to the lucene index is minimal). I'm using a strong machine (16GB 2xQuadCore, CentOS5) for this and from all monitoring the CPU/Disk IO seems minimal through-out the day. The update runs every 10 minutes, takes about 200seconds to complete (with my current autoWarm settings)., and the index size is about 20million documents (~6GB). queries/warmup involve mostly some pre-fixed set of filters and facets.
I'm using a nightly build of solr from few months back ( nightly exported - yonik - 2009-01-11 08:05:52 ) which should already have the benefits of readonlyindexreader, NIOFS, UnInvertedIndex and ConcurrentLRUCache (for filterCache). I've read a related thread a guy called oleg posted ( http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200901.mbox/%3c339777.57289...@web50307.mail.re2.yahoo.com%3e) , but did not see the thread concluded with any definite conclusion. Please advise! Best regards, -Chak
