2007/4/3, Chun Wei Ho <[EMAIL PROTECTED]>:
As the index have been growing in size, we have been noticing that the
search response time on the searcher machine increases drastically
when an index (about 15GB) is being copied from the crawler to the
searcher. Both machines run Fedora Core 4 and are on a gbps lan.
We've tried a number of ways to reduce the impact of the copy over NFS
on searching performance, such as "nice"ing the copy process, but to
no avail. I wonder if anyone is running a lucene search service over a
similar architecture and how you are managing the updates to the
lucene index.
Have you tried limiting the transfer rate? If you don't mind the
additional index refresh latency, it might be a simple way to give
more CPU and disk IO time to the search service...you could pull (wget
--limit-rate=1M index) instead of push (cp src dest) the index and in
doing so, get the added flexibility of controling the copy rate.
Cheers,
t.n.a.
