Thanks all, Tom, your results are interesting. We both have about 5 million documents, but my index is 20 gigs vs. yours 2 TB. I imagine we'll have a much easier time getting quick responses against these small documents compared to your multi-second queries. As for index/search disk contention we're planning to have independent indexing and searching machines, probably following some of the guidelines in this great article, http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr#resources.haproxy .
-Kallin Nagelberg -----Original Message----- From: Burton-West, Tom [mailto:tburt...@umich.edu] Sent: Tuesday, April 27, 2010 6:03 PM To: solr-user@lucene.apache.org Subject: RE: nfs vs sas in production Hi Kallin, Given the previous postings on the list about terrible NFS performance we were pleasantly surprised when we did some tests against a well tuned NFS RAID array on a private network. We got reasonably good results (given our large index sizes.) See http://www.hathitrust.org/blogs/large-scale-search/current-hardware-used-testing and http://www.hathitrust.org/blogs/large-scale-search/tuning-search-performance. Just prior to going into production we moved from direct attached storage to a very high performance NAS in production for a number of reasons including ease of management as we scale out. One of the reasons was to reduce contention between indexing/optimizing and search instances for disk I/O. See http://www.hathitrust.org/blogs/large-scale-search/scaling-large-scale-search-500000-volumes-5-million-volumes-and-beyond for details. Tom -----Original Message----- From: Nagelberg, Kallin [mailto:knagelb...@globeandmail.com] Sent: Tuesday, April 27, 2010 4:13 PM To: 'solr-user@lucene.apache.org' Subject: nfs vs sas in production Hey, A question was raised during a meeting about our new Solr based search projects. We're getting 4 cutting edge servers each with something like 24 Gigs of ram dedicated to search. However there is some problem with the amount of SAS based storage each machine can handle, and people wonder if we might have to use a NFS based drive instead. Does anyone have any experience using SAS vs. NFS drives for Solr? Any feedback would be appreciated! Thanks, -Kallin Nagelberg
