On Fri, 2009-10-23 at 08:49 +0200, Jake Mannix wrote: > One of the big problems you'll run into with this index size is that > you'll never have enough RAM to give your OS's IO cache enough room to keep > much of this index in memory, so you're going to be seeking in this monster > file a lot. [...]
Solid State Drives helps a lot in this aspect. We've done experiments with a 40GB index and adjustments of the amount of RAM available for file cache. We observed that search-speed using SSD's weren't near as susceptible to cache-size as conventional harddisks. Some quick and fairly unstructured notes on our observations: http://wiki.statsbiblioteket.dk/summa/Hardware > [...] > This may be mitigated by using really fast disks, possibly, which is yet > another reason why you'll need to do some performance profiling on a > variety of sizes with similar-to-production data sets. For our setup, a switch from conventional harddisks to SSDs moved the bottleneck from I/O to CPU/RAM. --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
