Stealing this thread/idea, but changing subject, so we can branch and I don't look like a thread thief.
I never played with /proc/sys/vm/swappiness, but I wonder if there are points in the lifetime of an index where this number should be changed. For example, does it make sense to in/decrease that number once we know the index is going to be read-only for a while? Does it make sense to in/decrease it during merges or optimizations? Thanks, Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Michael McCandless <luc...@mikemccandless.com> > To: java-user@lucene.apache.org > Sent: Wednesday, June 24, 2009 5:06:25 AM > Subject: Re: Analyzing performance and memory consumption for boolean queries > > Is it possible the occasional large merge is clearing out the IO cache > (thus "unwarming" your searcher)? (Though since you're rsync'ing your > updates in, it sounds like a separate machine is building the index). > > Or... linux will happily swap out a process's core in favor of IO > cache (though I'd expect this effect to be much less spikey). You can > tune "swappiness" to have it not do that: > > http://kerneltrap.org/node/3000 > > Maybe Lucene's norms/deleted docs/field cache were getting swapped out? > > Lucene's postings reside entirely on disk (ie, Lucene doesn't cache > those in RAM; we rely on the OS's IO cache). Lucene does a linear > scan through the terms in the query... Linux will readahead, though, > if things are fragmented this could mean lots of seeking. --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
