Let me add to that that I clearly recall having a hard time getting the tests for that particular section of LIA1 to clearly and consistently show that using the RAMDirectory buffering approach instead of vanilla IndexWriter yields faster indexing. Even back then IndexWriter buffered indexed data in memory, though today's IndexWriter is much, much better at it.
Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Michael McCandless <luc...@mikemccandless.com> > To: java-user@lucene.apache.org > Sent: Saturday, December 20, 2008 4:25:13 AM > Subject: Re: Default and optimal use of RAMDirectory > > Actually, things have improved since LIA1 was written a few years ago: > IndexWriter now does a good job managing the RAM buffer you assign to > it, so you should not see much benefit by doing your own buffering > with RAMDirectory (and if you somehow do, I'd like to know about > it!). > > Instead you should call IndexWriter.setRAMBufferSizeMB. > > Also, FSDirectory does no RAM buffering on its own. > > See here for further ways to tune for indexing throughput: > > http://wiki.apache.org/lucene-java/ImproveIndexingSpeed > > Mike > > wrote: > > > > > Hi all, > > > > First of I'd like to say I'm quite pleased to be a part of this mailing > > list - its even more exciting to know that we have Otis G. and Erik H., > > authors of (at least in my opinion) the Lucene Bible - Lucene in Action, > > actively answering all these inquiries =) > > > > We're currently in the initial stages of implementing lucene as part of our > > product and one problem that we need to resolve is optimizing lucene. I've > > been reading Lucene in Action book and one of the tips for optimizing > > lucene indexing is by using RAMDirectory as a buffer before writing to > > FSDirectory. According to the book, this is done internally and > > automatically when I use FSDirectory. My questions are 1.) What's the > > default implementation/ computation used in allocating RAMdirectory when we > > implement FSDirectory and 2.) What's the optimal way of customizing > > RAMDirectory usage - any tips on how to do it. > > > > BTW, we're using Lucene 2.3.2 > > > > Thanks for all the help > > > > Joseph > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
