OutOfMemory Problems Lucene 2.4 / Tomcat

[Todd Benge]

Hi,

I'm the lead engineer for search on a large website using lucene for search.

We're indexing about 300M documents in ~ 100 indices.  The indices add
 up to ~ 60G.

The indices are sorted into 4 different Multisearcher with the largest
handling ~50G.

The code is basically like the following:

private static MultiSearcher searcher;

public void init(File files) {

     IndexSearcer [] searchers = new IndexSearcher[files.length] ();
     int i = 0;
     for ( File file: files ) {
          searchers[i++] = new IndexSearcher(FSDirectory.getDirectory(file);
     }

searcher = new MultiSearcher(searchers);
}

public Searcher getSearcher() {
   return searcher;
}

We're seeing a high cache rate with Term & TermInfo in Lucene 2.4.
Performance is good but servers are consistently hanging with
OutOfMemory errors.

We're allocating 4G in the heap to each server.

Is there any way to control the amount of memory Lucene consume for
caching?  Any other suggestions on fixing the memory errors?

Thanks,

Todd

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]