What have you set various indexwriter properties to? Particularly things like merge factor, max buffered docs and ram buffer size.
The first thing I'd look at is MergeFactor. From the JavaDocs: Determines how often segment indices are merged by addDocument(). With smaller values, less RAM is used while indexing, and searches on unoptimized indices are faster, but indexing speed is slower. With larger values, more RAM is used during indexing, and while searches on unoptimized indices are slower, indexing is faster. Thus larger values (> 10) are best for batch index creation, and smaller values (< 10) for indices that are interactively maintained. HTH Erick On Thu, May 27, 2010 at 11:52 AM, Saurabh Agarwal <srbh.g...@gmail.com>wrote: > Hi, > when I am running Lucene on a 512 MB system. > I am getting the following error > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > at > > org.apache.lucene.index.DocumentsWriter$ByteBlockAllocator.getByteBlock(DocumentsWriter.java:1206) > and sometimes > An unexpected error has been detected by Java Runtime Environment: > # Problematic frame: > # J java.nio.ByteBuffer.arrayOffset()I > > > if I decrease my corpus size it indexes it perfectly!!! > > Can someone tell me how to index a larger corpus even if it takes more > time!!! > > Thanks > > Saurabh Agarwal >
