> Running Lucene IndexSearcher in JVM with -Xms3072m -Xmx3072m (more memory > than that then the system kills the JVM).
I'd try and fix that first. Use a different JVM? No point in buying more memory if you can't even use all you've got now. What version of lucene are you using? http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html implies you might be better off with a bleeding edge version. Or maybe you need to look at splitting your index. -- Ian. On Thu, Aug 4, 2011 at 10:41 PM, Trieu, Jason T <trieu.ja...@con-way.com> wrote: > Hello, > > I keep getting java.lang.OutOfMemoryError while doing search and would like > to get some ideas to resolve the issue.. > > BACKGROUND: > > IBM AIX server with 8 GIG of memory. > Lucene index contains 625 million documents with each having roughly 200 > bytes of stored fields/data. > Running Lucene IndexSearcher in JVM with -Xms3072m -Xmx3072m (more memory > than that then the system kills the JVM). > > PROBLEM: > > I can search successfully search the index if I do not user sort or > filtering. Although I need to invoke IndexReader with a > termInfosIndexDivisor of at least 100 > (IndexReader.open(dir,NoDeletionPolicy.INSTANCE,true,100) > If I include a sort ( of a numeric field) or a numeric range filter, I get > OutOfMemoryError no matter how big the termInfosIndexDivisor is. I've tried > it as big as 100,000,000 and still got OutOfMemoryError. > It seems I am running against hard limits of resources. Perhaps 8 GB of > memory is just simply not enough to handle an index of 600 million documents. > But before telling management that they must get more memory, I'd to see if > there might be other ways to accomplish this. > Thanks in advance. > > Jason > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
