You don't get a heap-related OOM in your stack trace, it is "Map failed" - caused by MMapDirectory. You don't have enough virtual memory to map the index into address space. I think your heap is way too mch (-Xmx25g is way too big for any existing index and drives GC crazy). How big is your index?
See http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Bernd Fehling [mailto:bernd.fehl...@uni-bielefeld.de] > Sent: Wednesday, August 15, 2012 1:25 PM > To: java-user@lucene.apache.org > Subject: howto run CheckIndex on huge index size > > > I'm trying to run CheckIndex as seperate tool on a large index to get nice infos > about number of terms, number of tokens, ... but always get OOM exception. > > Already have JAVA_OPTS -d64 -Xmx25g -Xms25g -Xmn6g > > > Any idea how to use CheckIndex on huge index size? > > > > Opening index @ /srv/www/solr/solr/solrserver/solr/data/index > Segments file=segments_5u numSegments=1 version=3.6.1 > format=FORMAT_3_1 [Lucene 3.1+] > 1 of 1: name=_2rhe docCount=37821132 > compound=false > hasProx=true > numFiles=8 > size (MB)=182.992,35 > diagnostics = {mergeFactor=1, os.version=2.6.32.23-0.3-xen, os=Linux, > lucene.version=3.6.1-SNAPSHOT 1325634 - solr - 2012-04-13 14:08:0 6, > source=merge, os.arch=amd64, mergeMaxNumSegments=1, > java.version=1.6.0_29, java.vendor=Sun Microsystems Inc.} > no deletions > test: open reader.........FAILED > WARNING: fixIndex() would remove reference to this segment; full > exception: > java.io.IOException: Map failed > at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748) > at > org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirect > ory.java:271) > at > org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:221) > at > org.apache.lucene.index.TermInfosReader.<init>(TermInfosReader.java:104) > at > org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.jav > a:83) > at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116) > at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:94) > at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:523) > at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:1064) > Caused by: java.lang.OutOfMemoryError: Map failed > at sun.nio.ch.FileChannelImpl.map0(Native Method) > at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745) > ... 8 more > WARNING: 1 broken segments (containing 37821132 documents) detected > WARNING: would write new segments file, and 37821132 documents would be > lost, if -fix were specified > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
