Hi, Also came across this error recently, while indexing with > 10 DIH processes in parallel + default index setting. The JVM grinds to a halt and throws this error. Checking the index of a core reveals thousands of files! Tuning the default autocommit from 15000ms to 900000ms solved the problem for us. (no 'autosoftcommit').
Boon ----- Boon Low Search UX and Engine Developer brightsolid Online Publishing On 14 Apr 2012, at 17:40, Gopal Patwa wrote: > I checked it was "MMapDirectory.UNMAP_SUPPORTED=true" and below are my > system data. Is their any existing test case to reproduce this issue? I am > trying understand how I can reproduce this issue with unit/integration test > > I will try recent solr trunk build too, if it is some bug in solr or > lucene keeping old searcher open then how to reproduce it? > > SYSTEM DATA > =========== > PROCESSOR: Intel(R) Xeon(R) CPU E5504 @ 2.00GHz > SYSTEM ID: x86_64 > CURRENT CPU SPEED: 1600.000 MHz > CPUS: 8 processor(s) > MEMORY: 49449296 kB > DISTRIBUTION: CentOS release 5.3 (Final) > KERNEL NAME: 2.6.18-128.el5 > UPTIME: up 71 days > LOAD AVERAGE: 1.42, 1.45, 1.53 > JBOSS Version: Implementation-Version: 4.2.2.GA (build: > SVNTag=JBoss_4_2_2_GA date=20 > JAVA Version: java version "1.6.0_24" > > > On Thu, Apr 12, 2012 at 3:07 AM, Michael McCandless < > luc...@mikemccandless.com> wrote: > >> Your largest index has 66 segments (690 files) ... biggish but not >> insane. With 64K maps you should be able to have ~47 searchers open >> on each core. >> >> Enabling compound file format (not the opposite!) will mean fewer maps >> ... ie should improve this situation. >> >> I don't understand why Solr defaults to compound file off... that >> seems dangerous. >> >> Really we need a Solr dev here... to answer "how long is a stale >> searcher kept open". Is it somehow possible 46 old searchers are >> being left open...? >> >> I don't see any other reason why you'd run out of maps. Hmm, unless >> MMapDirectory didn't think it could safely invoke unmap in your JVM. >> Which exact JVM are you using? If you can print the >> MMapDirectory.UNMAP_SUPPORTED constant, we'd know for sure. >> >> Yes, switching away from MMapDir will sidestep the "too many maps" >> issue, however, 1) MMapDir has better perf than NIOFSDir, and 2) if >> there really is a leak here (Solr not closing the old searchers or a >> Lucene bug or something...) then you'll eventually run out of file >> descriptors (ie, same problem, different manifestation). >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> 2012/4/11 Gopal Patwa <gopalpa...@gmail.com>: >>> >>> I have not change the mergefactor, it was 10. Compound index file is >> disable >>> in my config but I read from below post, that some one had similar issue >> and >>> it was resolved by switching from compound index file format to >> non-compound >>> index file. >>> >>> and some folks resolved by "changing lucene code to disable >> MMapDirectory." >>> Is this best practice to do, if so is this can be done in configuration? >>> >>> >> http://lucene.472066.n3.nabble.com/MMapDirectory-failed-to-map-a-23G-compound-index-segment-td3317208.html >>> >>> I have index document of core1 = 5 million, core2=8million and >>> core3=3million and all index are hosted in single Solr instance >>> >>> I am going to use Solr for our site StubHub.com, see attached "ls -l" >> list >>> of index files for all core >>> >>> SolrConfig.xml: >>> >>> >>> <indexDefaults> >>> <useCompoundFile>false</useCompoundFile> >>> <mergeFactor>10</mergeFactor> >>> <maxMergeDocs>2147483647</maxMergeDocs> >>> <maxFieldLength>10000</maxFieldLength--> >>> <ramBufferSizeMB>4096</ramBufferSizeMB> >>> <maxThreadStates>10</maxThreadStates> >>> <writeLockTimeout>1000</writeLockTimeout> >>> <commitLockTimeout>10000</commitLockTimeout> >>> <lockType>single</lockType> >>> >>> <mergePolicy class="org.apache.lucene.index.TieredMergePolicy"> >>> <double name="forceMergeDeletesPctAllowed">0.0</double> >>> <double name="reclaimDeletesWeight">10.0</double> >>> </mergePolicy> >>> >>> <deletionPolicy class="solr.SolrDeletionPolicy"> >>> <str name="keepOptimizedOnly">false</str> >>> <str name="maxCommitsToKeep">0</str> >>> </deletionPolicy> >>> >>> </indexDefaults> >>> >>> >>> <updateHandler class="solr.DirectUpdateHandler2"> >>> <maxPendingDeletes>1000</maxPendingDeletes> >>> <autoCommit> >>> <maxTime>900000</maxTime> >>> <openSearcher>false</openSearcher> >>> </autoCommit> >>> <autoSoftCommit> >>> >> <maxTime>${inventory.solr.softcommit.duration:1000}</maxTime> >>> </autoSoftCommit> >>> >>> </updateHandler> >>> >>> >>> Forwarded conversation >>> Subject: Large Index and OutOfMemoryError: Map failed >>> ------------------------ >>> >>> From: Gopal Patwa <gopalpa...@gmail.com> >>> Date: Fri, Mar 30, 2012 at 10:26 PM >>> To: solr-user@lucene.apache.org >>> >>> >>> I need help!! >>> >>> >>> >>> >>> >>> I am using Solr 4.0 nightly build with NRT and I often get this error >> during >>> auto commit "java.lang.OutOfMemoryError: Map failed". I have search this >>> forum and what I found it is related to OS ulimit setting, please se >> below >>> my ulimit settings. I am not sure what ulimit setting I should have? and >> we >>> also get "java.net.SocketException: Too many open files" NOT sure how >> many >>> open file we need to set? >>> >>> >>> I have 3 core with index size : core1 - 70GB, Core2 - 50GB and Core3 - >> 15GB, >>> with Single shard >>> >>> >>> We update the index every 5 seconds, soft commit every 1 second and hard >>> commit every 15 minutes >>> >>> >>> Environment: Jboss 4.2, JDK 1.6 , CentOS, JVM Heap Size = 24GB >>> >>> >>> ulimit: >>> >>> core file size (blocks, -c) 0 >>> data seg size (kbytes, -d) unlimited >>> scheduling priority (-e) 0 >>> file size (blocks, -f) unlimited >>> pending signals (-i) 401408 >>> max locked memory (kbytes, -l) 1024 >>> max memory size (kbytes, -m) unlimited >>> open files (-n) 1024 >>> pipe size (512 bytes, -p) 8 >>> POSIX message queues (bytes, -q) 819200 >>> real-time priority (-r) 0 >>> stack size (kbytes, -s) 10240 >>> cpu time (seconds, -t) unlimited >>> max user processes (-u) 401408 >>> virtual memory (kbytes, -v) unlimited >>> file locks (-x) unlimited >>> >>> >>> >>> ERROR: >>> >>> >>> >>> >>> >>> 2012-03-29 15:14:08,560 [] priority=ERROR app_name= >> thread=pool-3-thread-1 >>> location=CommitTracker line=93 auto commit error...:java.io.IOException: >> Map >>> failed >>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748) >>> at >>> >> org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:293) >>> at >> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:221) >>> at >>> >> org.apache.lucene.codecs.lucene40.Lucene40PostingsReader.<init>(Lucene40PostingsReader.java:58) >>> at >>> >> org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat.fieldsProducer(Lucene40PostingsFormat.java:80) >>> at >>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.visitOneFormat(PerFieldPostingsFormat.java:189) >>> at >>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.<init>(PerFieldPostingsFormat.java:280) >>> at >>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.<init>(PerFieldPostingsFormat.java:186) >>> at >>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:186) >>> at >>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:256) >>> at >>> >> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:108) >>> at >> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:51) >>> at >>> >> org.apache.lucene.index.IndexWriter$ReadersAndLiveDocs.getReader(IndexWriter.java:494) >>> at >>> >> org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:214) >>> at >>> >> org.apache.lucene.index.IndexWriter.applyAllDeletes(IndexWriter.java:2939) >>> at >>> >> org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:2930) >>> at >> org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2681) >>> at >>> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2804) >>> at >> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2786) >>> at >>> >> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:391) >>> at org.apache.solr.update.CommitTracker.run(CommitTracker.java:197) >>> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>> at >>> >>> ... >>> >>> [Message clipped] >>> ---------- >>> From: Michael McCandless <luc...@mikemccandless.com> >>> Date: Sat, Mar 31, 2012 at 3:15 AM >>> To: solr-user@lucene.apache.org >>> >>> >>> It's the virtual memory limit that matters; yours says unlimited below >>> (good!), but, are you certain that's really the limit your Solr >>> process runs with? >>> >>> On Linux, there is also a per-process map count: >>> >>> cat /proc/sys/vm/max_map_count >>> >>> I think it typically defaults to 65,536 but you should check on your >>> env. If a process tries to map more than this many regions, you'll >>> hit that exception. >>> >>> I think you can: >>> >>> cat /proc/<pid>/maps | wc >>> >>> to see how many maps your Solr process currently has... if that is >>> anywhere near the limit then it could be the cause. >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> On Sat, Mar 31, 2012 at 1:26 AM, Gopal Patwa <gopalpa...@gmail.com> >> wrote: >>>> *I need help!!* >>>> >>>> * >>>> * >>>> >>>> *I am using Solr 4.0 nightly build with NRT and I often get this error >>>> during auto commit "**java.lang.OutOfMemoryError:* *Map* *failed". I >>>> have search this forum and what I found it is related to OS ulimit >>>> setting, please se below my ulimit settings. I am not sure what ulimit >>>> setting I should have? and we also get "**java.net.SocketException:* >>>> *Too* *many* *open* *files" NOT sure how many open file we need to >>>> set?* >>>> >>>> >>>> I have 3 core with index size : core1 - 70GB, Core2 - 50GB and Core3 - >>>> 15GB, with Single shard >>>> >>>> * >>>> * >>>> >>>> *We update the index every 5 seconds, soft commit every 1 second and >>>> hard commit every 15 minutes* >>>> >>>> * >>>> * >>>> >>>> *Environment: Jboss 4.2, JDK 1.6 , CentOS, JVM Heap Size = 24GB* >>>> >>>> * >>>> * >>>> >>>> ulimit: >>>> >>>> core file size (blocks, -c) 0 >>>> data seg size (kbytes, -d) unlimited >>>> scheduling priority (-e) 0 >>>> file size (blocks, -f) unlimited >>>> pending signals (-i) 401408 >>>> max locked memory (kbytes, -l) 1024 >>>> max memory size (kbytes, -m) unlimited >>>> open files (-n) 1024 >>>> pipe size (512 bytes, -p) 8 >>>> POSIX message queues (bytes, -q) 819200 >>>> real-time priority (-r) 0 >>>> stack size (kbytes, -s) 10240 >>>> cpu time (seconds, -t) unlimited >>>> max user processes (-u) 401408 >>>> virtual memory (kbytes, -v) unlimited >>>> file locks (-x) unlimited >>>> >>>> >>>> * >>>> * >>>> >>>> *ERROR:* >>>> >>>> * >>>> * >>>> >>>> *2012-03-29* *15:14:08*,*560* [] *priority=ERROR* *app_name=* >>>> *thread=pool-3-thread-1* *location=CommitTracker* *line=93* *auto* >>>> *commit* *error...:java.io.IOException:* *Map* *failed* >>>> *at* *sun.nio.ch.FileChannelImpl.map*(*FileChannelImpl.java:748*) >>>> *at* >>>> >> *org.apache.lucene.store.MMapDirectory$MMapIndexInput.*<*init*>(*MMapDirectory.java:293*) >>>> *at* >>>> >> *org.apache.lucene.store.MMapDirectory.openInput*(*MMapDirectory.java:221*) >>>> *at* >>>> >> *org.apache.lucene.codecs.lucene40.Lucene40PostingsReader.*<*init*>(*Lucene40PostingsReader.java:58*) >>>> *at* >>>> >> *org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat.fieldsProducer*(*Lucene40PostingsFormat.java:80*) >>>> *at* >>>> >> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.visitOneFormat*(*PerFieldPostingsFormat.java:189*) >>>> *at* >>>> >> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.*<*init*>(*PerFieldPostingsFormat.java:280*) >>>> *at* >>>> >> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.*<*init*>(*PerFieldPostingsFormat.java:186*) >>>> *at* >>>> >> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.*<*init*>(*PerFieldPostingsFormat.java:186*) >>>> *at* >>>> >> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer*(*PerFieldPostingsFormat.java:256*) >>>> *at* >>>> >> *org.apache.lucene.index.SegmentCoreReaders.*<*init*>(*SegmentCoreReaders.java:108*) >>>> *at* >>>> >> *org.apache.lucene.index.SegmentReader.*<*init*>(*SegmentReader.java:51*) >>>> *at* >>>> >> *org.apache.lucene.index.IndexWriter$ReadersAndLiveDocs.getReader*(*IndexWriter.java:494*) >>>> *at* >>>> >> *org.apache.lucene.index.BufferedDeletesStream.applyDeletes*(*BufferedDeletesStream.java:214*) >>>> *at* >>>> >> *org.apache.lucene.index.IndexWriter.applyAllDeletes*(*IndexWriter.java:2939*) >>>> *at* >>>> >> *org.apache.lucene.index.IndexWriter.maybeApplyDeletes*(*IndexWriter.java:2930*) >>>> *at* >>>> >> *org.apache.lucene.index.IndexWriter.prepareCommit*(*IndexWriter.java:2681*) >>>> *at* >>>> >> *org.apache.lucene.index.IndexWriter.commitInternal*(*IndexWriter.java:2804*) >>>> *at* >>>> *org.apache.lucene.index.IndexWriter.commit*(*IndexWriter.java:2786*) >>>> *at* >>>> >> *org.apache.solr.update.DirectUpdateHandler2.commit*(*DirectUpdateHandler2.java:391*) >>>> *at* >>>> *org.apache.solr.update.CommitTracker.run*(*CommitTracker.java:197*) >>>> *at* >>>> >> *java.util.concurrent.Executors$RunnableAdapter.call*(*Executors.java:441*) >>>> *at* >>>> *java.util.concurrent.FutureTask$Sync.innerRun*(*FutureTask.java:303*) >>>> *at* *java.util.concurrent.FutureTask.run*(*FutureTask.java:138*) >>>> *at* >>>> >> *java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301*(*ScheduledThreadPoolExecutor.java:98*) >>>> *at* >>>> >> *java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run*(*ScheduledThreadPoolExecutor.java:206*) >>>> *at* >>>> >> *java.util.concurrent.ThreadPoolExecutor$Worker.runTask*(*ThreadPoolExecutor.java:886*) >>>> *at* >>>> >> *java.util.concurrent.ThreadPoolExecutor$Worker.run*(*ThreadPoolExecutor.java:908*) >>>> *at* *java.lang.Thread.run*(*Thread.java:662*)*Caused* *by:* >>>> *java.lang.OutOfMemoryError:* *Map* *failed* >>>> *at* *sun.nio.ch.FileChannelImpl.map0*(*Native* *Method*) >>>> *at* *sun.nio.ch.FileChannelImpl.map*(*FileChannelImpl.java:745*) >>>> *...* *28* *more* >>>> >>>> * >>>> * >>>> >>>> * >>>> * >>>> >>>> * >>>> >>>> >>>> SolrConfig.xml: >>>> >>>> >>>> <indexDefaults> >>>> <useCompoundFile>false</useCompoundFile> >>>> <mergeFactor>10</mergeFactor> >>>> <maxMergeDocs>2147483647</maxMergeDocs> >>>> <maxFieldLength>10000</maxFieldLength--> >>>> <ramBufferSizeMB>4096</ramBufferSizeMB> >>>> <maxThreadStates>10</maxThreadStates> >>>> <writeLockTimeout>1000</writeLockTimeout> >>>> <commitLockTimeout>10000</commitLockTimeout> >>>> <lockType>single</lockType> >>>> >>>> <mergePolicy >> class="org.apache.lucene.index.TieredMergePolicy"> >>>> <double name="forceMergeDeletesPctAllowed">0.0</double> >>>> <double name="reclaimDeletesWeight">10.0</double> >>>> </mergePolicy> >>>> >>>> <deletionPolicy class="solr.SolrDeletionPolicy"> >>>> <str name="keepOptimizedOnly">false</str> >>>> <str name="maxCommitsToKeep">0</str> >>>> </deletionPolicy> >>>> >>>> </indexDefaults> >>>> >>>> >>>> <updateHandler class="solr.DirectUpdateHandler2"> >>>> <maxPendingDeletes>1000</maxPendingDeletes> >>>> <autoCommit> >>>> <maxTime>900000</maxTime> >>>> <openSearcher>false</openSearcher> >>>> </autoCommit> >>>> <autoSoftCommit> >>>> >>>> <maxTime>${inventory.solr.softcommit.duration:1000}</maxTime> >>>> </autoSoftCommit> >>>> >>>> </updateHandler> >>>> >>>> >>>> >>>> Thanks >>>> Gopal Patwa >>>> * >>> >>> ---------- >>> From: Gopal Patwa <gopalpa...@gmail.com> >>> Date: Tue, Apr 10, 2012 at 8:35 PM >>> To: solr-user@lucene.apache.org >>> >>> >>> Michael, Thanks for response >>> >>> it was 65K as you mention the default value for "cat >>> /proc/sys/vm/max_map_count" . How we determine what value this should be? >>> is it number of document during hard commit in my case it is 15 >> minutes? or >>> it is number of index file or number of documents we have in all cores. >>> >>> I have raised the number to 140K but I still get when it reaches to >> 140K, we >>> have to restart jboss server to free up the map count, sometime OOM error >>> happen during "Error opening new searcher" >>> >>> is making this number to unlimited is only solution?'' >>> >>> >>> Error log: >>> >>> location=CommitTracker line=93 auto commit >>> error...:org.apache.solr.common.SolrException: Error opening new searcher >>> at >> org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1138) >>> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1251) >>> at >>> >> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:409) >>> at org.apache.solr.update.CommitTracker.run(CommitTracker.java:197) >>> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>> at >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>> at >>> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98) >>> at >>> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206) >>> at >>> >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> Caused by: java.io.IOException: Map failed >>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748) >>> at >>> >> org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:293) >>> at >> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:221) >>> at >>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.<init>(PerFieldPostingsFormat.java:262) >>> at >>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$1.<init>(PerFieldPostingsFormat.java:316) >>> at >>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.files(PerFieldPostingsFormat.java:316) >>> at org.apache.lucene.codecs.Codec.files(Codec.java:56) >>> at org.apache.lucene.index.SegmentInfo.files(SegmentInfo.java:423) >>> at >> org.apache.lucene.index.SegmentInfo.sizeInBytes(SegmentInfo.java:215) >>> at >>> >> org.apache.lucene.index.IndexWriter.prepareFlushedSegment(IndexWriter.java:2220) >>> at >>> >> org.apache.lucene.index.DocumentsWriter.publishFlushedSegment(DocumentsWriter.java:497) >>> at >>> >> org.apache.lucene.index.DocumentsWriter.finishFlush(DocumentsWriter.java:477) >>> at >>> >> org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:201) >>> at >>> >> org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:119) >>> at >>> >> org.apache.lucene.index.DocumentsWriterFlushQueue.tryPurge(DocumentsWriterFlushQueue.java:148) >>> at org.apache.lucene.index. >>> >>> ... >>> >>> [Message clipped] >>> ---------- >>> From: Michael McCandless <luc...@mikemccandless.com> >>> Date: Wed, Apr 11, 2012 at 2:20 AM >>> To: solr-user@lucene.apache.org >>> >>> >>> Hi, >>> >>> 65K is already a very large number and should have been sufficient... >>> >>> However: have you increased the merge factor? Doing so increases the >>> open files (maps) required. >>> >>> Have you disabled compound file format? (Hmmm: I think Solr does so >>> by default... which is dangerous). Maybe try enabling compound file >>> format? >>> >>> Can you "ls -l" your index dir and post the results? >>> >>> It's also possible Solr isn't closing the old searchers quickly enough >>> ... I don't know the details on when Solr closes old searchers... >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> >>> >>> On Tue, Apr 10, 2012 at 11:35 PM, Gopal Patwa <gopalpa...@gmail.com> >> wrote: >>>> Michael, Thanks for response >>>> >>>> it was 65K as you mention the default value for "cat >>>> /proc/sys/vm/max_map_count" . How we determine what value this should >> be? >>>> is it number of document during hard commit in my case it is 15 >> minutes? >>>> or it is number of index file or number of documents we have in all >>>> cores. >>>> >>>> I have raised the number to 140K but I still get when it reaches to >> 140K, >>>> we have to restart jboss server to free up the map count, sometime OOM >>>> error happen during "*Error opening new searcher"* >>>> >>>> is making this number to unlimited is only solution?'' >>>> >>>> >>>> Error log: >>>> >>>> *location=CommitTracker line=93 auto commit >>>> error...:org.apache.solr.common.SolrException: Error opening new >>>> searcher >>>> at >>>> org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1138) >>>> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1251) >>>> at >>>> >> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:409) >>>> at >> org.apache.solr.update.CommitTracker.run(CommitTracker.java:197) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>> at >>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>> at >>>> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98) >>>> at >>>> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206) >>>> at >>>> >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>> at >>>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>> at java.lang.Thread.run(Thread.java:662)Caused by: >>>> java.io.IOException: Map failed >>>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748) >>>> at >>>> >> org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:293) >>>> at >>>> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:221) >>>> at >>>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.<init>(PerFieldPostingsFormat.java:262) >>>> at >>>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$1.<init>(PerFieldPostingsFormat.java:316) >>>> at >>>> >> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.files(PerFieldPostingsFormat.java:316) >>>> at org.apache.lucene.codecs.Codec.files(Codec.java:56) >>>> at >> org.apache.lucene.index.SegmentInfo.files(SegmentInfo.java:423) >>>> at >>>> org.apache.lucene.index.SegmentInfo.sizeInBytes(SegmentInfo.java:215) >>>> at >>>> >> org.apache.lucene.index.IndexWriter.prepareFlushedSegment(IndexWriter.java:2220) >>>> at >>>> >> org.apache.lucene.index.DocumentsWriter.publishFlushedSegment(DocumentsWriter.java:497) >>>> at >>>> >> org.apache.lucene.index.DocumentsWriter.finishFlush(DocumentsWriter.java:477) >>>> at >>>> >> org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:201) >>>> at >>>> >> org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:119) >>>> at >>>> >> org.apache.lucene.index.DocumentsWriterFlushQueue.tryPurge(DocumentsWriterFlushQueue.java:148) >>>> at >>>> >> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:438) >>>> at >>>> >> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:553) >>>> at >>>> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:354) >>>> at >>>> >> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:258) >>>> at >>>> >> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:243) >>>> at >>>> >> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:250) >>>> at >>>> org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1091) >>>> ... 11 moreCaused by: java.lang.OutOfMemoryError: Map failed >>>> at sun.nio.ch.FileChannelImpl.map0(Native Method) >>>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)* >>>> >>>> >>>> >>>> And one more issue we came across i.e >>> >>> >> > > > ______________________________________________________________________ > This email has been scanned by the brightsolid Email Security System. Powered > by MessageLabs > ______________________________________________________________________ ______________________________________________________________________ "brightsolid" is used in this email to collectively mean brightsolid online innovation limited and its subsidiary companies brightsolid online publishing limited and brightsolid online technology limited. findmypast.co.uk is a brand of brightsolid online publishing limited. brightsolid online innovation limited, Gateway House, Luna Place, Dundee Technology Park, Dundee DD2 1TP. Registered in Scotland No. SC274983. brightsolid online publishing limited, The Glebe, 6 Chapel Place, Rivington Street, London EC2A 3DQ. Registered in England No. 04369607. brightsolid online technology limited, Gateway House, Luna Place, Dundee Technology Park, Dundee DD2 1TP. Registered in Scotland No. SC161678. Email Disclaimer This message is confidential and may contain privileged information. You should not disclose its contents to any other person. If you are not the intended recipient, please notify the sender named above immediately. It is expressly declared that this e-mail does not constitute nor form part of a contract or unilateral obligation. Opinions, conclusions and other information in this message that do not relate to the official business of brightsolid shall be understood as neither given nor endorsed by it. ______________________________________________________________________ This email has been scanned by the brightsolid Email Security System. Powered by MessageLabs ______________________________________________________________________
