How often are you committing index updates? This kind of thing can happen if you commit too often. Consider setting commitWithin to something like, say, 5 minutes. Or doing the equivalent with the autoCommit parameters in solrconfig.xml
If that isn't relevant, you need to provide some more details about what you're doing and how you're using Solr Best Erick On Sun, Apr 1, 2012 at 10:47 PM, Gopal Patwa <gopalpa...@gmail.com> wrote: > I am using Solr 4.0 nightly build with NRT and I often get this > error during auto commit "Too many open files". I have search this forum > and what I found it is related to OS ulimit setting, please see below my > ulimit settings. I am not sure what ulimit setting I should have for open > file? ulimit -n unlimited?. > > Even if I set to higher number, it will just delay the issue until it reach > new open file limit. What I have seen that Solr has kept deleted index file > open by java process, which causing issue for our application server jboss > to shutdown gracefully due this open files by java process. > > I have seen recently this issue was resolved in lucene, is it TRUE? > > https://issues.apache.org/jira/browse/LUCENE-3855 > > > I have 3 core with index size : core1 - 70GB, Core2 - 50GB and Core3 > - 15GB, with Single shard > > We update the index every 5 seconds, soft commit every 1 second and hard > commit every 15 minutes > > Environment: Jboss 4.2, JDK 1.6 64 bit, CentOS , JVM Heap Size = 24GB* > > > ulimit: > > core file size (blocks, -c) 0 > > data seg size (kbytes, -d) unlimited > > scheduling priority (-e) 0 > > file size (blocks, -f) unlimited > > pending signals (-i) 401408 > > max locked memory (kbytes, -l) 1024 > > max memory size (kbytes, -m) unlimited > > open files (-n) 4096 > > pipe size (512 bytes, -p) 8 > > POSIX message queues (bytes, -q) 819200 > > real-time priority (-r) 0 > > stack size (kbytes, -s) 10240 > > cpu time (seconds, -t) unlimited > > max user processes (-u) 401408 > > virtual memory (kbytes, -v) unlimited > > file locks (-x) unlimited > > > ERROR:* > > *2012-04-01* *20:08:35*,*323* [] *priority=ERROR* *app_name=* > *thread=pool-10-thread-1* *location=CommitTracker* *line=93* *auto* > *commit* *error...:org.apache.solr.common.SolrException:* *Error* > *opening* *new* *searcher* > *at* > *org.apache.solr.core.SolrCore.openNewSearcher*(*SolrCore.java:1138*) > *at* *org.apache.solr.core.SolrCore.getSearcher*(*SolrCore.java:1251*) > *at* > *org.apache.solr.update.DirectUpdateHandler2.commit*(*DirectUpdateHandler2.java:409*) > *at* > *org.apache.solr.update.CommitTracker.run*(*CommitTracker.java:197*) > *at* > *java.util.concurrent.Executors$RunnableAdapter.call*(*Executors.java:441*) > *at* > *java.util.concurrent.FutureTask$Sync.innerRun*(*FutureTask.java:303*) > *at* *java.util.concurrent.FutureTask.run*(*FutureTask.java:138*) > *at* > *java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301*(*ScheduledThreadPoolExecutor.java:98*) > *at* > *java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run*(*ScheduledThreadPoolExecutor.java:206*) > *at* > *java.util.concurrent.ThreadPoolExecutor$Worker.runTask*(*ThreadPoolExecutor.java:886*) > *at* > *java.util.concurrent.ThreadPoolExecutor$Worker.run*(*ThreadPoolExecutor.java:908*) > *at* *java.lang.Thread.run*(*Thread.java:662*)*Caused* *by:* > *java.io.FileNotFoundException:* > */opt/mci/data/srwp01mci001/inventory/index/_4q1y_0.tip* (*Too many > open files*) > *at* *java.io.RandomAccessFile.open*(*Native* *Method*) > *at* *java.io.RandomAccessFile.*<*init*>(*RandomAccessFile.java:212*) > *at* > *org.apache.lucene.store.FSDirectory$FSIndexOutput.*<*init*>(*FSDirectory.java:449*) > *at* > *org.apache.lucene.store.FSDirectory.createOutput*(*FSDirectory.java:288*) > *at* > *org.apache.lucene.codecs.BlockTreeTermsWriter.*<*init*>(*BlockTreeTermsWriter.java:161*) > *at* > *org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat.fieldsConsumer*(*Lucene40PostingsFormat.java:66*) > *at* > *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.addField*(*PerFieldPostingsFormat.java:118*) > *at* > *org.apache.lucene.index.FreqProxTermsWriterPerField.flush*(*FreqProxTermsWriterPerField.java:322*) > *at* > *org.apache.lucene.index.FreqProxTermsWriter.flush*(*FreqProxTermsWriter.java:92*) > *at* *org.apache.lucene.index.TermsHash.flush*(*TermsHash.java:117*) > *at* *org.apache.lucene.index.DocInverter.flush*(*DocInverter.java:53*) > *at* > *org.apache.lucene.index.DocFieldProcessor.flush*(*DocFieldProcessor.java:81*) > *at* > *org.apache.lucene.index.DocumentsWriterPerThread.flush*(*DocumentsWriterPerThread.java:475*) > *at* > *org.apache.lucene.index.DocumentsWriter.doFlush*(*DocumentsWriter.java:422*) > *at* > *org.apache.lucene.index.DocumentsWriter.flushAllThreads*(*DocumentsWriter.java:553*) > *at* > *org.apache.lucene.index.IndexWriter.getReader*(*IndexWriter.java:354*) > *at* > *org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter*(*StandardDirectoryReader.java:258*) > *at* > *org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged*(*StandardDirectoryReader.java:243*) > *at* > *org.apache.lucene.index.DirectoryReader.openIfChanged*(*DirectoryReader.java:250*) > *at* > *org.apache.solr.core.SolrCore.openNewSearcher*(*SolrCore.java:1091*) > > > Thanks > > Gopal
