Yes this was happening because of JVM heap size But the real issue is that if our index size is growing (very high)
then indexing time is taking very long (using streaming) earlier for indexing 15,000 docs at a time (commit after 15000 docs) , it was taking 3 mins 20 secs time, after deleting the index data, it is taking 9 secs What would be approach to have better indexing performance as well as index size should also at the same time. The index size was around 4.5 GB Thanks Naveen On Thu, Aug 11, 2011 at 3:47 PM, Peter Sturge <peter.stu...@gmail.com>wrote: > Hi, > > When you get this exception with no other error or explananation in > the logs, this is almost always because the JVM has run out of memory. > Have you checked/profiled your mem usage/GC during the stream operation? > > > > On Thu, Aug 11, 2011 at 3:18 AM, Naveen Gupta <nkgiit...@gmail.com> wrote: > > Hi, > > > > We are doing streaming update to solr for multiple user, > > > > We are getting > > > > > > Aug 10, 2011 11:56:55 AM org.apache.solr.common.SolrException log > > > > SEVERE: org.apache.lucene.store.LockObtainFailedException: Lock obtain > timed > > out: NativeFSLock@/var/lib/solr/data/index/write.lock > > at org.apache.lucene.store.Lock.obtain(Lock.java:84) > > at > org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1097) > > at > > org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:83) > > at > > > org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:102) > > at > > > org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:174) > > at > > > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:222) > > at > > > org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61) > > at > > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147) > > at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77) > > at > > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:55) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > > at > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > > at > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > > at > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > > at > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > > at > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > > at > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) > > at > > > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) > > at > > > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) > > at org.apache.tomcat.util.net.JIoEndpoint > > > > Aug 10, 2011 12:00:16 PM org.apache.solr.common.SolrException log > > SEVERE: org.apache.lucene.store.LockObtainFailedException: Lock obtain > timed > > out: NativeFSLock@/var/lib/solr/data/index/write.lock > > at org.apache.lucene.store.Lock.obtain(Lock.java:84) > > at > org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1097) > > at > > org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:83) > > at > > > org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:102) > > at > > > org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:174) > > at > > > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:222) > > at > > > org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61) > > at > > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147) > > at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77) > > at > > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:55) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > > at > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > > at > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > > at > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > > at > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > > at > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > > at > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) > > at > > > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) > > at > > > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) > > at > > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) > > at java.lang.Thread.run(Thread.java:662) > > > > Aug 10, 2011 12:59:56 PM org.apache.solr.update.SolrIndexWriter finalize > > SEVERE: SolrIndexWriter was not closed prior to finalize(), indicates a > bug > > -- POSSIBLE RESOURCE LEAK!!! > > > > Kindly tell me where it is failing > > > > We have increased timelockout. But still it is giving the same problem > > > > Thanks > > Naveen > > >
