Hi,
we're still using Solr 4.10.4 without SolrCloud and create cores
dynamically using the Core Admin API.
We have multiple applications that access a core and create it if it
doesn't exist. To this end, we use the STATUS action to see if a
required core exists and if it doesn't, create it with the CREATE
action. In general, this works very well.
However we have a problem if two applications try to create the same
core concurrenty. With unlucky timing, both see a non-existing core in
the STATUS response and then try to CREATE it. One CREATE succeeds, the
other fails with 400 Bad Request. We can handle that properly in our
application.
But Solr logs ugly errors for the failed CREATE call. See below.
I'd guess, I can ignore those errors, including the "POSSIBLE RESOURCE
LEAK"?
Is there a better way to programmaticaly create a core if it doesn't
exist, maybe with just one admin api request (CREATE with a parameter
such as ignoreIfExists=true)
Or can we change the logging somehow (e.g. to WARN) so that
administrators don't get alarmed?
Thank you,
Andreas
Example requests, first succeeds, second fails:
[16/Jun/2016:02:48:08 +0200] "GET
/solr/admin/cores?action=CREATE&name=blueprint_helios_comments&instanceDir=cores%2Fblueprint_helios_comments&dataDir=data&configSet=elastic&wt=javabin&version=2
HTTP/1.1" 200 73 414ms
[16/Jun/2016:02:48:09 +0200] "GET
/solr/admin/cores?action=CREATE&name=blueprint_helios_comments&instanceDir=cores%2Fblueprint_helios_comments&dataDir=data&configSet=elastic&wt=javabin&version=2
HTTP/1.1" 400 314 1325ms
From the log:
2016-06-16 02:48:09.156 [ERROR] org.apache.solr.core.CoreContainer -
Error creating core [blueprint_helios_comments]: Error opening new searcher
org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:881)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:654)
at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:466)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:575)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:199)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:188)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:729)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:683)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at
org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1574)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1686)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:853)
... 27 more
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock
obtain timed out:
NativeFSLock@/opt/coremedia/cm7-solr-master-tomcat/solr-home/cores/blueprint_helios_comments/data/index/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755)
at
org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
at
org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
at
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:284)
at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116)
at
org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1537)
... 29 more
2016-06-16 02:48:09.169 [ERROR] org.apache.solr.common.SolrException -
org.apache.solr.common.SolrException: Error CREATEing SolrCore
'blueprint_helios_comments': Unable to create core
[blueprint_helios_comments] Caused by: Lock obtain timed out:
NativeFSLock@/opt/coremedia/cm7-solr-master-tomcat/solr-home/cores/blueprint_helios_comments/data/index/write.lock
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:613)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:199)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:188)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:729)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:683)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: Unable to create core
[blueprint_helios_comments]
at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:507)
at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:466)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:575)
... 23 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:881)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:654)
at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
... 25 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at
org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1574)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1686)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:853)
... 27 more
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock
obtain timed out:
NativeFSLock@/opt/coremedia/cm7-solr-master-tomcat/solr-home/cores/blueprint_helios_comments/data/index/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755)
at
org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
at
org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
at
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:284)
at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116)
at
org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1537)
... 29 more
2016-06-16 02:49:58.095 [ERROR] org.apache.solr.update.SolrIndexWriter -
SolrIndexWriter was not closed prior to finalize(), indicates a bug --
POSSIBLE RESOURCE LEAK!!!
2016-06-16 02:49:58.097 [ERROR] org.apache.solr.update.SolrIndexWriter -
Error closing IndexWriter
java.lang.NullPointerException
at
org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3271)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3240)
at
org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:910)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:987)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:957)
at
org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:129)
at
org.apache.solr.update.SolrIndexWriter.finalize(SolrIndexWriter.java:182)
at java.lang.System$2.invokeFinalize(System.java:1270)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
