On 15/09/15 21:00, Todd Detwiler wrote:
Hi Andy,
In this case I am running the version of Jena that came with a recent
(within the last couple of months) Fuseki2 war file distribution. The
core libs look to be 2.13.*.
Fuseki 2.3.0 / Jena 3.0.0 was July.
I can't say what Tomcat does under the hood when you start and stop
apps, but the behavior I saw wasn't just through the text-based http
tools, the same behavior occurs via the manager gui. I know that there
are start-up and shutdown listener hooks in Tomcat. So I could insert
some clean-up code there if I knew what needed to happen.
I don't know about the text index. For TDB, something needs to release
the database (StoreConnection.reset() - caution - what about active
transactions?)
Yes, I could do as you say and limit the update endpoint to localhost.
Will rebuilding the whole store (I want to drop everything in it,
rebuild from two large files, and regenerate the Lucene indexes) be as
efficient as using the bulk loader? Is that your recommended method?
It depends on "large". The TDB bulkloader is faster than just loading
via the usual API. Maybe the same for the text index.
Andy
Your last sentence said,
SPARQL Update (in the latest version where text changes and database
changes are coordinated) is
What did you intend to say after "is"?
Thanks,
Todd
Landon Todd Detwiler
Structural Informatics Group (SIG)
University of Washington
phone: 206-351-7721
On 9/15/15 12:46 PM, Andy Seaborne wrote:
On 15/09/15 17:39, Todd Detwiler wrote:
Let me start with what I'm trying to do. Basically, I'm writing a
script, that will run off of a cron job, that will rebuild my TDB store
(and Lucene indexes) on a regular basis. I looked into using the TDB
command line scripts, but they fail because another JVM (the tomcat app)
is already using the TDB store. I looked into using Sparql update
through Fuseki, but I like the idea of the store being read only (and
therefore update is not allowed). So, I am experimenting with having my
script interact with the Tomcat Manager, stopping the running Fuseki app
(but not all of tomcat), making changes, and then restarting. I am able
to stop it fine, using a url like:
(domain):(port)/manager/text/stop?path=/fuseki
However, the corresponding call to start fails:
(domain):(port)/manager/text/start?path=/fuseki
I am getting a stack trace in the tomcat log file that indicates an
issue with file locks related to Lucene indexing. Now, if I stop tomcat
entirely, and start it again, everything works fine. But I don't want to
shut down the other services. Do I need to add a listener that performs
some sort of operation when Tomcat stops? Have others encountered the
same issue. I've included the stack trace at the bottom of this message.
I'd appreciate any insight anyone could offer. Or let me know if you
think using Sparql update would be a preferred way of rebuilding the
store?
Thanks,
Todd
Hi Todd,
Firstly, which version of jena are you using? There were some changes
to the Lucene indexing around updates that changed the way it works
which may play into this. I see "com.hp.hpl.jena" so it's not the
latest.
Also, I don't know what stopping a webapp in Tomcat actually does. The
standard ServletContextListener only has an operation called
synchronously so it is tricky to clear up in-flight transactions (TDB
does not have forced system aborts by design). Fuseki2, and maybe
wider, needs a proper service lifecycle.
Restarting is clearly calling assembler processing.
Could you protect the update endpoints with the tomcat configuration
to, say, only respond to requests for localhost? While forcing
listening to a specific host isn't very safe, localhost is different
because it means the request was on the loop back interface so can't
be done from outside.
SPARQL Update (in the latest version where text changes and database
changes are coordinated) is
Andy
15-Sep-2015 09:02:55.968 SEVERE [http-nio-8080-exec-10]
org.apache.catalina.core.StandardContext.listenerStart Exception sending
context initialized event to listener instance of class
org.apache.jena.fuseki.server.FusekiServerListener
com.hp.hpl.jena.assembler.exceptions.AssemblerException: caught: null
doing:
root: http://localhost/jena_example/#indexLucene with type:
http://jena.apache.org/text#TextIndexLucene assembler class: class
org.apache.jena.query.text.assembler.TextIndexLuceneAssembler
root: http://localhost/jena_example/#text_dataset with type:
http://jena.apache.org/text#TextDataset assembler class: class
org.apache.jena.query.text.assembler.TextDatasetAssembler
at
com.hp.hpl.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:138)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.open(AssemblerGroup.java:117)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerGroup$ExpandingAssemblerGroup.open(AssemblerGroup.java:81)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:52)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:48)
at
org.apache.jena.query.text.assembler.TextDatasetAssembler.open(TextDatasetAssembler.java:63)
at
org.apache.jena.query.text.assembler.TextDatasetAssembler.open(TextDatasetAssembler.java:41)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:130)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.open(AssemblerGroup.java:117)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerGroup$ExpandingAssemblerGroup.open(AssemblerGroup.java:81)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:52)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:48)
at
org.apache.jena.fuseki.build.Builder.buildDataService(Builder.java:76)
at
org.apache.jena.fuseki.build.Builder.buildDataAccessPoint(Builder.java:61)
at
org.apache.jena.fuseki.build.FusekiConfig.servicesAndDatasets(FusekiConfig.java:133)
at
org.apache.jena.fuseki.build.FusekiConfig.readConfigFile(FusekiConfig.java:75)
at
org.apache.jena.fuseki.server.FusekiServer.initServerConfiguration(FusekiServer.java:248)
at
org.apache.jena.fuseki.server.FusekiServer.initializeDataAccessPoints(FusekiServer.java:216)
at
org.apache.jena.fuseki.server.FusekiServerListener.init(FusekiServerListener.java:70)
at
org.apache.jena.fuseki.server.FusekiServerListener.contextInitialized(FusekiServerListener.java:45)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4729)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5167)
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at
org.apache.catalina.manager.ManagerServlet.start(ManagerServlet.java:1293)
at
org.apache.catalina.manager.HTMLManagerServlet.start(HTMLManagerServlet.java:673)
at
org.apache.catalina.manager.HTMLManagerServlet.doPost(HTMLManagerServlet.java:221)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.filters.CsrfPreventionFilter.doFilter(CsrfPreventionFilter.java:213)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:617)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1521)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.OverlappingFileLockException
at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1075)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155)
at
org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:217)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:702)
at
org.apache.jena.query.text.TextIndexLucene.openIndexWriter(TextIndexLucene.java:121)
at
org.apache.jena.query.text.TextIndexLucene.<init>(TextIndexLucene.java:114)
at
org.apache.jena.query.text.TextDatasetFactory.createLuceneIndex(TextDatasetFactory.java:97)
at
org.apache.jena.query.text.assembler.TextIndexLuceneAssembler.open(TextIndexLuceneAssembler.java:96)
at
org.apache.jena.query.text.assembler.TextIndexLuceneAssembler.open(TextIndexLuceneAssembler.java:47)
at
com.hp.hpl.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:130)
... 54 more
