Summarising from Amit's reply:
On 01/06/2019 13:59, Andy Seaborne wrote:
Hi Amit,
You are correct - when used in a JVM, the dataset remains manged by the
JVM after loading. This is intentional - the app may want to use the
dataset some more in Java code.
You can:
1/ Fork the process as a separate OS process using java.util.ProcessBuilder
2/ Force it out of the running JVM with
TDBInternal.expel(dataset)
Works.
or
StoreConenction.release(location)
This does not work.
3/ Work with the dataset in Fuseki - you can POST data to it
4/ Not use a war file but use Fuseki main, and run your program and
Fuseki in the same JVM.
Symbolic links make no difference - they are only involved in file
system path naming. Once opened, the OS pathname isn't involved.
Andy
On 31/05/2019 21:23, Amit Kumar wrote:
Hi,
I'm having issue an issue with using Fuseki as a war file inside
jetty. We
have a java application which looks for new data and loads it onto
fuseki.
When it is doing it for the first time, it has to create a new index,
so we
call the tdb2.tdbloader class directly with appropriate parameters.
Here is
the relevant code snippet
import tdb2.tdbloader;
...
public static List<String> TDBLoaderParams = Arrays.asList("-q",
"--loader", "parallel", "--loc");
...
List<String> tdbParams = new ArrayList<>(TDBLoaderParams);
tdbParams.add(dstPath);
tdbParams.addAll(srcPaths);
try {
...
tdbloader.main(tdbParams.toArray(new String[tdbParams.size()]));
}
...
...
This works as expected and create a new index at the desired location.
Now,
the problem is the same java process has to apply other feeds onto the
same
graph, so we move the softlink pointing to the fuseki database and
execute
a jetty restart command by calling a shell action using Java
ProcessBuilder
process.
The jetty though fails to restart with the following error:
org.apache.jena.assembler.exceptions.AssemblerException: caught:
Failed to
get a lock: file='/home/amit/fuseki/databases/201905302214/tdb.lock':
held by process 35918 at
org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:165)
at
org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.open(AssemblerGroup.java:144)
at
org.apache.jena.assembler.assemblers.AssemblerGroup$ExpandingAssemblerGroup.open(AssemblerGroup.java:93)
at
org.apache.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:39)
at
org.apache.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:35)
at
org.apache.jena.fuseki.build.FusekiConfig.getDataset(FusekiConfig.java:353)
at
org.apache.jena.fuseki.build.FusekiConfig.buildDataService(FusekiConfig.java:303)
at
org.apache.jena.fuseki.build.FusekiConfig.buildDataAccessPoint(FusekiConfig.java:292)
at
org.apache.jena.fuseki.build.FusekiConfig.readConfiguration(FusekiConfig.java:275)
at
org.apache.jena.fuseki.build.FusekiConfig.readConfigurationDirectory(FusekiConfig.java:254)
... ... Caused by: org.apache.jena.dboe.DBOpEnvException: Failed to
get a
lock: file='/home/amit/fuseki/databases/201905302214/tdb.lock': held by
process 35918 at
org.apache.jena.dboe.base.file.ProcessFileLock.lockOperation(ProcessFileLock.java:199)
at
org.apache.jena.dboe.base.file.ProcessFileLock.lockEx(ProcessFileLock.java:125)
at
org.apache.jena.tdb2.sys.DatabaseConnection.buildForCache(DatabaseConnection.java:88)
at
org.apache.jena.tdb2.sys.DatabaseConnection.lambda$make$0(DatabaseConnection.java:76)
at
java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
So the Jetty Server fails to restart while the jvm that created the index
is still active, even though it has completed the index building part. It
restarts fine once the loader JVM dies. It looks like the parallel
loader
class is not releasing the lock() properly. FYI, I'm using 3.11 Jena
From apache jena github
https://github.com/apache/jena/blob/66d14eda44dd0f37a9df31ed27c55c26600ff5f8/jena-db/jena-dboe-base/src/main/java/org/apache/jena/dboe/base/file/ProcessFileLock.java#L184-L218
Regards
Amit Kumar
