Re: Issue with TDB file locking

Andy Seaborne Thu, 06 Jun 2019 03:22:35 -0700


On 05/06/2019 20:42, Amit Kumar wrote:

Hi Andy,
Thanks for your reply. I tried your 2nd Option.
The StoreConnection.release(location) option didn't seem to work.
I'm assuming the location should point to the main TopLevel Directory

/etc/fuseki/databases/201906031724/
├── Data-0001
│   ├── GOSP.bpt
│   ├── GOSP.dat
|     | ...
│   └── tdb.lock
└── tdb.lock

I gave  "/etc/fuseki/databases/201906031724" as location input

Checking the code, I see that for StoreConnection, it'll be/etc/fuseki/databases/201906031724/Data-0001 (StoreConnection in TDB2manages the the current dataset; TDBInternal.expel understandsswitchable datasets - the level up at /etc/fuseki/databases/201906031724/.


Too much TDB1 thinking!

private boolean postProcessingWebServer(String path) {

         Location location = Location.create(path);

         String lockFilename = location.getPath(Names.TDB_LOCK_FILE);

         LOGGER.info("location {} is presence in StoreConnection.cache
is {}", lockFilename, StoreConnection.isSetup(location));

         StoreConnection.release(location);

         return ExecUtil.restartJetty();

     }


INFO location /etc/fuseki/databases/201906031724/tdb.lock is presence in
StoreConnection.cache is false

This option failed with the same original error. The storeConnection has a
cache, and this location was not in that cache().

=============
So next I tried the following option

private boolean postProcessingWebServer(String path) {
         Location location = Location.create(path);
         String lockFilename = location.getPath(Names.TDB_LOCK_FILE);
         LOGGER.info("location of the lock file is {}", lockFilename);
         StoreConnection storeConnection =
StoreConnection.connectCreate(location);
         storeConnection.getLock().unlock();
         return ExecUtil.restartJetty();
     }


This time I got the following error

org.apache.jena.dboe.base.file.AlreadyLocked: Failed to get a lock:
file='/etc/fuseki/databases/201906031727/tdb.lock': Lock already held
         at 
org.apache.jena.dboe.base.file.ProcessFileLock.lockOperation(ProcessFileLock.java:187)
         at 
org.apache.jena.dboe.base.file.ProcessFileLock.lockEx(ProcessFileLock.java:125)
         at 
org.apache.jena.tdb2.sys.StoreConnection.make(StoreConnection.java:87)
         at 
org.apache.jena.tdb2.sys.StoreConnection.connectCreate(StoreConnection.java:61)
         at 
org.apache.jena.tdb2.sys.StoreConnection.connectCreate(StoreConnection.java:52)


========
Then I tried to use the ProcessFileLock directly

   private boolean postProcessingWebServer(String path) {
         Location location = Location.create(path);
         String lockFilename = location.getPath(Names.TDB_LOCK_FILE);
         ProcessFileLock fileLock = ProcessFileLock.create(lockFilename);
         LOGGER.info("Lock exists for {} is {}", lockFilename,
fileLock.isLockedHere());
         fileLock.unlock();
         return ExecUtil.restartJetty();
     }

INFO Lock exists for /etc/fuseki/databases/201906031755/tdb.lock is true

Caused by: org.apache.jena.dboe.DBOpEnvException: Failed to get a
lock: file='/etc/fuseki/databases/201906031755/Data-0001/tdb.lock':
held by process 8211
        at 
org.apache.jena.dboe.base.file.ProcessFileLock.lockOperation(ProcessFileLock.java:199)
        at 
org.apache.jena.dboe.base.file.ProcessFileLock.lockEx(ProcessFileLock.java:125)
        at 
org.apache.jena.tdb2.sys.StoreConnection.make(StoreConnection.java:87)
        at 
org.apache.jena.tdb2.sys.StoreConnection.connectCreate(StoreConnection.java:61)
        at 
org.apache.jena.tdb2.sys.DatabaseOps.createSwitchable(DatabaseOps.java:97)
        at org.apache.jena.tdb2.sys.DatabaseOps.create(DatabaseOps.java:78)
        at 
org.apache.jena.tdb2.sys.DatabaseConnection.buildForCache(DatabaseConnection.java:90)
        at 
org.apache.jena.tdb2.sys.DatabaseConnection.lambda$make$0(DatabaseConnection.java:76)
        at 
java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
        at 
org.apache.jena.tdb2.sys.DatabaseConnection.make(DatabaseConnection.java:76)
        at 
org.apache.jena.tdb2.sys.DatabaseConnection.connectCreate(DatabaseConnection.java:61)
        at 
org.apache.jena.tdb2.sys.DatabaseConnection.connectCreate(DatabaseConnection.java:52)

This time the error was for the lock file which is inside the Data-0001
folder instead.
=====
When I tried using the unlock on both the lock file explicitly, it worked.
I also tried the TDBInternal.expel(dataset) and it worked

private boolean postProcessingWebServer(String path) {
         Location location = Location.create(path);
         DatabaseConnection databaseConnection =
DatabaseConnection.connectCreate(location);
         TDBInternal.expel(databaseConnection.getDatasetGraph());
         return ExecUtil.restartJetty();
     }



I'm going ahead with the TDBInternal.expel option for now.

Regards
Amit

On Sat, Jun 1, 2019 at 5:59 AM Andy Seaborne <[email protected]> wrote:

Hi Amit,

You are correct - when used in a JVM, the dataset remains manged by the
JVM after loading. This is intentional - the app may want to use the
dataset some more in Java code.

You can:
1/ Fork the process as a separate OS process using java.util.ProcessBuilder

2/ Force it out of the running JVM with
     TDBInternal.expel(dataset)
or
     StoreConenction.release(location)

3/ Work with the dataset in Fuseki - you can POST data to it

4/ Not use a war file but use Fuseki main, and run your program and
Fuseki in the same JVM.

Symbolic links make no difference - they are only involved in file
system path naming. Once opened, the OS pathname isn't involved.

      Andy

On 31/05/2019 21:23, Amit Kumar wrote:

Hi,
I'm having issue an issue with using Fuseki as a war file inside jetty.

We

have a java application which looks for new data and loads it onto

fuseki.

When it is doing it for the first time, it has to create a new index, so

we

call the tdb2.tdbloader class directly with appropriate parameters. Here

is

the relevant code snippet

import tdb2.tdbloader;

   ...

public static List<String> TDBLoaderParams = Arrays.asList("-q",

"--loader", "parallel", "--loc");
...
     List<String> tdbParams = new ArrayList<>(TDBLoaderParams);
            tdbParams.add(dstPath);
            tdbParams.addAll(srcPaths);
try {
       ...
   tdbloader.main(tdbParams.toArray(new String[tdbParams.size()]));
     }
...
...



This works as expected and create a new index at the desired location.

Now,

the problem is the same java process has to apply other feeds onto the

same

graph, so we move the softlink pointing to the fuseki database and

execute

a jetty restart command by calling a shell action using Java

ProcessBuilder

process.
The jetty though fails to restart with the following error:


org.apache.jena.assembler.exceptions.AssemblerException: caught: Failed

to

get a lock: file='/home/amit/fuseki/databases/201905302214/tdb.lock':
held by process 35918 at

org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:165)

at

org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.open(AssemblerGroup.java:144)

at

org.apache.jena.assembler.assemblers.AssemblerGroup$ExpandingAssemblerGroup.open(AssemblerGroup.java:93)

at

org.apache.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:39)

at

org.apache.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:35)

at

org.apache.jena.fuseki.build.FusekiConfig.getDataset(FusekiConfig.java:353)

at

org.apache.jena.fuseki.build.FusekiConfig.buildDataService(FusekiConfig.java:303)

at

org.apache.jena.fuseki.build.FusekiConfig.buildDataAccessPoint(FusekiConfig.java:292)

at

org.apache.jena.fuseki.build.FusekiConfig.readConfiguration(FusekiConfig.java:275)

at

org.apache.jena.fuseki.build.FusekiConfig.readConfigurationDirectory(FusekiConfig.java:254)

... ... Caused by: org.apache.jena.dboe.DBOpEnvException: Failed to get

lock: file='/home/amit/fuseki/databases/201905302214/tdb.lock': held by
process 35918 at

org.apache.jena.dboe.base.file.ProcessFileLock.lockOperation(ProcessFileLock.java:199)

at

org.apache.jena.dboe.base.file.ProcessFileLock.lockEx(ProcessFileLock.java:125)

at

org.apache.jena.tdb2.sys.DatabaseConnection.buildForCache(DatabaseConnection.java:88)

at

org.apache.jena.tdb2.sys.DatabaseConnection.lambda$make$0(DatabaseConnection.java:76)

at

java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)


So the Jetty Server fails to restart while the jvm that created the index
is still active, even though it has completed the index building part. It
restarts fine once the loader JVM dies.  It looks like the parallel

loader

class is not releasing the lock() properly. FYI, I'm using 3.11 Jena

  From apache jena github

https://github.com/apache/jena/blob/66d14eda44dd0f37a9df31ed27c55c26600ff5f8/jena-db/jena-dboe-base/src/main/java/org/apache/jena/dboe/base/file/ProcessFileLock.java#L184-L218




Regards
Amit Kumar

Re: Issue with TDB file locking

Reply via email to