On 19/03/13 15:35, Reto Bachmann-Gmür wrote:
On Tue, Mar 19, 2013 at 3:59 PM, Andy Seaborne <[email protected]> wrote:
On 19/03/13 13:57, Reto Bachmann-Gmür wrote:
So for the problem with the tdb tc provider I've made a first commit on
issue CLEREZZA-749.
It doesn't solve the problem only shows it more clearly.
Apparently jena tdb keeps the files open even after Graph.close has been
called making it impossible to delete them. I'm not sure how this could be
solved or if we should raise an issue again Jena TDB.
(you are using an old version of TDB - even without transactions, updating
will pick up various important bug fixes)
Ok.
A graph is just a stateless view on the database so closing a graph is a
bit of a no-op (it syncs the files if not transactional). Just forgetting
about them is fine, no close required. Or do a removeAll() to clear them.
The problem is that the underlying files are not closed and so they cannot
be deleted under windows. removeAll seems not to be available in the
version we are using.
Graph.remove(null, null, null) ;
Model.removeAll() ;
@Minto is the patch for CLEREZZA-726 ready to be applied or are you working
on an update version?
TDB caches databases - it's expensive to keep opening and closing them (no
disk caching benefits) and tracking whether its in use in Java isn't very
helpful. You can flush the cache but that will not avoid the windows issue.
It could close the file when close on the graph is explicitely invoked?
One graph != one file. It's a groups of files for all graphs in the
dataset. A named graph is a view of the quad table.
Besides the window issue unix systems aren't happy either if they have to
keep unsued filehandler open.
If you flush the cache, the database is closed.
Outside testing, don't delete a database while running.
So in tests there's apossibility I can remove the database files?
Yes - the TDB test suite does - on Windows it creates a new directory
for each test using a unique name generator. They go away when the JVM
exits but only then.
This is probably the best way. It's most realistic.
You can run in direct mode when database fileds are deletable (as happens
when running on a 32 bit JVM).
I'm also having the issue when invoking:
TDB.getContext().set(Symbol.create("tdb:fileMode"), "direct");
This must be done very early - once TDB starts, it's fixed.
The tests have a same-package hack to allow them to play with it to test
the block manager implementations.
SystemTDB.setFileMode
And the symbol is "http://jena.hpl.hp.com/TDB#fileMode"; (the tdb: stuff
is command line veneer).
The "unique directory" approach is better.
Andy
Cheers,
Reto
Andy
Cheers,
Reto
On Tue, Mar 19, 2013 at 11:23 AM, Reto Bachmann-Gmür <[email protected]
wrote:
Hi Minto
I can reproduce the tdb error and I found what's going wrong. I'm working
on a fix.
I don't have any other failing module (fixed file storage earlier).
Cheers,
Reto
On Tue, Mar 19, 2013 at 11:09 AM, Minto van der Sluis <[email protected]
wrote:
I get the following errors:
_*platform.typerendering.**seedsnipe*_
Failed tests:
ifTest(org.apache.clerezza.**platform.typerendering.**
seedsnipe.GenericTest)
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0
_*rdf.jena.tdb.storage*_
Tests in error:
dateStorage(org.apache.**clerezza.rdf.jena.tdb.storage.**
TdbMGraphTest)
dateStorage(org.apache.**clerezza.rdf.jena.tdb.storage.**
TdbMGraphTest)
testAddCountAndGetTriples(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testAddCountAndGetTriples(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testRemoveAllTriples(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testRemoveAllTriples(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testUseTypedLiterals(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testUseTypedLiterals(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testUseLanguageLiterals(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testUseLanguageLiterals(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testRemoveViaIterator(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testRemoveViaIterator(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testAddSingleTriple(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testAddSingleTriple(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testAddSameTripleTwice(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testAddSameTripleTwice(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testRemoveSingleTriple(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testRemoveSingleTriple(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testRemoveSameTripleTwice(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testRemoveSameTripleTwice(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testGetSameBNode(org.apache.**clerezza.rdf.jena.tdb.storage.**
TdbMGraphTest)
testGetSameBNode(org.apache.**clerezza.rdf.jena.tdb.storage.**
TdbMGraphTest)
testContainsIfContained(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testContainsIfContained(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testContainsIfEmpty(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testContainsIfEmpty(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testContainsIfNotContained(**org.apache.clerezza.rdf.jena.**
tdb.storage.TdbMGraphTest)
testContainsIfNotContained(**org.apache.clerezza.rdf.jena.**
tdb.storage.TdbMGraphTest)
testFilterEmptyGraph(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testFilterEmptyGraph(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testFilterSingleEntry(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testFilterSingleEntry(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
testFilterByObject(org.apache.**clerezza.rdf.jena.tdb.storage.**
TdbMGraphTest)
testFilterByObject(org.apache.**clerezza.rdf.jena.tdb.storage.**
TdbMGraphTest)
graphEventTestAddRemove(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
graphEventTestAddRemove(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
graphEventTestFilterRemove(**org.apache.clerezza.rdf.jena.**
tdb.storage.TdbMGraphTest)
graphEventTestFilterRemove(**org.apache.clerezza.rdf.jena.**
tdb.storage.TdbMGraphTest)
graphEventTestIteratorRemove(**org.apache.clerezza.rdf.jena.**
tdb.storage.TdbMGraphTest)
graphEventTestIteratorRemove(**org.apache.clerezza.rdf.jena.**
tdb.storage.TdbMGraphTest)
graphEventTestClear(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
graphEventTestClear(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
graphEventTestWithDelay(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
graphEventTestWithDelay(org.**apache.clerezza.rdf.jena.tdb.**
storage.TdbMGraphTest)
graphEventTestAddAllRemoveAll(**org.apache.clerezza.rdf.jena.**
tdb.storage.TdbMGraphTest)
graphEventTestAddAllRemoveAll(**org.apache.clerezza.rdf.jena.**
tdb.storage.TdbMGraphTest)
testGetSize(org.apache.**clerezza.rdf.jena.tdb.storage.**
TdbMGraphTest)
testGetSize(org.apache.**clerezza.rdf.jena.tdb.storage.**
TdbMGraphTest)
org.apache.clerezza.rdf.jena.**tdb.storage.**
SingleTdbDatasetMGraphTest
testListGraph(org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**)
testCreateGraph(org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**)
testCreateMGraph(org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**)
testGetMGraph(org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**)
testGetTriples(org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**)
testDeleteEntity(org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**)
testCreateMGraphExtended(org.**apache.clerezza.rdf.jena.tdb.**storage.*
*SingleTdbDatasetTcProviderTest**)
testCreateGraphExtended(org.**apache.clerezza.rdf.jena.tdb.**storage.**
SingleTdbDatasetTcProviderTest**)
testGraphIsNotMutable(org.**apache.clerezza.rdf.jena.tdb.**storage.**
SingleTdbDatasetTcProviderTest**)
testGraphDeletion(org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**)
testGetTriplesGraph(org.**apache.clerezza.rdf.jena.tdb.**storage.**
SingleTdbDatasetTcProviderTest**)
testGetTriplesMGraph(org.**apache.clerezza.rdf.jena.tdb.**storage.**
SingleTdbDatasetTcProviderTest**)
testCreateGraphNoDuplicateName**s(org.apache.clerezza.rdf.**
jena.tdb.storage.**SingleTdbDatasetTcProviderTest**)
testCreateMGraphNoDuplicateNam**es(org.apache.clerezza.rdf.**
jena.tdb.storage.**SingleTdbDatasetTcProviderTest**)
testCreateGraphWithInitialColl**ection(org.apache.clerezza.**
rdf.jena.tdb.storage.**SingleTdbDatasetTcProviderTest**)
testGetGraph(org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**)
Tests run: 109, Failures: 0, Errors: 65, Skipped: 0
The test logs contain errors like:
1) <testcase time="0.003"
classname="org.apache.**clerezza.rdf.jena.tdb.storage.**TdbMGraphTest"
name="dateStorage">
<error message="An entity with this name already exists:
<http://text.example.org/&**gt <http://text.example.org/>>;"
type="org.apache.clerezza.rdf.**core.access.**
EntityAlreadyExistsException">**org.apache.clerezza.rdf.core.**access.*
*EntityAlreadyExistsException:
An entity with this name already exists: <http://text.example.org/&*
*gt <http://text.example.org/>>;
2) <testcase time="0.005"
classname="org.apache.**clerezza.rdf.jena.tdb.storage.**TdbMGraphTest"
name="dateStorage">
<error
type="java.lang.**NullPointerException">java.**
lang.NullPointerException
at
org.apache.clerezza.rdf.jena.**tdb.storage.TdbTcProvider.**
deleteTcDir(TdbTcProvider.**java:242)
at
org.apache.clerezza.rdf.jena.**tdb.storage.TdbTcProvider.**
deleteTripleCollection(**TdbTcProvider.java:233)
3) <testcase time="0.078"
classname="org.apache.**clerezza.rdf.jena.tdb.storage.**
SingleTdbDatasetTcProviderTest**"
name="testListGraph">
<error message="forwarding ..."
type="java.lang.**RuntimeException">java.lang.**RuntimeException:
forwarding
...
at
org.apache.clerezza.rdf.jena.**tdb.storage.**
SingleTdbDatasetTcProviderTest**.getInstance(**
SingleTdbDatasetTcProviderTest**.java:62)
at
org.apache.clerezza.rdf.jena.**tdb.storage.**
SingleTdbDatasetTcProviderTest**.testListGraph(**
SingleTdbDatasetTcProviderTest**.java:154)
_*rdf.file.storage*_
Tests in error:
testGetTriples(org.apache.**clerezza.rdf.file.storage.**
FileTcProviderTest)
testDeleteEntity(org.apache.**clerezza.rdf.file.storage.**
FileTcProviderTest)
testDataFile(org.apache.**clerezza.rdf.file.storage.**
FileTcProviderTest)
testAutoMGraphCreationFromExis**tingFile(org.apache.clerezza.**
rdf.file.storage.**FileTcProviderTest)
testCreateMGraph(org.apache.**clerezza.rdf.file.storage.**
FileTcProviderTest)
testGetMGraph(org.apache.**clerezza.rdf.file.storage.**
FileTcProviderTest)
testCreateMGraphExtended(org.**apache.clerezza.rdf.file.**
storage.FileTcProviderTest)
testGetTriplesMGraph(org.**apache.clerezza.rdf.file.**
storage.FileTcProviderTest)
testCreateMGraphNoDuplicateNam**es(org.apache.clerezza.rdf.**
file.storage.**FileTcProviderTest)
org.apache.clerezza.rdf.file.**storage.FileMGraphTest
Tests run: 19, Failures: 0, Errors: 10, Skipped: 0
The test logs contain errors like:
1) <testcase time="0.031"
classname="org.apache.**clerezza.rdf.file.storage.**FileMGraphTest"
name="org.apache.clerezza.rdf.**file.storage.FileMGraphTest">
<error message="Illegal character in authority at index 7:
file://C:\Users\Minto\AppData\**Local\Temp\/test-04.nt"
type="java.lang.**IllegalArgumentException">**java.lang.**
IllegalArgumentException:
Illegal character in authority at index 7:
file://C:\Users\Minto\AppData\**Local\Temp\/test-04.nt
at java.net.URI.create(URI.java:**859)
at
org.apache.clerezza.rdf.file.**storage.FileMGraphTest.**
createTempFileFromResource(**FileMGraphTest.java:73)
_*rdf.jena.tdb.storage*_
Op 19-3-2013 6:57, Reto Bachmann-Gmür schreef:
On Tue, Mar 19, 2013 at 6:18 AM, Reto Bachmann-Gmür <[email protected]>
wrote:
On Mon, Mar 18, 2013 at 11:06 PM, Rupert Westenthaler <
[email protected]> wrote:
On Mon, Mar 18, 2013 at 9:56 PM, Minto van der Sluis <[email protected]>
wrote:
Hmm, odd!
Might this be platform related? I am building on Windows 7 with Java
7.
Yes. Under Windows you can not delete Files that are still used. On
Linux and Mac this is possible.
I don't think one of or tests does this.
I've just tried a build under windows and it fails in the jena.tdb
module
with lots of errors like:
com.hp.hpl.jena.tdb.base.**block.BlockException:
BlockMgrMapped.**segmentAllocate: Segment = 0
at
com.hp.hpl.jena.tdb.base.**block.BlockMgrMapped.**
allocSegment(BlockMgrMapped.**java:173)
is this the same as you're having?
This was probably due to my full disk. Trying again now with space on
the
disk and with -fae option.
Reto
Cheers,
Reto
I had similar issues with deleting SolrCores after UnitTests on
Windows.
best
Rupert
Regards,
Minto
Op 18-3-2013 20:00, Reto Bachmann-Gmür schreef:
On Mon, Mar 18, 2013 at 7:38 PM, Reto Bachmann-Gmür <
[email protected]>
wrote:
On Thu, Mar 14, 2013 at 11:10 AM, Minto van der Sluis <
[email protected]
wrote:
Hi Folks,
Doing a fresh build of the lastest and greatest from git results
in
test
errors for the following projects:
scala-scripting
platform.typerendering.**seedsnipe
rdf.jena.tdb.storage (not my modified version)
rdf.file.storage
These components do build, it's only testing that fails.
Are these known issues? Is there some kind of continuous build
for
Clerezza?
Let me answer the easy question first. Yes there is continuos
integration,
it's here:
https://builds.apache.org/job/**clerezza-trunk-1.6/<https://builds.apache.org/job/clerezza-trunk-1.6/>
I'll have a look if I broke something with my work on the parent.
It builds here:
[INFO]
------------------------------**------------------------------**
------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------**------------------------------**
------------
[INFO] Total time: 10:20.344s
[INFO] Finished at: Mon Mar 18 19:51:59 CET 2013
[INFO] Final Memory: 217M/925M
[INFO]
------------------------------**------------------------------**
------------
reto@mufu-desktop-hp:~/**projects/apache/clerezza$ svn status
reto@mufu-desktop-hp:~/**projects/apache/clerezza$
The latest failure on Jenkins is due to the missing plugin:
-DartifactId=maven-ontologies-**plugin -Dversion=0.4-SNAPSHOT
The problem with plugins is that they are not taken into account by
maven when creating the reactor build order. I've now deployed the
plugin manually and I hope that the build will
now succeed.
Reto
--
ir. ing. Minto van der Sluis
Software innovator / renovator
Xup BV
Mobiel: +31 (0) 626 014541
--
| Rupert Westenthaler [email protected]
| Bodenlehenstraße 11 ++43-699-11108907
| A-5500 Bischofshofen
--
ir. ing. Minto van der Sluis
Software innovator / renovator
Xup BV
Mobiel: +31 (0) 626 014541
