Hi Andy, all,
On 6 December 2013 09:28, Andy Seaborne <[email protected]> wrote: > On 05/12/13 22:56, Enrico Daga wrote: > >> Hi all, >> I am having difficulties compiling clerezza, precisely >> >> [INFO] Building Clerezza - SCB Jena TDB Storage Provider 0.7-SNAPSHOT >> >> ... >> TdbTcProviderTest>TcProviderTest.testGraphDeletion:433 ? OutOfMemory >> Java >> heap... >> TdbTcProviderTest>TcProviderTest.testGetGraph:99 ? OutOfMemory Java >> heap >> space >> TdbTcProviderTest>TcProviderTest.testGetTriplesGraph:459 ? OutOfMemory >> Java he... >> TdbTcProviderTest>TcProviderTest.testGraphIsNotMutable:339 ? >> OutOfMemory >> Java ... >> TdbTcProviderTest.testCreateGraph ? OutOfMemory Java heap space >> TdbTcProviderTest.testGetTriplesMGraph ? OutOfMemory Java heap space >> TdbTcProviderTest.testGetTriples ? OutOfMemory Java heap space >> TdbTcProviderTest.testDeleteEntity ? OutOfMemory Java heap space >> TdbTcProviderTest.testCreateMGraphNoDuplicateNames ? OutOfMemory Java >> heap spa... >> TdbTcProviderTest.testCreateMGraphExtended ? OutOfMemory Java heap >> space >> TdbTcProviderTest.testGetMGraph ? OutOfMemory Java heap space >> TdbTcProviderTest.testCreateGraphNoDuplicateNames ? OutOfMemory Java >> heap >> spac... >> TdbTcProviderTest.testCreateGraphWithInitialCollection ? OutOfMemory >> Java >> heap... >> TdbTcProviderTest.testCreateGraphExtended ? OutOfMemory Java heap >> space >> >> I tried increasing the heap until 3g (I have 4 on this machine). >> > > Is your machine a 32bit machine with 32 bit java? I'm unclear here > because you say heap is 3G and I thought the 32 bit limit was 1.5G-ish. > $ mvn --version && java -version Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 13:51:28+0000) Maven home: /opt/local/share/java/maven3 Java version: 1.6.0_65, vendor: Apple Inc. Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home Default locale: en_US, platform encoding: MacRoman OS name: "mac os x", version: "10.9", arch: "x86_64", family: "mac" java version "1.6.0_65" Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609) Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode) > > Does change in heap size move the point of failure? > I checked with 1g and 3g and no, changing the heap size does not change anything, all tests fail (after the same time each) > (TDB can't use memory mapped files on a 32 bit system so it uses old-style > file IO and caches in heap; downside, uses heap.) > > Otherwise - internally there is a cache of open databases because opening > one, and having it filesystem-cache-cold is prohibitively expensive. > > If the tests are creating different DB each time, this might be a partial > cause of heap usage. > This is the case, at a first look [1]. > > There is StoreConnection.reset() and related operations to carefully > manipulate this cache. TDB uses these for @BeforeClass/@AfterClass clean > tests > For testing, in-memory TDB databases are very useful. The default is a > fresh, uncached daatset on each call of TDBFactory.createDataset(). > > Your disk will rattle less. The tests will run faster. They have been > reported as faster than normal Jena in-memory datasets but I don't see why. > They work by having a RAM disk like are that bytes are written into and > copied out of so it is good simulation of a disk - no accidental sharing of > data. I understand, having tests with an in memory TDB would be better. But there should be something wrong with the tests, because I can't see how the tdb caching itself could led to a OOM on all tests after a second or the like: testCreateGraph(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 1.125 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testGetTriplesMGraph(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 0.948 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testGetTriples(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 1.108 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testDeleteEntity(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 0.95 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testCreateMGraphNoDuplicateNames(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 1.101 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testCreateMGraphExtended(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 0.927 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testGetMGraph(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 1.088 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testCreateGraphNoDuplicateNames(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 0.988 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testCreateGraphWithInitialCollection(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 1.256 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space testCreateGraphExtended(org.apache.clerezza.rdf.jena.tdb.storage.TdbTcProviderTest) Time elapsed: 0.935 sec <<< ERROR! java.lang.OutOfMemoryError: Java heap space Bests, Enrico > > > Andy > > > >> Any hint? >> >> Thank you! >> Enrico >> >> > [1] https://git-wip-us.apache.org/repos/svn?p=clerezza.git;a=blob;f=parent/rdf.jena.tdb.storage/src/test/java/org/apache/clerezza/rdf/jena/tdb/storage/TdbTcProviderTest.java;h=3ab68476ec397f63715155c51e9c08c9e773681a;hb=HEAD -- Enrico Daga -- http://www.enridaga.net skype: enri-pan
