Thanks once again, Andy. Perhaps the most relevant line from the transaction web page is the following sentence: "A TDB-backed dataset can be used non-transactionally but once used in a transaction, it should be used transactionally after that."
I had been using it non-transactionally, which seemed to work fine. But then I put in some use of TDB transactions for some updates, but not for some updates are for any read accesses, and that is when the problems started showing up. The docs read as if encasing read operations in a read transaction are not going to impact performance, so I will convert everything to be transaction based. It is unfortunate that there does not seem to be a single, consistent API for doing transactions that works for both TDB and SDB. But based on the performance numbers we have been getting, I think we will stick with TDB, SDB was just too slow. On Apr 27, 2013, at 11:46 AM, Andy Seaborne wrote: > On 27/04/13 16:24, David Jordan wrote: >> >> Tomorrow I will try to get into work, download the TDB source, set a >> breakpoint on this FileException to try and diagnose this myself. This >> happens when I first try to access a named Model in TDB that had been stored >> previously. Unless there is a bug in TDB, I am guessing that this suggests >> that an earlier transaction left the file in a corrupted state? >> >> So what can cause TDB to get corrupted? Does every access need to be >> enclosed in a transaction? If you are strictly reading the TDB database, do >> you need to enclose all accesses in a read transaction, when you know there >> is no update transactions? Is there a need to (always) call the sync() >> method? Does sync() automatically get called when you commit or end a >> transaction? (I would hope so.) >> >> I have never examined TDB source before, it could take quiet a while to >> figure out what is going wrong. Any suggestions for resolving this issue are >> greatly appreciated. >> >> On Apr 26, 2013, at 3:22 PM, David Jordan wrote: >> >>> >>> I seem to randomly get the following file exception. Once I get this, I >>> continue to get it until I remove all the data for the model in question. I >>> reported this before, but I don’t believe I got any response. I am not sure >>> what is triggering this. The same code works fine for a while, and then >>> this happens. > > http://mail-archives.apache.org/mod_mbox/jena-users/201304.mbox/%3C517AA3E5.7050609%40apache.org%3E > > Once the node table is corrupted, it's permanently broken. > > Using TDB (more generalised) transactions will protected better - Model > transactions are only a weak sync and a exit can cause problems. > > When all the triple using that part of the node table, it'll appear to work > but add back a triple using a term that is broken and it'll reappear. > > Andy > >>> >>> >>> [junit] Caused by: com.hp.hpl.jena.tdb.base.file.FileException: >>> ObjectFileStorage.read[nodes.dat](30314841)[filesize=30315565][file.size()=30315565]: >>> Impossibly large object : 876033377 bytes > filesize-(loc+SizeOfInt)=720 >>> [junit] at >>> com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:346) >>> [junit] at >>> com.hp.hpl.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78) >>> [junit] at >>> com.hp.hpl.jena.tdb.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:178) >>> [junit] at >>> com.hp.hpl.jena.tdb.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103) >>> [junit] at >>> com.hp.hpl.jena.tdb.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:74) >>> [junit] at >>> com.hp.hpl.jena.tdb.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:103) >>> [junit] at >>> com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:74) >>> [junit] at >>> com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:55) >>> [junit] at >>> com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67) >>> [junit] at com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:159) >>> [junit] at com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:153) >>> [junit] at >>> com.hp.hpl.jena.tdb.lib.TupleLib.access$100(TupleLib.java:45) >>> [junit] at >>> com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:87) >>> [junit] at >>> com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:83) >>> [junit] at org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:301) >>> [junit] at >>> com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.next(GraphTDBBase.java:207) >>> [junit] at >>> com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.next(GraphTDBBase.java:195) >>> [junit] at >>> com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:94) >>> [junit] at >>> com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:94) >>> [junit] at >>> com.hp.hpl.jena.mem.TrackingTripleIterator.next(TrackingTripleIterator.java:47) >>> [junit] at >>> com.hp.hpl.jena.mem.TrackingTripleIterator.next(TrackingTripleIterator.java:31) >>> [junit] at >>> com.hp.hpl.jena.reasoner.rulesys.impl.RETEEngine.fastInit(RETEEngine.java:151) >>> [junit] at >>> com.hp.hpl.jena.reasoner.rulesys.FBRuleInfGraph.prepare(FBRuleInfGraph.java:483) >>> [junit] at >>> com.hp.hpl.jena.ontology.impl.OntModelImpl.prepare(OntModelImpl.java:2694) >>> [junit] at >>> com.sas.ta.om.test.TestBase.setupConfiguration(TestBase.java:213) >>> [junit] at com.sas.ta.om.test.TestBase.<clinit>(TestBase.java:85) >>> [junit] >>> [junit] >>> [junit] TEST com.sas.ta.om.test.HasValueRestrictionOnPropertyTest FAILED >>> [junit] Running >>> com.sas.ta.om.test.PreventAssigningInvalidPropertyValueTest >>> [junit] Testsuite: >>> com.sas.ta.om.test.PreventAssigningInvalidPropertyValueTest >>> [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec >>> [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec >>> [junit] >>> [junit] Null Test: Caused an ERROR >>> [junit] com.sas.ta.om.test.TestBase >>> [junit] java.lang.NoClassDefFoundError: com.sas.ta.om.test.TestBase >>> [junit] at java.lang.Class.forName0(Native Method) >>> >>> >>> David Jordan >>> Senior Software Developer >>> SAS Institute Inc. >>> Health & Life Sciences, Research & Development >>> Bldg R ▪ Office 4467 >>> 600 Research Drive ▪ Cary, NC 27513 >>> Tel: 919 531 1233 ▪ [email protected]<mailto:[email protected]> >>> www.sas.com<http://www.sas.com/> >>> SAS® … THE POWER TO KNOW® >>> >> >
