[ https://issues.apache.org/jira/browse/JENA-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565378#comment-16565378 ]
Scott Patterson commented on JENA-1581: --------------------------------------- I will try to put together a small example to reproduce this. In the meantime I tested out a small change to org.apache.jena.tdb.lib.NodeLib.encodeStore() to always allocate a new buffer and the problem went away. Could this be a problem with NodeLib using a bytebuffer in a single threaded fashion? {code:java} //throw BufferOverflowException final private static ByteBuffer workspace = ByteBuffer.allocate(SIZE); public static long encodeStore(Node node, ObjectFile file) { int maxSize = nodec.maxSize(node); ByteBuffer bb = workspace; if ( maxSize >= SIZE ) // Large object. Special buffer. bb = ByteBuffer.allocate(maxSize); else bb.clear(); int len = nodec.encode(node, bb, null); long x = file.write(bb); return x; } //no problem public static long encodeStore(Node node, ObjectFile file) { int maxSize = nodec.maxSize(node); ByteBuffer bb = ByteBuffer.allocate(maxSize); int len = nodec.encode(node, bb, null); long x = file.write(bb); return x; } {code} But that also made me wonder if my application is not using the api in a thread safe manner. Upon startup my application does: {code:java} String directory = "/path/to/tdb"; StoreConnection storeConn = StoreConnection.make(directory); {code} And during shutdown it does {code:java} storeConn.getBaseDataset().close(); StoreConnection.release(Location.create(directory));{code} While the application is running I only create 1 writer thread at a time and it is using the TDB like this: {code:java} Dataset ds = TDBFactory.createDataset(directory); try { ds.begin(ReadWrite.WRITE); try { //add graphs } finally { ds.end(); } } finally { ds.close(); }{code} but the application may have at least two other reading threads using the TDB like this: {code:java} Dataset ds = TDBFactory.createDataset(directory); try { ds.begin(ReadWrite.READ); try { // } finally { ds.end(); } } finally { ds.close(); } {code} Is this usage correct? I'm wondering if the reason the NodeLib is tripping up is because multiple threads are using the private static ByteBuffer. > BufferOverflowException when > ---------------------------- > > Key: JENA-1581 > URL: https://issues.apache.org/jira/browse/JENA-1581 > Project: Apache Jena > Issue Type: Bug > Components: TDB > Environment: Jena 3.8 > Reporter: Scott Patterson > Priority: Major > > I can not add a small number of named graphs (~1000) to the TDB without > encountering a BufferOverflowException. I have a single thread that adds all > the graphs and I get lots of these exceptions with different named graphs > each time. Any thoughts? > {noformat} > Caused by: java.nio.BufferOverflowException > at java.nio.Buffer.nextPutIndex(Buffer.java:532) > at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:169) > at org.apache.jena.atlas.io.BlockUTF8.fromCharsBuffer(BlockUTF8.java:172) > at org.apache.jena.atlas.io.BlockUTF8.fromChars(BlockUTF8.java:86) > at org.apache.jena.atlas.io.BlockUTF8.fromChars(BlockUTF8.java:230) > at org.apache.jena.tdb.store.nodetable.NodecSSE.encode(NodecSSE.java:89) > at org.apache.jena.tdb.lib.NodeLib.encodeStore(NodeLib.java:74) > at > org.apache.jena.tdb.store.nodetable.NodeTableNative.writeNodeToTable(NodeTableNative.java:175) > at > org.apache.jena.tdb.store.nodetable.NodeTableNative.accessIndex(NodeTableNative.java:152) > at > org.apache.jena.tdb.store.nodetable.NodeTableNative._idForNode(NodeTableNative.java:125) > at > org.apache.jena.tdb.store.nodetable.NodeTableNative.getAllocateNodeId(NodeTableNative.java:79) > at > org.apache.jena.tdb.store.nodetable.NodeTableCache._idForNode(NodeTableCache.java:152) > at > org.apache.jena.tdb.store.nodetable.NodeTableCache.getAllocateNodeId(NodeTableCache.java:91) > at > org.apache.jena.tdb.store.nodetable.NodeTableWrapper.getAllocateNodeId(NodeTableWrapper.java:40) > at > org.apache.jena.tdb.store.nodetable.NodeTableInline.getAllocateNodeId(NodeTableInline.java:51) > at > org.apache.jena.tdb.store.nodetupletable.NodeTupleTableConcrete.addRow(NodeTupleTableConcrete.java:85) > at org.apache.jena.tdb.store.QuadTable.add(QuadTable.java:60) > at > org.apache.jena.tdb.store.DatasetGraphTDB.addToNamedGraph(DatasetGraphTDB.java:97) > at > org.apache.jena.sparql.core.DatasetGraphTriplesQuads.add(DatasetGraphTriplesQuads.java:44) > at org.apache.jena.sparql.core.GraphView.performAdd(GraphView.java:152) > at > org.apache.jena.graph.GraphUtil$$Lambda$145.0000000020B643F0.accept(Unknown > Source) > at java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:910) > at > org.apache.jena.graph.GraphUtil.addIteratorWorkerDirect(GraphUtil.java:151) > at org.apache.jena.graph.GraphUtil.addIteratorWorker(GraphUtil.java:145) > at org.apache.jena.graph.GraphUtil.addInto(GraphUtil.java:139) > at > org.apache.jena.sparql.core.DatasetGraphTriplesQuads.addGraph(DatasetGraphTriplesQuads.java:80) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)