We are using Jena 2.11.0 and trying to do some concurrency stress testing.
When running our tests, after a while we get the following error.
com.hp.hpl.jena.tdb.base.block.BlockException: No such block
at
com.hp.hpl.jena.tdb.base.recordbuffer.RecordRangeIterator.iterator(
RecordRangeIterator.java:39)
at com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.iterator(
BPlusTree.java:383)
at com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.iterator(
BPlusTree.java:366)
at com.hp.hpl.jena.tdb.index.TupleIndexRecord.findWorker(
TupleIndexRecord.java:164)
at com.hp.hpl.jena.tdb.index.TupleIndexRecord.findOrScan(
TupleIndexRecord.java:84)
at com.hp.hpl.jena.tdb.index.TupleIndexRecord.performFind(
TupleIndexRecord.java:78)
at com.hp.hpl.jena.tdb.index.TupleIndexBase.find(
TupleIndexBase.java:91)
at com.hp.hpl.jena.tdb.index.TupleTable.find(TupleTable.java:197)
at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.find(
NodeTupleTableConcrete.java:169)
at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.find(
NodeTupleTableConcrete.java:157)
at
com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.findAsNodeIds(
NodeTupleTableConcrete.java:146)
at com.hp.hpl.jena.tdb.store.QuadTable.find(QuadTable.java:87)
at
com.hp.hpl.jena.tdb.store.DatasetGraphTDB.findInSpecificNamedGraph(
DatasetGraphTDB.java:89)
at com.hp.hpl.jena.sparql.core.DatasetGraphBaseFind.findNG(
DatasetGraphBaseFind.java:59)
at com.hp.hpl.jena.sparql.core.DatasetGraphBaseFind.find(
DatasetGraphBaseFind.java:48)
at com.hp.hpl.jena.sparql.core.GraphView.graphBaseFind(
GraphView.java:110)
at com.hp.hpl.jena.sparql.core.GraphView.graphBaseFind(
GraphView.java:104)
at com.hp.hpl.jena.graph.impl.GraphBase.find(GraphBase.java:268)
at com.hp.hpl.jena.graph.GraphUtil.findAll(GraphUtil.java:128)
at com.hp.hpl.jena.graph.GraphUtil.addInto(GraphUtil.java:183)
at com.hp.hpl.jena.rdf.model.impl.ModelCom.add(ModelCom.java:208)
at com.hp.hpl.jena.rdf.model.impl.ModelCom.add(ModelCom.java:202)
at com.ibm.jenatest.CreateThread.getModelCopy(
CreateThread.java:154)
at com.ibm.jenatest.CreateThread.checkModel(CreateThread.java:125)
at com.ibm.jenatest.CreateThread.test(CreateThread.java:86)
at com.ibm.jenatest.CreateThread.run(CreateThread.java:52)
Can anyone please explain why we are getting this error, ie if this is a
bug or we are doing something wrong in our test.
Occasionally we get other issues in the reader thread when we retrieve the
statements from the resource, the number that are returned do not match
the number that are expected (ie 1000) but if we get the statements again
all 1000 are there. Hopefully there is an explanation for each of these
issues.
We are running out test on a RHEL system using the IBM JVM, but have also
used other JVMs and we get the same issues.
Our test has a writer thread and several (currently 2) reader threads. The
number of iterations necessary for the test to fail vary greatly but it
does fail at some point.
Here are the classes for our test and the classes for the 2 threads:
public class ReaderWriterTest {
public static void main(String[] args) {
int i = 1;
GenericTestThread writer = null;
writer = new WriterThread("writer" + (i++), args[0] + "jenatest");
List<ReaderThread> readers = new ArrayList<ReaderThread>();
// setup the reader threads
readers.add(new ReaderThread("reader" + (i++), args[0] +
"jenatest"));
readers.add(new ReaderThread("reader" + (i++), args[0] +
"jenatest"));
if (writer!=null)
writer.setReaderThreads(readers);
for (GenericTestThread reader : readers) {
if (writer!=null)
reader.setWriterThread(writer);
// create the specific data for each of the reader threads to
read
reader.setReaderThreads(readers);
Dataset createSession = TDBFactory.createDataset
(((ReaderThread)reader).location);
createSession.begin(ReadWrite.WRITE);
String uri = "http://www.ibm.com/"; + reader.getName() + "/" +
0;
// Create Jena model with a number of properties
Model model = createSession.getNamedModel(uri);
Resource res = model.createResource(uri);
for (int j = 0; j < 1000; j++) {
Property p = model.createProperty("
http://www.ibm.com/prop/"; + reader.getName() + "/" + j);
res.addProperty(p, model.createTypedLiteral("Property" +
reader.getName() + "/" + j));
}
model.close();
createSession.commit();
createSession.end();
createSession.close();
}
// start the writer thread
if (writer!=null)
writer.start();
// start the reader threads
for (GenericTestThread reader : readers) {
reader.start();
}
}
}
public class WriterThread extends GenericTestThread {
private static final int NUM_PROPS = 1000;
private String location;
public WriterThread(String name, String location) {
super(name);
this.location = location;
}
@Override
protected void test() {
for (int i = 0; true; i++) {
String uri = "http://www.ibm.com/"; + getName() + "/" + i;
checkStop(uri);
Dataset createSession = TDBFactory.createDataset(location);
createSession.begin(ReadWrite.WRITE);
info(""+i+" size="+createSession.asDatasetGraph().size());
// Create Jena model with a number of properties
Model model = createSession.getNamedModel(uri);
Resource res = model.createResource(uri);
for (int i1 = 0; i1 < NUM_PROPS; i1++) {
Property p = model.createProperty("
http://www.ibm.com/prop/"; + getName() + "/" + i1);
res.addProperty(p, model.createTypedLiteral("Property" +
getName() + "/" + i1));
}
createSession.commit();
createSession.end();
model.close();
createSession.begin(ReadWrite.WRITE);
model = createSession.getNamedModel(uri);
model.removeAll();
createSession.removeNamedModel(uri);
createSession.commit();
createSession.end();
model.close();
createSession.close();
}
}
}
public class ReaderThread extends GenericTestThread {
private static final int NUM_PROPS = 1000;
protected String location;
public ReaderThread(String name, String location) {
super(name);
this.location = location;
}
@Override
protected void test() {
String uri = "http://www.ibm.com/"; + getName() + "/" + 0;
int iteration = 0;
while (true) {
if (((iteration++) % 500) == 0) {
info("check: "+iteration);
// need to add a sleep to the thread otherwise jena
can never write the data and the journal.jml file
// gets excessively large and we run out of memory
in the JVM
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Dataset checkSession = TDBFactory.createDataset(location);
checkSession.begin(ReadWrite.READ);
Model model = checkSession.getNamedModel(uri);
StmtIterator found = model.listStatements(new
SimpleSelector((Resource)null, (Property)null, (RDFNode)null));
Int count = found.toList().size();
if (count != NUM_PROPS) {
System.out.println("Models do not compare expected size: "
+ NUM_PROPS + " was: " + count);
}
found.close();
checkSession.end();
// model.close(); // removed as causes errors with jena
saying we have multiple writers !!!!!!
checkSession.close();
}
}
}
public abstract class GenericTestThread extends Thread {
private GenericTestThread writerThread;
private volatile boolean stopped = false;
private List<ReaderThread> readers;
public GenericTestThread(String name) {
setName(name);
}
protected abstract void test();
public void run() {
try {
info("Running thread");
test();
info("Thread done");
} catch (Exception e) {// STYLE reason: fault barrier
synchronized (WriterThread.class) {
snapshotWriterThread();
for (GenericTestThread reader : readers) {
reader.shutdown();
}
e.printStackTrace(System.out);// STYLE reason: unit test
// System.exit(0); // STYLE reason: TODO
}
}
}
protected void snapshotWriterThread() {
if (writerThread != null) {
info(writerThread.getStackTrace());
writerThread.shutdown();
}
}
private void shutdown() {
stopped = true;
}
private void info(StackTraceElement[] stackTraceElements) {
for (StackTraceElement trace : stackTraceElements) {
info(" " + trace);
}
}
protected void info(Exception e) {
System.out.println(getName() + ": " + e.toString());// STYLE
reason: unit test
}
protected void info(String message) {
System.out.println(getName() + ": " + message);// STYLE reason:
unit test
}
protected void checkStop(String message) {
if (stopped) {
throw new RuntimeException("thread stopped: " + message);
}
}
public void setWriterThread(GenericTestThread other) {
this.writerThread = other;
}
public GenericTestThread getWriterThread() {
return this.writerThread;
}
public void setReaderThreads(List<ReaderThread> readers) {
this.readers = readers;
}
public List<ReaderThread> getReaderThreads() {
return this.readers;
}
}
Thanks
Steve Groeger
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
