On 26/05/14 18:57, Claude Warren wrote:
I was looking at how I could get a minimal set but the data is business
confidential and I am not certian that changing it would produce the same
issue. I will see what I can do about a minimal set -- though I expect
that will still be large.
Claude
I would be surprised if there were no effects except at scale although
where it's going to show first is not clear to me.
If you want to try something amusing possibly with side effects ... but
SPARQL query on the model would by the general engine in Node space (it
can't see that the graoh is TDB-backed).
Graph g = GraphView.createDefaultGraph(ds.asDatasetGraph()) ;
model = ModelFactory.createModelForGraph(g) ;
which inverts the stack to put the view over the top of the switching
datasetgraph.
Getting the named model for the union may work but uses more RAM to get
set semantics.
Andy
On Mon, May 26, 2014 at 5:55 PM, Andy Seaborne <[email protected]> wrote:
On 26/05/14 15:54, Claude Warren wrote:
Look like I needed to add a model=dataset.getDefaultModel() statement to
the commit block os that it reads:
if (cnt % 500 == 0) {
System.out.println( cnt );
dataset.commit();
dataset.begin( ReadWrite.WRITE );
model=dataset.getDefaultModel();
}
Why? I would expect that the model would track the transaction set on the
dataset.
Should be possible but it's hard to balance compatibility
(non-transactional use) with being about to mix TDB graphs into general
purpose dataset and also get efficiency (get SPARQL processing to be
handled by TDB, code that calls .getDefaultModel and not assigning to a
local variable).
Come Jena3, this could be improved with changes to the application
contract for the more corner cases (single thread overlapping transactions
for example). Maybe drop the direct SPARQL execution of TDB graphs in
general purpose, mixed datasets and resort to general SPARQL execution.
It's not just Models - Resources have an implicit graph.
What would be really helpful is to have an example test case that
illustrated some aspect of the problem which was compact and standalone.
Andy
Claude
On Mon, May 26, 2014 at 3:48 PM, Claude Warren <[email protected]> wrote:
I have code that uses Jena-Libs 2.11.1 and executes the following block:
(edited for brevity)
dataset.begin( ReadWrite.WRITE);
try {
Model model = dataset.getDefaultModel();
Property[] props = ( create an array of properties in Model)
String dcsId = "someStringValue";
int cnt = 0;
..... (get an iterator on a set of rows from a db table)
while (rIter.hasNext())
{
cnt++;
Row r = rIter.next();
Resource eventR = model.createResource( makeURI( eventUri,
r.getColumn(dcsId).toString()), typeURI);
for (int i=0;i<cols.size();i++)
{
Object o = r.getColumn(i);
if (props[i] != null && o != null)
{
String val = o.toString();
if (val.length()>0)
{
eventR.addProperty( props[i], val); // <-- line 134 as
noted
in call stack below
}
}
}
// commit every 500 entries
if (cnt % 500 == 0) {
System.out.println( cnt );
dataset.commit();
dataset.begin( ReadWrite.WRITE );
}
}
}
}
}
finally {
dataset.commit();
}
The table I am reading from contains several 10's of thousands of rows.
I
am basically converting from relational table to RDF graph.
However, after the first 500 subjects are committed (5000 +/- triples) I
run into the following problem while adding the 6th property to subject
502.
DEBUG [main] (MySQLTableDef.java:261) - Query String: SELECT ... <--
Initial debugging statement
500 <-- output from commit block in code above
ERROR [main] (Log.java:94) - Not active: 1
ERROR [main] (Log.java:94) - **** Not active: 1
Now, I know this is a transaction error. But I think I have a
transaction, as I restart one right after the commit after the 500th
record. Log is made in the BlockMgrJournal.checkActive() and the call
stack appears as follows:
BlockMgrJournal.checkActive() line: 306
BlockMgrJournal._promote(Block) line: 220
BlockMgrJournal.promote(Block) line: 215
BPTreeRecordsMgr(PageBlockMgr<T>).promote(Page) line: 110
BPTreeRecords.promote() line: 119
BPTreeRecords.internalInsert(Record) line: 131
BPTreeNode.internalInsert(Record) line: 468
BPTreeNode.internalInsert(Record) line: 468
BPTreeNode.insert(BPTreeNode, Record) line: 212
BPlusTree.addAndReturnOld(Record) line: 328
BPlusTree.add(Record) line: 320
TupleIndexRecord.performAdd(Tuple<NodeId>) line: 60
TupleIndexRecord(TupleIndexBase).add(Tuple<NodeId>) line: 64
TupleTable.add(Tuple<NodeId>) line: 96
NodeTupleTableConcrete.addRow(Node...) line: 87
TripleTable.add(Node, Node, Node) line: 58
DatasetGraphTDB.addToDftGraph(Node, Node, Node) line: 100
DatasetGraphTDB(DatasetGraphTriplesQuads).add(Node, Node, Node, Node)
line: 47
GraphTDB(GraphView).performAdd(Triple) line: 140
GraphTDB.performAdd(Triple) line: 87
GraphTDB(GraphBase).add(Triple) line: 202
ModelCom.add(Resource, Property, RDFNode) line: 1159
ModelCom.add(Resource, Property, String, String, boolean) line: 161
ModelCom.add(Resource, Property, String) line: 149
ResourceImpl.addProperty(Property, String) line: 241
JenaSink.updateTable(UpdateBlock) line: 134
Any help fixing this problem would be appreciated.
Claude
--
I like: Like Like - The likeliest place on the web<
http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
