Hi Radek,
A way to split the action but still have a single transaction would be
to write it as two SPARQL operations in the one HTTP request:
DELETE{} WHERE {} ; INSERT DATA { .... }
Caution: this always executes the INSERT DATA even if the WHERE{} does
not match. It's possible to get clever and mark that the insert should
happen:
# Insert a triple to note INSERT should be done.
# EXISTS part is your WHERE pattern.
INSERT { markertriple } WHERE { FILTER EXISTS { ... } }
# Do the delete.
DELETE{} WHERE {} ;
# INSERT if an donyl if the marker triple is present.
INSERT { 30k triple ) WHERE { markertriple } ;
# Remove work item.
DELETE { markertriple }
What seems to be happening is that the code is not taking advantage of
the fact that there are lots of ground triples (no variables) in the
INSERT part and doing the INSERT for every WHERE solution generated as
if it were a general template. That makes for a lot of repeated work;
it loops on the result of WHERE and does the INSERT over and over again.
I've created JENA-1059 to record the optimization. (Contributions welcome!)
Thanks for the report,
Andy
On 30/10/15 15:53, Radek Hubner wrote:
Hi,
we are using jena-fuseki with TDB store and discovered some weird behaviour
for DELETE/INSERT query.
When we run query(see below) it takes too long or never finishes(we killed
it after 30min). But when we split this query to separate delete and
insert, it finishes in seconds(3-5s). The query is deleting around 30 000
triples and adding new 30 000.
Versions which we tested : jena-fuseki-0.2.6, apache-jena-fuseki-2.3.0. On
both it behaves the same.
Thanks Radek.
Query example :
PREFIX dri: <http://nationalarchives.gov.uk/terms/dri#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dcterms: <http://purl.org/dc/terms/>
WITH <http://nationalarchives.gov.uk/dri/catalogue>
DELETE {
?recordList ?property ?value .
}
INSERT {
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
rdf:type dri:Export .
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:username "username"^^xsd:string .
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dcterms:created "2015-10-29T16:28:53.627Z"^^xsd:dateTime .
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportStatus <
http://nationalarchives.gov.uk/dri/catalogue/exportStatus#ReadyToProcess> .
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dcterms:identifier "b83c7512-4c51-467a-9cb7-31cb1ecb6ea0"^^xsd:string .
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportMember <
http://nationalarchives.gov.uk/dri/catalogue/file/b78bbd71-5c10-4c3e-8572-e10c40ee4aad>
.
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportMember <
http://nationalarchives.gov.uk/dri/catalogue/file/a2e40103-918b-4c80-9c3e-c3abec53fd03>
.
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportMember <
http://nationalarchives.gov.uk/dri/catalogue/file/73ddced5-0db2-4c64-b975-d2c2954b8a0b>
.
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportMember <
http://nationalarchives.gov.uk/dri/catalogue/file/e50cbd3f-ec22-4bb1-8a80-03bd55031874>
.
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportMember <
http://nationalarchives.gov.uk/dri/catalogue/file/1efb14ce-7027-4d74-9b01-1d21c04ad151>
.
around 30 000 similar lines with different object
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportMember <
http://nationalarchives.gov.uk/dri/catalogue/file/5f089799-9fba-4019-9718-5fb618bf7f4d>
.
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportMember <
http://nationalarchives.gov.uk/dri/catalogue/file/f8e5c82e-3821-450f-b8b3-1319a51b900a>
.
<
http://nationalarchives.gov.uk/dri/catalogue/export/b83c7512-4c51-467a-9cb7-31cb1ecb6ea0>
dri:exportMember <
http://nationalarchives.gov.uk/dri/catalogue/file/c4c93f1b-c5ca-465a-ae35-6f1ef0f3e57f>
.
}
WHERE
{ ?recordList dcterms:identifier
"b83c7512-4c51-467a-9cb7-31cb1ecb6ea0"^^xsd:string .
?recordList ?property ?value
}
