On 14/03/15 01:43, Abduladem Eljamel wrote:
Hi All
I loaded DBPedia RDF dump File to JenaTDB store. The triples were loaded
successively including the indexingphase and “stats,opt” file was created;
however, the SPARQL queries are notworking for complex queries. It takes very
long time (and non-stop runningsometimes). I ran “tdbstats” command to optimize
TDB but there was no change tothe query running or to “stats.opt”. The
“tdbstats” command does not create anyOptimizer control file.After I removed
the “stats.opt” fileand ran the query, I got a result even after a very long
time. I tried those opertions in two kind of computers. The first is with
Windows7-x64,Intel i7 4 cores, 32GB RAM. 2TB HD. The second is with OS X
10.10.2, Intel i7,16GB RAM, 1TB
HD.**************************************************************
The query that I am using to test
is:*******************************************************************
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>PREFIX dbpo:
<http://dbpedia.org/ontology/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>SELECT
DISTINCT ?lblWHERE { { ?entity1 ?rel ?entity2 } UNION { ?entity2 ?rel ?entity1 } ?entity1
rdf:type dbpo:Person . ?entity2 rdf:type dbpo:Place . ?rel rdfs:label ?lbl FILTER ( lang(?lbl) =
"en" )} ********************************************************************
the JAVA code to run query
is:************************************************************************
Location location = new Location("C:/myFiles/RDF-DBPedia/TDB-DP") ;
Dataset dataset = TDBFactory.createDataset(location);
dataset.begin(ReadWrite.READ) ;
try{
QueryExecution ex = QueryExecutionFactory.create(query, dataset);
ResultSet res = ex.execSelect() ;
ResultSetFormatter.out(res) ;
} finally {
dataset.end() ;
}
******************************************************************
The total time taken in Windows was: 20 minutes 34seconds.The total time taken
in MAC was: 26 minutes 58seconds.The total time taken in online DBPedia
endpoint was seconds.My question is: what kind of instructions or stepsI miss
to get query results in a local TDB storage faster?
Thanks in advanceAbdul
Hi there,
Which version of Jena TDB are you using?
How big is the database? (in triples and in bytes)?
I tried all the illustrations below with Jena 2.13.0 which includes some
changes in the area.
Reformatting:
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dbpo: <http://dbpedia.org/ontology/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT DISTINCT ?lbl
WHERE
{ { ?entity1 ?rel ?entity2 }
UNION
{ ?entity2 ?rel ?entity1 }
?entity1 rdf:type dbpo:Person .
?entity2 rdf:type dbpo:Place .
?rel rdfs:label ?lbl
FILTER ( lang(?lbl) = "en" )
}
which is the SPARQL algebra:
http://www.sparql.org/query-validator.html
(prefix ((rdfs: <http://www.w3.org/2000/01/rdf-schema#>)
(dbpo: <http://dbpedia.org/ontology/>)
(rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>))
(distinct
(project (?lbl)
(filter (= (lang ?lbl) "en")
(join
(union
(bgp (triple ?entity1 ?rel ?entity2))
(bgp (triple ?entity2 ?rel ?entity1)))
(bgp
(triple ?entity1 rdf:type dbpo:Person)
(triple ?entity2 rdf:type dbpo:Place)
(triple ?rel rdfs:label ?lbl)
))))))
The ARQ/TDB optimizer isn't powerful enough to do some of the reordering
possible on this query. It does leave things alone where it is not
optimizing so rewriting the query can make a big difference. The query
has a cross product and a full scan (twice) in it. These can be avoided.
Support that
?rel rdfs:label ?lbl
FILTER ( lang(?lbl) = "en" )
is reasonably selective on properties. (the rdf:type may be more
selective.)
To do this first, you could try this query:
SELECT DISTINCT ?lbl
WHERE
{
{
?rel rdfs:label ?lbl
FILTER ( lang(?lbl) = "en" )
}
{ ?entity1 ?rel ?entity2 }
UNION
{ ?entity2 ?rel ?entity1 }
{
?entity1 rdf:type dbpo:Person .
?entity2 rdf:type dbpo:Place .
}
}
This part:
?entity1 rdf:type dbpo:Person .
?entity2 rdf:type dbpo:Place .
is possibly better merged with the union though (it's all down to your
data shape).
SELECT DISTINCT ?lbl
WHERE
{
{ ?entity1 ?rel ?entity2 .
?entity1 rdf:type dbpo:Person .
?entity2 rdf:type dbpo:Place .
}
UNION
{ ?entity2 ?rel ?entity1 .
?entity1 rdf:type dbpo:Person .
?entity2 rdf:type dbpo:Place .
}
{
?rel rdfs:label ?lbl
FILTER ( lang(?lbl) = "en" )
}
}
or
SELECT DISTINCT ?lbl
WHERE
{
{
?rel rdfs:label ?lbl
FILTER ( lang(?lbl) = "en" )
}
{ ?entity1 ?rel ?entity2 .
?entity1 rdf:type dbpo:Person .
?entity2 rdf:type dbpo:Place .
}
UNION
{ ?entity2 ?rel ?entity1 .
?entity1 rdf:type dbpo:Person .
?entity2 rdf:type dbpo:Place .
}
}
may be better (label/filter before or after the UNION.
Refining that: if there are less places than people:
SELECT DISTINCT ?lbl
WHERE
{
# Whether the "dbpo:Place" or the filter block is better
# needs experimentation.
?entity2 rdf:type dbpo:Place .
{
?rel rdfs:label ?lbl
FILTER ( lang(?lbl) = "en" )
}
{ ?entity1 ?rel ?entity2 .
}
UNION
{ ?entity2 ?rel ?entity1 .
}
?entity1 rdf:type dbpo:Person .
}
If you try these, could you please let us know what effect that have?
Finally, if you run your example just once, then you also incur the
database warming up costs as a program starts with no database cached.
Especially true with an HD, rather than an SSD.
Andy
