I am using Sparql queries via Dataset and QueryExecution. No in-memory
inference defined. The stats file with my predicates is attached. It has those
for Has_Id, Has_Dyspnea_Score and Has_Dysphagia_Score, but it apparently cannot
infer that the latter two are also Has_Finding and therefore can be used to
narrow down the result set.
-Wolfgang
-----Original Message-----
From: Andy Seaborne <[email protected]>
To: users <[email protected]>
Sent: Sat, May 18, 2013 1:17 pm
Subject: Re: Unexpectedly slow query
On 17/05/13 13:27, [email protected] wrote:
>
>
> I ran tdbstats again on the fully loaded triple store (with all the patient
data as individuals and their relationships). My properties appear now in the
stats file. But only the properties with explicit triples, not the inferred
parent properties. E.g. I am using the following property type hierarchy:
What is your inference setup?
>
> Has_Finding
> - Has_Dysnpea_Score
> - Has_Dysphagia_Score
> - Is_Dead
>
>
> There are no explicit triples stating e.g. that a patient Has_Finding
Dyspnea_Score_2. But there are triples using the sub-properties, e.g. Patient
Has_Dysnpea_Score Dyspnea_Score_2.
Can you share the stats file? I can't investigate the situation without
a test case.
>
> The stats file now contains entries for the sub-properties, but not for
Has_Finding.
>
> The execution plan changed slightly though, but the crucial triple patterns
are still in the "wrong" order.
>
> It used to be:
> (?pat <http://www.siemens.com/euroCAT/2011/8/euroCAT.owl/instances#Has_Id>
?patId)
The original stats file had no mention of #Has_Id and actaully said (at
the end) that missing predciates were to be counted as having zero
occurences. The optimizer puts one of these first because the rest of
the pattern will never be reached if it's accurate.
> (?pat rdf:type <http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Patient>)
> (?findingProp rdfs:subPropertyOf
> <http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Finding>)
> (?finding rdf:type ?findingType)
> (?pat ?findingProp ?finding)
>
> Now it is:
> (?findingProp rdfs:subPropertyOf
> <http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Finding>)
> (?pat rdf:type <http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Patient>)
> (?pat <http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Id> ?patId)
> (?finding rdf:type ?findingType)
> (?pat ?findingProp ?finding)
>
> The triple pattern for the Has_Finding sub-properties moved to the start, but
the crucial (?finding rdf:type ?findingType) is still evaluated before (?pat
?findingProp ?finding) -> the query is still taking a very long time.
>
> I can go ahead and use "fixed.opt" instead of "stats.opt", but I am still
interested in whether there is a solution to this problem. I am using Jena
2.10.0.
>
> Hope this info helps!
>
> -Wolfgang
>
>
>
>
> -----Original Message-----
> From: hueyl16 <[email protected]>
> To: users <[email protected]>
> Sent: Fri, May 17, 2013 1:43 pm
> Subject: Re: Unexpectedly slow query
>
>
> I was wondering about that too. I could only find entries related to NCI
terms. How or when is the stats file generated?
>
> I am using the .bat versions of the tdbloader for importing the NCIt first
> and
then my own ontology, which contains Has_Id and Has_Finding plus more.
>
>
> I also ran tdbstats once but it did not change the stats file, just printed
it.
>
>
>
>
>
>
>
> -----Original Message-----
> From: Andy Seaborne <[email protected]>
> To: users <[email protected]>
> Sent: Fri, May 17, 2013 1:25 pm
> Subject: Re: Unexpectedly slow query
>
>
> (I now have the stats file)
>
> Wolfgang,
>
> I don't see entries for ec:Has_Id and ec:Has_Finding.
>
> Andy
>
>
>
>
>
>
(stats
(meta
(timestamp
"2013-05-17T14:09:08.758+02:00"^^<http://www.w3.org/2001/XMLSchema#dateTime>)
(run@ "2013/05/17 14:09:08 MESZ")
(count 3589072))
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Accepted_Therapeutic_Use_For>
571)
(<http://www.w3.org/2002/07/owl#equivalentClass> 11809)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Fraction_Dose> 22873)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#PDQ_Open_Trial_Search_ID>
3299)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#miRBase_ID> 148)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Preferred_Name> 91430)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#valueMapping> 15)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#GO_Annotation> 6350)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Clinical_T-Stage>
11812)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Disease> 20277)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Tolerable_Level> 27)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Is_Alive> 4)
(<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> 556563)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Image_Link> 6)
(<http://www.w3.org/2002/07/owl#intersectionOf> 12034)
(<http://www.w3.org/2002/07/owl#versionInfo> 1)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#code> 91433)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#OLD_KIND> 3024)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#EntrezGene_ID> 2780)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Use_For> 341)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Fractions> 22873)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Clinical_N-Stage>
11652)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Nutrient> 55)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Dysphagia_Score> 8)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Dyspnea_Score> 7)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Micronutrient> 32)
(<http://www.w3.org/2000/01/rdf-schema#subPropertyOf> 53)
(<http://protege.stanford.edu/plugins/owl/protege#readOnly> 2)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#OLD_STATE> 7)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#INFOODS> 203)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#NSC_Code> 2316)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Age_At_Pathology>
20286)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Gender> 24203)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#IntegerValue> 83837)
(<http://www.w3.org/2000/01/rdf-schema#comment> 2)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#FULL_SYN> 273608)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Concept_In_Subset> 40777)
(<http://protege.stanford.edu/plugins/owl/protege#defaultLanguage> 1)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Has_NICHD_Parent> 897)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Legacy_Concept_Name>
72308)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Clinical_M-Stage>
12009)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#SNP_ID> 87)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#OLD_CHILD> 2663)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#OID> 7)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#FDA_Table> 1821)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Semantic_Type> 97429)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#MGI_Accession_ID> 154)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Contributing_Source>
24759)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#PID_ID> 169)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#PDQ_Closed_Trial_Search_ID>
3299)
(<http://www.w3.org/2000/01/rdf-schema#label> 91423)
(<http://www.w3.org/2002/07/owl#onProperty> 74872)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Age_Value> 20286)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#UMLS_CUI> 76670)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Is_Related_To_Endogenous_Product>
22)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#OLD_ROLE> 529)
(<http://www.w3.org/2002/07/owl#unionOf> 65)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Split_From> 1)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Has_Free_Acid_Or_Base_Form>
1775)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Macronutrient> 23)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#FDA_UNII_Code> 12191)
(<http://www.w3.org/2000/01/rdf-schema#range> 155)
(<http://www.w3.org/2002/07/owl#propertyDisjointWith> 1)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Is_Dead> 7)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#GenBank_Accession_Number>
2929)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Swiss_Prot> 2526)
(<http://www.w3.org/2002/07/owl#imports> 2)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#CAS_Registry> 11646)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Has_CDRH_Parent> 1995)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#CHEBI_ID> 3107)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#KEGG_ID> 237)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#ALT_DEFINITION> 21551)
(<http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> 61456)
(<http://www.w3.org/2002/07/owl#disjointWith> 196)
(<http://www.w3.org/2000/01/rdf-schema#domain> 150)
(<http://www.w3.org/2002/07/owl#someValuesFrom> 74872)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#ICD-O-3_Code> 1062)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Homologous_Gene> 205)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#ecName> 42)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Qualifier_Applies_To> 1)
(<http://www.w3.org/1999/02/22-rdf-syntax-ns#first> 61456)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Chemical_Formula> 9253)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Display_Name> 5626)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Date_Of_Death> 7)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Dose_Value> 45739)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Alive_Last_Checked> 4)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#OMIM_Number> 8168)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#DEFINITION> 53843)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Neoplastic_Status> 7614)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#OLD_ASSOCIATION> 11)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Id> 290525)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Zubrod_Perf_Status>
8482)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#PubMedID_Primary_Reference>
408)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#descendantOf> 827693)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Fractionation_Part>
22873)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Relative_Enzyme_Activity>
1)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#DesignNote> 4506)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#OLD_PARENT> 3589)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#DateTime_Value> 14125)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Has_Target> 389)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Has_Salt_Form> 1775)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Concept_Status> 4125)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#descendantPropertyOf> 332)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Quantity_Value> 22873)
(<http://www.w3.org/2000/01/rdf-schema#subClassOf> 133293)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#NCI_META_CUI> 10136)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Role_Has_Domain> 119)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Date_Of_Diagnosis>
14114)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#US_Recommended_Intake>
43)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Radiation_Therapy>
19035)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Gene_Encodes_Product>
1975)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Essential_Amino_Acid> 10)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Unit> 332)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#USDA_ID> 134)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Subsource> 1261)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#BioCarta_ID> 335)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#NCBI_Taxon_ID> 563)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Role_Has_Parent> 17)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#In_Clinical_Trial_For>
834)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Essential_Fatty_Acid> 6)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Extensible_List> 138)
(<http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#Role_Has_Range> 119)
(<http://www.siemens.com/euroCAT/2011/8/euroCAT.owl#Has_Frac_Cumulative_Dose>
22873)
(other 0))
