Sorry, my bad. The gcr cache was renamed to hpc in 2.0 so use cache_type=hpc
Can you check graph.db/messages.log or data/logs/* for an exception related to the missing node? > EntityNotFoundException: Node with id 462370 can you also try this for me? MATCH (n) where id(n) = 462370 return n Thanks Michael Am 17.12.2013 um 10:11 schrieb JDS <[email protected]>: > Hello Michael, > > Thanks for the advice. I've upgraded to 2.0.0 enterprise and set all the > settings: > > neostore.nodestore.db.mapped_memory=50M > neostore.relationshipstore.db.mapped_memory=756M > neostore.propertystore.db.mapped_memory=100M > neostore.propertystore.db.strings.mapped_memory=324M > neostore.propertystore.db.arrays.mapped_memory=50M > cache_type=gcr > > wrapper.java.initmemory=1536 > wrapper.java.maxmemory=3072 > > I have 1 new question and 1 problem/question: > > 1) Why is gcr not documented at > http://docs.neo4j.org/chunked/stable/configuration-caches.html#_object_cache > 2) I can't run the query as I'm getting an exception with a missing node of > some sort (do I need to rebuild my indexes?) > > neo4j-sh (?)$ PROFILE MATCH > (nt:NHS_TRUST)-[r1:HAS_NHS_TRUST_LOCATION]->(n:NHS_TRUST_LOCATION)<-[r2:IS_NHS_TRUST_LOCATION_INCIDENT]-(i:INCIDENT)-[r3:IS_NHS_TRUST_INCIDENT]->(nt) > USING SCAN nt:NHS_TRUST USING SCAN n:NHS_TRUST_LOCATION RETURN nt.name AS > nhs_trust, type(r1), n.location_level_01 AS nhs_trust_location, type(r2), > COUNT(i.incident_id) AS incident_count ORDER BY nhs_trust, nhs_trust_location; > EntityNotFoundException: Node with id 462370 > > On Tuesday, December 17, 2013 8:10:01 AM UTC, Michael Hunger wrote: > Update Neo4j to 2.0 > there have been huge improvements in that area. > > Please report back after that, then we can continue to improve your query. > > Still it shouldn't be that slow, you could also increase the rel-store mmio > to 500M. > > And perhaps use cache_type=gcr when you are using Enterprise anyway. > > Something that might help a tiny bit as well (test it after the update), is > to leave off the labels except the first if your relationships already make > the node-types unique. > > Cheers > > Michael > > Am 17.12.2013 um 08:58 schrieb JDS <[email protected]>: > >> I'm looking at two queries. They both return the same results, but the 1st >> one took an hour to run (when I tried to run it again recently it didn't >> finish after 8 hours), chomps the cpu, and ups the heap usage, while the >> other took less than 10 seconds and is barely noticeable. Note that I am >> loading incident data while these queries are running. What I'm trying to >> figure out is if there's a way I can improve my relationships, queries, or >> configuration to have the same results without having to use scan (i.e. am I >> missing something obvious) or is it just better to continue using scan? >> Would I also be right in thinking I should increase my >> neostore.relationshipstore.db.mapped_memory setting to gain better >> performance (looking at jconsole it looks like my current heap size is never >> exceeding 2G and is usually closer to about 1.7G, so I could tune down my >> init/max and give more memory to cache as well as other processes)? Also, >> another strange thing, watching jconsole, if I run the second query it seems >> to trigger a massive release of the heap if I run it while the first query >> has been running (which I'm assuming is causing the heap usage to increase), >> see attached screenshot for the jconsole capture. I saw a post saying that >> PROFILE documentation was on it's way, is that done now? >> >> Neo4j Instance: Neo4j 2.0.0 M06 Enterprise >> >> Server: VM with 2 x Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz & 8G's RAM >> running 2.6.32-358.18.1.el6.x86_64 >> >> Config Details: >> wrapper.java.initmemory=3584 >> wrapper.java.maxmemory=5632 >> >> neostore.nodestore.db.mapped_memory=50M >> neostore.relationshipstore.db.mapped_memory=100M >> neostore.propertystore.db.mapped_memory=180M >> neostore.propertystore.db.strings.mapped_memory=260M >> neostore.propertystore.db.arrays.mapped_memory=260M >> >> -rw-r--r--. 1 root root 63 Dec 16 20:35 >> /opt/neo4j/data/graph.db/neostore >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.id >> -rw-r--r--. 1 root root 50 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.labeltokenstore.db >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.labeltokenstore.db.id >> -rw-r--r--. 1 root root 418 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.labeltokenstore.db.names >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.labeltokenstore.db.names.id >> -rw-r--r--. 1 root root 10902528 Dec 16 20:35 >> /opt/neo4j/data/graph.db/neostore.nodestore.db >> -rw-r--r--. 1 root root 1681 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.nodestore.db.id >> -rw-r--r--. 1 root root 68 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.nodestore.db.labels >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.nodestore.db.labels.id >> -rw-r--r--. 1 root root 64591400 Dec 16 19:42 >> /opt/neo4j/data/graph.db/neostore.propertystore.db >> -rw-r--r--. 1 root root 128 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.arrays >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.arrays.id >> -rw-r--r--. 1 root root 1817 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.id >> -rw-r--r--. 1 root root 153 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.index >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.index.id >> -rw-r--r--. 1 root root 684 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.index.keys >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.index.keys.id >> -rw-r--r--. 1 root root 244170752 Dec 16 19:42 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.strings >> -rw-r--r--. 1 root root 73 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.propertystore.db.strings.id >> -rw-r--r--. 1 root root 524220279 Dec 16 20:35 >> /opt/neo4j/data/graph.db/neostore.relationshipstore.db >> -rw-r--r--. 1 root root 84203681 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.relationshipstore.db.id >> -rw-r--r--. 1 root root 45 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.relationshiptypestore.db >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.relationshiptypestore.db.id >> -rw-r--r--. 1 root root 380 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.relationshiptypestore.db.names >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.relationshiptypestore.db.names.id >> -rw-r--r--. 1 root root 1600 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.schemastore.db >> -rw-r--r--. 1 root root 9 Dec 15 20:50 >> /opt/neo4j/data/graph.db/neostore.schemastore.db.id >> >> Query without USING SCAN with PROFILE: >> >> PROFILE MATCH >> (nt:NHS_TRUST)-[r1:HAS_NHS_TRUST_LOCATION]->(n:NHS_TRUST_LOCATION)<-[r2:IS_NHS_TRUST_LOCATION_INCIDENT]-(i:INCIDENT)-[r3:IS_NHS_TRUST_INCIDENT]->(nt) >> RETURN nt.name AS nhs_trust, type(r1), n.location_level_01 AS >> nhs_trust_location, type(r2), COUNT(i.incident_id) AS incident_count ORDER >> BY nhs_trust, nhs_trust_location; >> >> (See attached screenshot for profile because now I can't even get it to >> finish running after 8 hours) >> >> Query with USING SCAN & PROFILE: >> >> PROFILE MATCH >> (nt:NHS_TRUST)-[r1:HAS_NHS_TRUST_LOCATION]->(n:NHS_TRUST_LOCATION)<-[r2:IS_NHS_TRUST_LOCATION_INCIDENT]-(i:INCIDENT)-[r3:IS_NHS_TRUST_INCIDENT]->(nt) >> USING SCAN nt:NHS_TRUST USING SCAN n:NHS_TRUST_LOCATION RETURNnt.name AS >> nhs_trust, type(r1), n.location_level_01 AS nhs_trust_location, type(r2), >> COUNT(i.incident_id) AS incident_count ORDER BY nhs_trust, >> nhs_trust_location; >> >> ColumnFilter(symKeys=["type(r1)", "type(r2)", "nhs_trust_location", " >> INTERNAL_AGGREGATEf02dc4cd-b519-4ab3-87a7-0aab228c7373", "nhs_trust"], >> returnItemNames=["nhs_trust", "type(r1)", "nhs_trust_location", "type(r2)", >> "incident_count"], _rows=48, _db_hits=0) >> Sort(descr=["SortItem(Cached(nhs_trust of type Any),true)", >> "SortItem(Cached(nhs_trust_location of type Any),true)"], _rows=48, >> _db_hits=0) >> EagerAggregation(keys=["Cached(nhs_trust of type Any)", "Cached(type(r1) >> of type String)", "Cached(nhs_trust_location of type Any)", "Cached(type(r2) >> of type String)"], aggregates=["( >> INTERNAL_AGGREGATEf02dc4cd-b519-4ab3-87a7-0aab228c7373,Count(Product(i,incident_id(5),true)))"], >> _rows=48, _db_hits=172287) >> Extract(symKeys=["n", "i", "r1", "r2", "r3", "nt"], >> exprKeys=["nhs_trust", "type(r1)", "nhs_trust_location", "type(r2)"], >> _rows=172287, _db_hits=344574) >> Filter(pred="(hasLabel(i:INCIDENT(3)) AND hasLabel(i:INCIDENT(3)))", >> _rows=172287, _db_hits=0) >> PatternMatch(g="(i)-['r3']-(nt),(i)-['r2']-(n),(nt)-['r1']-(n)", >> _rows=172287, _db_hits=0) >> Filter(pred="(hasLabel(n:NHS_TRUST_LOCATION(2)) AND >> hasLabel(n:NHS_TRUST_LOCATION(2)))", _rows=312, _db_hits=0) >> NodeByLabel(label="NHS_TRUST_LOCATION", identifier="n", >> _rows=312, _db_hits=0) >> Filter(pred="hasLabel(nt:NHS_TRUST(1))", _rows=26, _db_hits=0) >> NodeByLabel(label="NHS_TRUST", identifier="nt", _rows=26, >> _db_hits=0) >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> <Screenshot_2013-12-16-14-49-06.png><jconsole.png> > > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
