I must thank you very much Michael Hunger for the help you gave since I have fixed the slow query time I was experiencing previously. I went from queries taking 30+ minutes to sometimes a day to run now only run in milliseconds with a maximum of 10 seconds for queries returning 800 000 results. The issue was caused by the lack of indexes and labels as well as setting the store cache sizes for in memory caching. Now everythings running A OK!
On a side note, I was wondering if anyone knew if the Neo4j web admin graph visualization API is publicly available or not? I like the look and feel that the web admin interface gives for viewing graph data. The colors look really good too. I tried looking into some libraries and even checked out D3.js but If the neo4j javascript library is available then that will help cut down on the coding time. JP On Sunday, May 18, 2014 12:14:19 PM UTC-4, Michael Hunger wrote: > > No worries, we're here to help. > > I think it makes sense for you to explore the graph first, see which > labels, properties and relationships are there and what kind of domain > model they fill. > See here: jexp.de/blog/2014/02/exploring-an-unknown-neo4j-database/ (You > can also use the top-left sidebar of the browser to explore that > information) > > If your company has urgent need you can also contact Neo Technology for a > remote or local workshop on getting your work going again. > > While doing that you can attend our online training, see graphacademy.com > to learn all about cypher (shouldn't take more than a few hours), note that > our cypher reference card might also come handy. > http://neo4j.org/resources/cypher > > If you learned more about your domain and cypher, I'm sure you'll have no > problem writing performant queries. > > Just make sure, that you have the right indexes and constraints in place > (you can check them with the :schema command) in the browser. > > > > > > > > On Sun, May 18, 2014 at 3:48 PM, gfourhsean Ionessa < > [email protected] <javascript:>> wrote: > >> First I want to thank you for the quick reply. >> >> I think instead of wasting more of your time I will have to sit down and >> learn Neo4j's query language and understand the types of queries as well as >> the data stored in the database. I am a research student placed in a >> project where all those who have built the database have left and I am >> stuck with understanding what's in the database and how to query that >> information. Considering your point about how I should use graph queries >> instead of pulling the whole database from disk I believe a good start >> would be to learn the graph that I am dealing with then understanding the >> queries that obtain the information I need. >> >> Thank you for your time and have a nice day. >> >> JP >> >> >> On Sunday, May 18, 2014 8:43:25 AM UTC-4, Michael Hunger wrote: >> >>> You shouldn't remove information from your query, if need be anonymize >>> it or adapt it. Otherwise you can't get real help. >>> >>> What is the actual use-case you want to solve? What do you actually want >>> to do? >>> >>> The query you had will do a full graph scan, not something you want to >>> do. >>> >>> Make sure to use graph queries (i.e. ones that actually traverse >>> relationships). >>> >>> The query you have pulls all data in your database from disk and pushes >>> it through a Cypher query (i.e. 600M elements), certainly not a real >>> use-case. >>> >>> If you split your union into two queries, how long is the runtime of >>> each of the queries? >>> >>> for the second one I'd probably go with start r=rels(*) WHERE ... >>> >>> >>> >>> On Sun, May 18, 2014 at 6:12 AM, gfourhsean Ionessa < >>> [email protected]> wrote: >>> >>>> Hello Neo4j Guru's, >>>> >>>> I know this question has been asked already in previous posts (which I >>>> have read) but I feel like something else is causing my issue maybe. >>>> >>>> To lay out as much as I can, I have been assigned to manage a database >>>> based off of Neo4j version 2.0.0 with a database size of ~250 Gigabytes. >>>> The problem my team has been encountering is ridiculously slow database >>>> querying (from 30 minutes to an hour before receiving results). The >>>> queries >>>> are executed using the web front end on the webadmin page. Below are the >>>> details for the graph nodes and relationship details to get an ideal of >>>> the >>>> size of the database I am dealing with. I am wondering if the sheer size >>>> of >>>> the database is the cause for such a slow query performance or is there >>>> something else that may be going on which I must look into. >>>> >>>> Test Query (With actual label information removed for privacy reasons): >>>> >>>> MATCH (n) WHERE has(n.`person`) RETURN DISTINCT "node" as element, >>>> n.`person` AS `person` LIMIT 25 UNION ALL MATCH ()-[r]-() WHERE >>>> has(r.`person`) RETURN DISTINCT "relationship" AS element, r.`person` AS >>>> `person` LIMIT 25 >>>> >>>> *The server Configuration:* >>>> >>>> CPU's: 24x E5-2630 2.30 GHz >>>> Memory: 125 Gigabytes of DDR3 >>>> Hard disk: 5 Terrabytes >>>> >>>> *JVM Settings manually set (in neo4j-wrapper.conf):* >>>> >>>> # Initial Java Heap Size (in MB) >>>> wrapper.java.initmemory=16 >>>> >>>> # Maximum Java Heap Size (in MB) >>>> wrapper.java.maxmemory=32768 >>>> >>>> *Linux system settings manually set:* >>>> >>>> ulimit set to 40000 >>>> >>>> under /etc/sysctl.conf: >>>> vm.dirty_background_ratio = 50 >>>> vm.dirty_ratio = 80 >>>> >>>> *Node and relation information:* >>>> >>>> "description": "Estimates of the numbers of different kinds of Neo4j >>>> primitives", >>>> "description": "An estimation of the number of nodes used in this >>>> Neo4j instance", >>>> "name": "NumberOfNodeIdsInUse", >>>> "value": 255857300, >>>> >>>> "description": "An estimation of the number of relationships used >>>> in this Neo4j instance", >>>> "name": "NumberOfRelationshipIdsInUse", >>>> "value": 375613862, >>>> >>>> "description": "An estimation of the number of properties used in >>>> this Neo4j instance", >>>> "name": "NumberOfPropertyIdsInUse", >>>> "value": 1518707868, >>>> >>>> >>>> "description": "The number of relationship types used in this >>>> Neo4j instance", >>>> "name": "NumberOfRelationshipTypeIdsInUse", >>>> "value": 8, >>>> >>>> >>>> I want to apologize in advance for my lack of knowledge of Neo4j but I >>>> am willing to learn as much as I can to fix this issue. >>>> >>>> Sincerely, >>>> JP >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Neo4j" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
