On Fri, Jul 9, 2010 at 12:33 PM, <[email protected]> wrote: > Dear all, Dear you,
> I'm completely new to neo4j (and don't even really speak Java), I'm a sysadmin with poor object programing skill. It wasn't a problem to use neo4j, as the api is simple and clear. (No "enterprisy" factoryFactory.Proxy.Processor(thingy.stuff()). Just simple object) > but I have been struggling in vain for quite a while to get sensible > performance on my graph-data in MySQL and PostgreSQL. From your webpage and > other posts on the lists I got the great feeling that newbies are welcome > here, Friendly Greetings \o/ > so I hope it is all right if I tell you something about my data and what I > want to know about it so that maybe someone can tell whether I can actually > do this with neo4j. Yes, you can ! (c)(r)(tm) > My data is about 250 million separate graphs with a grand total of about 5 > billion nodes. > - The graphs are of a tree-like structure (many are actual trees, but not all > of them). > - Every graph has an id. > - Every node has 4 properties: It should work :) > 1. name (some names are very common, many occur only once or twice) > 2. category1 (there are about 40 different categories on this level) > 3. name-group (John, Jon, Jonathan form one group, many of the names that > occur only once get their own name group) > 4. category2 (there are about 10 different categories on this level) > - Every edge has one or two properties > 1. type (currently about 50 different ones) [obligatory] > 2. attribute [only there for 3 types; about 10 per cent of all edges] Please note that you can have many edge per node and many "type" of edge. Edges can have many properties. So no problems here. > [snip] > If no highlighting is done, we just return the ids. > If highlighting was done, let's say on n4.name, then I want all names that > occur in this position of any graph. I don't understand the "highlight" thing. [snip] > I hope I managed to make myself understood. If not, I am happy to draw some > graphs and upload them somewhere. > > I know that I will need indices on the name and name-family properties. Not > sure how well they would perform on the less selective properties, though. Lucene (prefered index engine for neo4j) is very powerfull :) [snip] > Performance: For many queries of the type outlined above, I have to wait for > more than two minutes on my SMALL dataset (6 million graphs, 100 million > nodes, 87 million edges) via PostgreSQL. For some it is more like 10 or 20 > minutes... I would prefer not to have to wait for more than 5 seconds on the > small dataset and 20 or 30 on the big dataset. I can try to help with postgresql (i'm postgresql DBA) but, imho, you should try with neo4j :) (and i'll love to hear your feedback about graph in pgsql vs graph in neo4j) I have a question : you say "250 millions separate graph". Each 250 millions graph are totally independant ? (okay, 250 millions shards are probably overkill but... well... just wondering :) ) *hugs* -- Laurent "ker2x" Laborde Sysadmin & DBA at http://www.over-blog.com/ _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
