Dear all, I spent the weekend learning a bit about Java and wrote a program that puts my small dataset into a neo4j database and creates a lucene index on the properties.
To be able to find my graphs, I created a subreference node. Then I created a node with the property graph_id for each graph I store. Then I create an edge from this node to every node in the respective graph, which results in a lot of edges. I have a few more questions on this: 1) Is it possible to have the property "id" attached to a graph without connecting every node to an id-node? 2) Data modelling: Many of my properties occur quite frequently, in particularly of course the 40 different values of category1 (see my first email below). Thus I could model them as nodes and have relationships point to them instead of using properties. Any idea which is better for graph-matching? 3) Graph-matching: I still need clarification on matching subgraphs without knowing a node. Can this be done? Is it possible to match a subgraph when I know properties of nodes? Is it possible to match a subgraph when I only know relationships but nothing at all about the nodes? If not: How do I efficiently traverse the graph to find such subgraphs? @Laurent: > I have a question : you say "250 millions separate graph". > Each 250 millions graph are totally independant ? (okay, 250 millions > shards are probably overkill but... well... just wondering :) ) Yes, they are independent as long as I do not make nodes out of the properties as discussed above. Looking forward to suggestions! Best regards, Jonathan On Fri, Jul 9, 2010 at 12:33 PM, <Gurkensalat at gmx.de> wrote: Dear all, I'm completely new to neo4j (and don't even really speak Java), but I have been struggling in vain for quite a while to get sensible performance on my graph-data in MySQL and PostgreSQL. From your webpage and other posts on the lists I got the great feeling that newbies are welcome here, so I hope it is all right if I tell you something about my data and what I want to know about it so that maybe someone can tell whether I can actually do this with neo4j. My data is about 250 million separate graphs with a grand total of about 5 billion nodes. - The graphs are of a tree-like structure (many are actual trees, but not all of them). - Every graph has an id. - Every node has 4 properties: 1. name (some names are very common, many occur only once or twice) 2. category1 (there are about 40 different categories on this level) 3. name-group (John, Jon, Jonathan form one group, many of the names that occur only once get their own name group) 4. category2 (there are about 10 different categories on this level) - Every edge has one or two properties 1. type (currently about 50 different ones) [obligatory] 2. attribute [only there for 3 types; about 10 per cent of all edges] There are two sorts of questions I want to be able to answer: 1. The user specifies a subgraph (currently even a subtree, but not sure whether it will remain that way) and wants the ids of all matching graphs. 2. The user specifies a subgraph and highlights one position he didn't fill in. As a result, he wants a list of all items that occur in this position ordered by their frequency in this position. Examples of queries: (sorry for the weird format, but I have no idea how to represent a tree in text) EXAMPLE 1: relations: n1 ----> n2 (relation type: t1) n1 ----> n3 (relation type: t2) n3 ----> n4 (relation type: t3) n3 ----> n5 (relation type: t3) n3 ----> n6 (relation type not specified, just has to exist) node properties: n1: name-group: John-like; category2: c2-13 n2: [no properties specified, just has to exist] n3: category1: c1-15 n4: [no properties specified, just has to exist] n5: [no properties specified, just has to exist] n6: name: Ben; category2: c2-13 If no highlighting is done, we just return the ids. If highlighting was done, let's say on n4.name, then I want all names that occur in this position of any graph. EXAMPLE 2: relations: n1 ----> n2 (relation type: t1) n1 ----> n3 (relation type: t2) n3 ----> n4 (relation type: t3) no node properties specified. I hope I managed to make myself understood. If not, I am happy to draw some graphs and upload them somewhere. I know that I will need indices on the name and name-family properties. Not sure how well they would perform on the less selective properties, though. Basically, my problem is similar to the one found here: http://lists.neo4j.org/pipermail/user/2009-June/001331.html But what makes me worry is a quote from here: http://components.neo4j.org/neo4j-graph-matching/ "The pattern matching is done by first defining a graph pattern and then searching for matching occurrences of that pattern in the graph around a given anchor node." I do not necessarily have an anchor node. And I have lots of graphs... Performance: For many queries of the type outlined above, I have to wait for more than two minutes on my SMALL dataset (6 million graphs, 100 million nodes, 87 million edges) via PostgreSQL. For some it is more like 10 or 20 minutes... I would prefer not to have to wait for more than 5 seconds on the small dataset and 20 or 30 on the big dataset. Sorry for the lengthy email and I'm looking forward to your replies! Best regards, Jonathan -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
