Pieter, by using an indexing structure on these relationships, you could possibly break up your query into chunks that only hit certain parts of the structure. So, if you for instance index along a "time" property and ask for the last day, only the chunks of the (tree) index structure that contain the time range for the last day will be traversed in the graph. Combine that with paging (persisting the result set for a certain amount of time and giving you URLs to it), you can minimize the amount of data hit by the query.
Not sure it works like that in your dataset though ... Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sun, Jul 24, 2011 at 11:52 PM, Pieter Martin <[email protected]> wrote: > Hi, > > Is Jim's paging support in 1.4 or shall I checkout a snapshot to have a > look at it? > > Regarding the millions of relationships, it is for now more of a generic > question for a framework we are working on. Paging almost always becomes > a issue. Certainly we try avoid such relationships. > > Yes we can almost always think of a way to index the relationship, but I > do not quite understand how in-graph indexing will help? Will neo4j > traverser be able to hit such a index and page the result? > > Thanks > Pieter > > On 24/07/2011 15:20, Peter Neubauer wrote: >> Hi Pieter, >> Jim recently added support for paging in the Neo4j server for traversals, >> >> For Cypher, there is SKIP and LIMIT, together acting as a kind of >> paging system, but executing the query again (just giving you a >> different chunk of the result), so I am not sure that works for your >> case. http://docs.neo4j.org/chunked/snapshot/cypher-query-lang.html. >> Gremlin supports similar things. >> >> Also, given that you have millions of relationships on one node, could >> you think of some way of doing some in-graph or other indexing on >> these structures? Can you break up the relationships along one or two >> properties like time or a scalar range? Otherwise, that query seems to >> be quite expensive. Do you need all results? >> >> Cheers, >> >> /peter neubauer >> >> GTalk: neubauer.peter >> Skype peter.neubauer >> Phone +46 704 106975 >> LinkedIn http://www.linkedin.com/in/neubauer >> Twitter http://twitter.com/peterneubauer >> >> http://www.neo4j.org - Your high performance graph database. >> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >> >> >> >> On Sat, Jul 16, 2011 at 6:31 PM, Pieter Martin<[email protected]> >> wrote: >>> Hi, >>> >>> I am trying out tinkerpop / neo4j and wondering if there is any support >>> for paging. >>> >>> We frequently have the requirement to traverse a one to many >>> relationship with millions of nodes on the many side. Previously using >>> JPA we would execute a sql query using the LIMIT keyword to return a >>> paged result set. >>> >>> Can a similar result be achieved using neo4j. So far all I have been >>> able to find on paging is in the rest api. However I am running neo4j in >>> the embedded mode. >>> >>> Thanks >>> Pieter >>> _______________________________________________ >>> Neo4j mailing list >>> [email protected] >>> https://lists.neo4j.org/mailman/listinfo/user >>> >> _______________________________________________ >> Neo4j mailing list >> [email protected] >> https://lists.neo4j.org/mailman/listinfo/user > > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
