Re: [Neo4j] Path finding
Hi John, One option could be to model the graph with 2 identical spatial layers and add walking relationships between the layers. Similar to the attached pic, its from http://bit.ly/hJN2BB and then only one walking relationship could be traversed. Cheers Paddy On Mon, Apr 11, 2011 at 7:50 AM, John Doran john.do...@hotmail.com wrote: Ive set up a graph to represent a transport network. A bus stop may be connected to others with a walk relationship. All nodes have relationships with dist and time. The purpose of it is to plan people's routes. I'm using aStar, and have tried using dijkstra but they both return similar results. I'm using distance as the cost evaluator. My l problem is yes the shortest path being returned is the quickest but it may mean jumping onto 5 separate buses(perhaps I should add a time penalty when changing off your current route). Another option may be an analysis of the actual transfer patterns returned from the shortest path query. Use the one with the least amount of transfers. My question is could I provide some kind o heuristic to encourage staying on a route as far as it can go? Peter mentioned Hop-path to me when discussing. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user attachment: screenshot.png___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Path finding
Hi, I think it can be helpful for controlling the amount of transfers to take, When I have two layers with walking connections between the layers every path which reaches the second layer will contain only one transfer. But the drawback is that you also have to search for end nodes on both layers. Previously I used one layer with an evaluator which counted how many transfer relationships were in a path and pruned the path if it contained more than one transfer relationship. But maybe a better approach would be to store a counter of the amount transfer relationships contained in a path? Cheers Paddy On Mon, Apr 11, 2011 at 2:45 PM, Mattias Persson matt...@neotechnology.comwrote: Hi Paddy, how would that help in decision-making of which route is the shortest when also taking into considering the amount of transfers? 2011/4/11 Paddy paddyf...@gmail.com: Hi John, One option could be to model the graph with 2 identical spatial layers and add walking relationships between the layers. Similar to the attached pic, its from http://bit.ly/hJN2BB and then only one walking relationship could be traversed. Cheers Paddy On Mon, Apr 11, 2011 at 7:50 AM, John Doran john.do...@hotmail.com wrote: Ive set up a graph to represent a transport network. A bus stop may be connected to others with a walk relationship. All nodes have relationships with dist and time. The purpose of it is to plan people's routes. I'm using aStar, and have tried using dijkstra but they both return similar results. I'm using distance as the cost evaluator. My l problem is yes the shortest path being returned is the quickest but it may mean jumping onto 5 separate buses(perhaps I should add a time penalty when changing off your current route). Another option may be an analysis of the actual transfer patterns returned from the shortest path query. Use the one with the least amount of transfers. My question is could I provide some kind o heuristic to encourage staying on a route as far as it can go? Peter mentioned Hop-path to me when discussing. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Populating a RelationshipExpander
Hi, How could I populate a RelationshipExpander directly with RelationshipType[] Direction[] propertys. As the Traversal.expanderForTypes takes varargs, How can I do something along the lines of below or should I implement an Expander? RelationshipType[] types = relationshipsToExpand(); Direction[] directions; // All are Direction.OUTGOING RelationshipExpander rel = Traversal.expanderForTypes( types, directions); Thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Populating a RelationshipExpander
Sorry I just seen a previous mail, explaining that it is an immutable class I was previously trying with Expander expander = Traversal.emptyExpander(); expander.add( types[i] ,directions[i]); when I should of been using expander = expander.add( types[i] ,directions[i]); Now it works :) Cheers Pad On Thu, Mar 10, 2011 at 6:43 PM, Paddy paddyf...@gmail.com wrote: Hi, How could I populate a RelationshipExpander directly with RelationshipType[] Direction[] propertys. As the Traversal.expanderForTypes takes varargs, How can I do something along the lines of below or should I implement an Expander? RelationshipType[] types = relationshipsToExpand(); Direction[] directions; // All are Direction.OUTGOING RelationshipExpander rel = Traversal.expanderForTypes( types, directions); Thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Finding All Available Paths
Hi Saikat, I have come across this problem too. If its possible to define the relationships between the nodes as a Train, Bus or Walking relationship. You could use a RelationshipExpander to define which relationship types you wish to Traverse when using the findAllPaths. Regards, Paddy On Fri, Feb 4, 2011 at 8:17 AM, Saikat Kanjilal sxk1...@hotmail.com wrote: Peter/Mattias, Thanks for your help, a few items to further clarify: 1) When I use findAllPaths with the Dijkstra algorithm and lets say only one path exists between two nodes and I proceed to print out all the paths using an iterator the iterator prints the same path many number of times and then crashes (essentially runs out of heap space) 2) Before I dive into the javadocs I want to kind of step back and explain what I'm trying to accomplish, I have a graph of which I have computed the shortest path successfully between any two nodes based on a weighted cost which is in this case is the length of the path, however what i want is the shortest path and any paths that have nodes that fall on a train line, in my case I have defined a property for each node identifying whether its on a train route or a bus route or a regular walking route. I am wondering whether I should be using findAllPaths at all to do this, what I am thinking is maybe to use a nodeFilter and do pattern matching on the graph to extract all paths that have train route nodes in it. Given that I'd like some insight as to whether a nodeFilter is appropriate for this or whether I use findAllPaths and then search for paths containing nodes on a train route and extract the paths that way. I will actually upload my whole project (test cases and source code) to github once I get some direction on point number 2. Best Regards Date: Fri, 4 Feb 2011 09:39:00 +0100 From: peter.neuba...@neotechnology.com To: user@lists.neo4j.org Subject: Re: [Neo4j] Finding All Available Paths Saikat, do you have some failing test case that should pass? You having these problems might mean that there is a bug somewhere in the Dijkstra, so if you feel that something is crashing or wrong, could you recreate the failure in a testcase and put it on GIThub GIST or send it to the list so we can take a look at it? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Feb 4, 2011 at 3:53 AM, Saikat Kanjilal sxk1...@hotmail.com wrote: Hi Folks,I'm slowly but surely climbing the learning path in the neo4j world. I'm using Dijkstra's algorithm for the shortest path calculation between two nodes which now seems to be working fine after some code rearranging, however I'd also like to get the list of available paths between two nodes regardless of the path length. I tried to use the findAllPaths function for the dijkstraFinder but I noticed that if there is only one available path between two nodes it returns this same path multiple times and then crashes (this only happens when there is only 1 path between 2 nodes). I was wondering whether I should be using some other finder like the finders that dont use weighted paths to get this information. If not some help would be much appreciated here, is findAllPaths supposed to return the same path multiple times. Thanks in advance ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Finding All Available Paths
Hi Saikat, I'm glad you like it, it would be great to compare approaches. I'll try to create some example code of transit routing with time and using transfer patterns over the weekend. Cheers, Paddy On Fri, Feb 4, 2011 at 1:37 PM, Saikat Kanjilal sxk1...@hotmail.com wrote: Hi Paddy,I've downloaded your work with the bus transit project from github and am keenly looking through it, very interesting, I like your idea about the relationship expander, I'll experiment with that, however it seems so far from my tests that there may be a bug in findAllPaths for the edge case where there is only one path between two nodes for the Dijkstra path finder. Peter I am assuming that you guys have unit tested this method (as well as load tested it), is that correct? I looked in your repository and didn't see any load tests that targeted this method specifically. Also I am trying to understand the tradeoffs between taking the nodefilter approach versus using the relationship expander that Paddy outlined, it seems at first glance that the relationship expander is an easier win. Regards Date: Fri, 4 Feb 2011 22:17:35 +0100 From: peter.neuba...@neotechnology.com To: user@lists.neo4j.org Subject: Re: [Neo4j] Finding All Available Paths Maybe there is a lack of examples? Paddy, if you can want, fork https://github.com/neo4j/examples and add some example code in there, I can merge it back, or just make a GIST that I can pull in to document this better for Saikat? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Feb 4, 2011 at 10:13 PM, Paddy paddyf...@gmail.com wrote: Hi Saikat, I have come across this problem too. If its possible to define the relationships between the nodes as a Train, Bus or Walking relationship. You could use a RelationshipExpander to define which relationship types you wish to Traverse when using the findAllPaths. Regards, Paddy On Fri, Feb 4, 2011 at 8:17 AM, Saikat Kanjilal sxk1...@hotmail.com wrote: Peter/Mattias, Thanks for your help, a few items to further clarify: 1) When I use findAllPaths with the Dijkstra algorithm and lets say only one path exists between two nodes and I proceed to print out all the paths using an iterator the iterator prints the same path many number of times and then crashes (essentially runs out of heap space) 2) Before I dive into the javadocs I want to kind of step back and explain what I'm trying to accomplish, I have a graph of which I have computed the shortest path successfully between any two nodes based on a weighted cost which is in this case is the length of the path, however what i want is the shortest path and any paths that have nodes that fall on a train line, in my case I have defined a property for each node identifying whether its on a train route or a bus route or a regular walking route. I am wondering whether I should be using findAllPaths at all to do this, what I am thinking is maybe to use a nodeFilter and do pattern matching on the graph to extract all paths that have train route nodes in it. Given that I'd like some insight as to whether a nodeFilter is appropriate for this or whether I use findAllPaths and then search for paths containing nodes on a train route and extract the paths that way. I will actually upload my whole project (test cases and source code) to github once I get some direction on point number 2. Best Regards Date: Fri, 4 Feb 2011 09:39:00 +0100 From: peter.neuba...@neotechnology.com To: user@lists.neo4j.org Subject: Re: [Neo4j] Finding All Available Paths Saikat, do you have some failing test case that should pass? You having these problems might mean that there is a bug somewhere in the Dijkstra, so if you feel that something is crashing or wrong, could you recreate the failure in a testcase and put it on GIThub GIST or send it to the list so we can take a look at it? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Feb 4, 2011 at 3:53 AM, Saikat Kanjilal sxk1...@hotmail.com wrote: Hi Folks,I'm slowly but surely climbing the learning path in the neo4j world. I'm using Dijkstra's algorithm
Re: [Neo4j] Neo4j + MapReduce
Hi, Would you like to create map-reduce views using only the graph nodes? That would be very interesting. Could you maybe provide a sample map-reduce process? One possibility might be to store all the data in a neo4j graph. Run the batch processing on the graph using multiple Mappers and build a new neo4j graph in one Reducer. Regards, Paddy On Mon, Jan 31, 2011 at 8:43 AM, Saikat Kanjilal sxk1...@hotmail.comwrote: John,If you havent already take a look at some of the algorithms in mahout, they are currently using hadoop (http://mahout.apache.org) and it may be the case where some of their algorithms are taylored more towards hadoop style data-stores. Regards Date: Mon, 31 Jan 2011 17:34:25 +0100 From: peter.neuba...@neotechnology.com To: user@lists.neo4j.org Subject: Re: [Neo4j] Neo4j + MapReduce John, Paddy Fitzgerald has been doing some great work on this theme, see https://github.com/paddydub/NeoHadoopTester for details. Paddy, do you have any additional pointers? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Jan 31, 2011 at 5:03 PM, yehohanan7 yehohan...@gmail.com wrote: Hi All, I am a neo4j newbie and haven't yet started to use it. I am basically trying to implement a recommendation engine to learn and understand machine learning algorithms so thought neo4j would be a great choice. however, I just want to understand if mapreduce feature will be provided to process the data stored in neo4j? i might be completely wrong in putting this question but what if i have an use-case where i don't need to traverse but process all the nodes irrespective of its relation to other nodes? Is it a wise idea to store data on a document database and relations/associations in neo4j? or should it be done as a batch processing where huge amount of data in a document database and run map-reduce on it to build neo4j DB as a daily job? I am new to mapreduce and grapdb, but trying to see how best i can use (learn) them as both has unique features... Thanks in advance, John -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/Neo4j-MapReduce-tp2389817p2389817.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Accessing DB data directory on Elastic MapReduce
Hi Peter, I have uploaded what I have so far. Its very early stages. I have added a bit of detail in the README, there are a few steps to get it going, hopefully it works. https://github.com/paddydub/NeoHadoopTester%20https://github.com/paddydub/NeoHadoopTester I'm testing using a Hadoop job with 50 mappers with the stopsListSmall file as input. To precompute all the transfer patterns I would need to use 15000 mappers using the stopsListLarge input file. I haven't had a chance to do much testing but if it takes roughly 7.5 minutes per mapper. To find the shortest path from the first departure event of the inputted stop, to any stop with Dijkstra's algorithm would take on average 30 milliseconds. (0.03sec x 15000 x 15000) = 1875 instance hours. The cost to run this on Elastic MapReduce is approx: (1875 hrs x $0.015) = $28.75 Running on 20 instances (my limit at the moment) would take about 93 hours or 2 hours running on 1000 instances. :) I have one Reducer/BatchInserter which will iterate through the results of all the Mappers and will convert the list of patterns into a DAG and insert them into a new graph db. I still have to rework this step though. Then output the new database back to s3 or even access an EBS volume directly if possible. Please let me know any suggestions on how to improve or speed it up. cheers, Paddy On Sun, Jan 2, 2011 at 12:44 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Very cool! Is this part of the GITHub setup? And, are you inserting the precomputed patterns into the graph in a later step then? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sat, Jan 1, 2011 at 11:38 PM, Paddy paddyf...@gmail.com wrote: Hi, Can I just add that setting another bootstrap action to set how many tasks are run per machine helped: --site-key-value mapred.tasktracker.map.tasks.maximum=1 . By default it is set to 2 for small ec2 instances and I was running into the Unable to lock store [/home/hadoop/neo-db/neostore] error for some jobs. Just tested running Neo4j graph-algo parallel on 20 machines, sweet :) cheers, Paddy On Sat, Nov 27, 2010 at 4:16 PM, Paddy paddyf...@gmail.com wrote: Hi, I'm a very much a Hadoop newbie but I think that would be very possible, maybe even making use of: http://incubator.apache.org/whirr/ to ensure capability with Amazon EC2 Rackspace Cloud Servers. Would creating custom types e.g. NodeWritable, WeightedPathWritable which implement org.apache.hadoop.io.Writable, be a good method to serialize and deserialize Neo4j objects between jobs? http://developer.yahoo.com/hadoop/tutorial/module5.html#types It would also be interesting to chain multiple traversals as Map-Reduce jobs. cheers Paddy On Fri, Nov 26, 2010 at 9:00 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: This looks good to me, but then, I am no Amazon and Hadoop expert. Do you think it would be possible to do a generic integration component that lets you run travesals on replicated Neo4j backends as Hadoop map-reduce jobs? I think that would be interesting to a number of use cases and a very cool use of Neo4j. Also, Alex Averbuch wants to look at using AKKA (Scala) to do similar things, so comparing the approaches would be great! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Nov 25, 2010 at 6:33 AM, Paddy paddyf...@gmail.com wrote: Hi Guys, I was testing out accessing a Neo4j DB from a Hadoop job on Elastic Mapreduce, I asked a question on the forum regarding loading a file from s3 to each ec2 at startup: https://forums.aws.amazon.com/thread.jspa?threadID=54919 In case anyone faces the same issue, the following bootstrap action will download a compressed neo4j database from a s3 bucket to each launched ec2 instance and changes the directory permissions to allow access with: private static GraphDatabaseService graphDb = new EmbeddedGraphDatabase(/home/hadoop/neo-db); #!/bin/bash set -e sudo wget -S -T 10 -t 5 http://yourbucket. s3.amazonaws.com/neo-db.tar.gz sudo tar -C /home/hadoop -xzf neo-db.tar.gz sudo chmod -R 777 /home/hadoop/neo-db Does this sound like the best method to use? cheers Paddy ___ Neo4j
Re: [Neo4j] Accessing DB data directory on Elastic MapReduce
Hi, Can I just add that setting another bootstrap action to set how many tasks are run per machine helped: --site-key-value mapred.tasktracker.map.tasks.maximum=1 . By default it is set to 2 for small ec2 instances and I was running into the Unable to lock store [/home/hadoop/neo-db/neostore] error for some jobs. Just tested running Neo4j graph-algo parallel on 20 machines, sweet :) cheers, Paddy On Sat, Nov 27, 2010 at 4:16 PM, Paddy paddyf...@gmail.com wrote: Hi, I'm a very much a Hadoop newbie but I think that would be very possible, maybe even making use of: http://incubator.apache.org/whirr/ to ensure capability with Amazon EC2 Rackspace Cloud Servers. Would creating custom types e.g. NodeWritable, WeightedPathWritable which implement org.apache.hadoop.io.Writable, be a good method to serialize and deserialize Neo4j objects between jobs? http://developer.yahoo.com/hadoop/tutorial/module5.html#types It would also be interesting to chain multiple traversals as Map-Reduce jobs. cheers Paddy On Fri, Nov 26, 2010 at 9:00 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: This looks good to me, but then, I am no Amazon and Hadoop expert. Do you think it would be possible to do a generic integration component that lets you run travesals on replicated Neo4j backends as Hadoop map-reduce jobs? I think that would be interesting to a number of use cases and a very cool use of Neo4j. Also, Alex Averbuch wants to look at using AKKA (Scala) to do similar things, so comparing the approaches would be great! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Nov 25, 2010 at 6:33 AM, Paddy paddyf...@gmail.com wrote: Hi Guys, I was testing out accessing a Neo4j DB from a Hadoop job on Elastic Mapreduce, I asked a question on the forum regarding loading a file from s3 to each ec2 at startup: https://forums.aws.amazon.com/thread.jspa?threadID=54919 In case anyone faces the same issue, the following bootstrap action will download a compressed neo4j database from a s3 bucket to each launched ec2 instance and changes the directory permissions to allow access with: private static GraphDatabaseService graphDb = new EmbeddedGraphDatabase(/home/hadoop/neo-db); #!/bin/bash set -e sudo wget -S -T 10 -t 5 http://yourbucket. s3.amazonaws.com/neo-db.tar.gz sudo tar -C /home/hadoop -xzf neo-db.tar.gz sudo chmod -R 777 /home/hadoop/neo-db Does this sound like the best method to use? cheers Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Accessing DB data directory on Elastic MapReduce
Hi, I'm a very much a Hadoop newbie but I think that would be very possible, maybe even making use of: http://incubator.apache.org/whirr/ to ensure capability with Amazon EC2 Rackspace Cloud Servers. Would creating custom types e.g. NodeWritable, WeightedPathWritable which implement org.apache.hadoop.io.Writable, be a good method to serialize and deserialize Neo4j objects between jobs? http://developer.yahoo.com/hadoop/tutorial/module5.html#types It would also be interesting to chain multiple traversals as Map-Reduce jobs. cheers Paddy On Fri, Nov 26, 2010 at 9:00 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: This looks good to me, but then, I am no Amazon and Hadoop expert. Do you think it would be possible to do a generic integration component that lets you run travesals on replicated Neo4j backends as Hadoop map-reduce jobs? I think that would be interesting to a number of use cases and a very cool use of Neo4j. Also, Alex Averbuch wants to look at using AKKA (Scala) to do similar things, so comparing the approaches would be great! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Nov 25, 2010 at 6:33 AM, Paddy paddyf...@gmail.com wrote: Hi Guys, I was testing out accessing a Neo4j DB from a Hadoop job on Elastic Mapreduce, I asked a question on the forum regarding loading a file from s3 to each ec2 at startup: https://forums.aws.amazon.com/thread.jspa?threadID=54919 In case anyone faces the same issue, the following bootstrap action will download a compressed neo4j database from a s3 bucket to each launched ec2 instance and changes the directory permissions to allow access with: private static GraphDatabaseService graphDb = new EmbeddedGraphDatabase(/home/hadoop/neo-db); #!/bin/bash set -e sudo wget -S -T 10 -t 5 http://yourbucket. s3.amazonaws.com/neo-db.tar.gz sudo tar -C /home/hadoop -xzf neo-db.tar.gz sudo chmod -R 777 /home/hadoop/neo-db Does this sound like the best method to use? cheers Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Accessing DB data directory on Elastic MapReduce
Hi Guys, I was testing out accessing a Neo4j DB from a Hadoop job on Elastic Mapreduce, I asked a question on the forum regarding loading a file from s3 to each ec2 at startup: https://forums.aws.amazon.com/thread.jspa?threadID=54919 In case anyone faces the same issue, the following bootstrap action will download a compressed neo4j database from a s3 bucket to each launched ec2 instance and changes the directory permissions to allow access with: private static GraphDatabaseService graphDb = new EmbeddedGraphDatabase(/home/hadoop/neo-db); #!/bin/bash set -e sudo wget -S -T 10 -t 5 http://yourbucket.s3.amazonaws.com/neo-db.tar.gz sudo tar -C /home/hadoop -xzf neo-db.tar.gz sudo chmod -R 777 /home/hadoop/neo-db Does this sound like the best method to use? cheers Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] BatchInserter - No index provider 'null' error
good stuff, thanks Mattias
On Tue, Oct 26, 2010 at 12:49 AM, Mattias Persson matt...@neotechnology.com
wrote:
Ah, it's a bug allright... only when using the BatchInserterIndex. The
problem goes away if you add provider:lucene to the configuration map,
like:
persons= indexProvider.nodeIndex(persons,
MapUtil.stringMap(type,exact,
provider, lucene));
It will be fixed in M03, where the LuceneBatchInserterIndexProvider fills
that in for you.
2010/10/26 Peter Neubauer peter.neuba...@neotechnology.com
Paddy,
just tested it, and it seems it only occurs when running standalone,
see the other mail. Seems to be a bug, and I think it is linked to the
behavior seen in the Python bindings, too.
Cheers,
/peter neubauer
VP Product Management, Neo Technology
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph
database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Tue, Oct 26, 2010 at 3:56 AM, Paddy paddyf...@gmail.com wrote:
Hi,
I'm trying out the new Index Framework Batch Inserter using version
1.2.M02.
I getting a No index provider 'null' found error when calling
graphDb.index().forNodes(persons) in my test code below
Exception in thread main java.lang.IllegalArgumentException: No index
provider 'null' found
at
org.neo4j.kernel.IndexManagerImpl.getIndexProvider(IndexManagerImpl.java:65)
at
org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:205)
at
ie.transportdublin.batchinserter.BatchIndexTest.main(BatchIndexTest.java:40)
From debugging it looks like config.get( KEY_INDEX_PROVIDER )@ L205 in
IndexManagerImpl
is returning a null value.
Or please let me know if I should be doing this a different way.
thanks
Paddy
public class BatchIndexTest {
static final String DB_PATH = data/neo-db-test10;
static BatchInserterIndexProvider indexProvider;
static BatchInserter inserter;
static BatchInserterIndex persons ;
static GraphDatabaseService graphDb;
public static void main(String[] args) {
inserter = new BatchInserterImpl(DB_PATH);
indexProvider = new LuceneBatchInserterIndexProvider(inserter);
persons= indexProvider.nodeIndex(persons, MapUtil.stringMap(type,
exact));
MapString, Object properties = MapUtil.map(name, test);
long node = inserter.createNode(properties);
persons.add(node, properties);
indexProvider.shutdown();
inserter.shutdown();
graphDb = new EmbeddedGraphDatabase(DB_PATH);
Transaction tx = graphDb.beginTx();
try {
IndexManager indexManager = graphDb.index();
System.out.println(indexManager.forRelationships(persons));
System.out.println(indexManager.existsForNodes(persons)); //returns
true
//Getting an error with the following code
System.out.println(indexManager.forNodes(persons));
tx.success();
} finally {
tx.finish();
graphDb.shutdown();
}
}
}
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] BatchInserter - No index provider 'null' error
Hi,
I'm trying out the new Index Framework Batch Inserter using version 1.2.M02.
I getting a No index provider 'null' found error when calling
graphDb.index().forNodes(persons) in my test code below
Exception in thread main java.lang.IllegalArgumentException: No index
provider 'null' found
at
org.neo4j.kernel.IndexManagerImpl.getIndexProvider(IndexManagerImpl.java:65)
at org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:205)
at
ie.transportdublin.batchinserter.BatchIndexTest.main(BatchIndexTest.java:40)
From debugging it looks like config.get( KEY_INDEX_PROVIDER )@ L205 in
IndexManagerImpl
is returning a null value.
Or please let me know if I should be doing this a different way.
thanks
Paddy
public class BatchIndexTest {
static final String DB_PATH = data/neo-db-test10;
static BatchInserterIndexProvider indexProvider;
static BatchInserter inserter;
static BatchInserterIndex persons ;
static GraphDatabaseService graphDb;
public static void main(String[] args) {
inserter = new BatchInserterImpl(DB_PATH);
indexProvider = new LuceneBatchInserterIndexProvider(inserter);
persons= indexProvider.nodeIndex(persons, MapUtil.stringMap(type,
exact));
MapString, Object properties = MapUtil.map(name, test);
long node = inserter.createNode(properties);
persons.add(node, properties);
indexProvider.shutdown();
inserter.shutdown();
graphDb = new EmbeddedGraphDatabase(DB_PATH);
Transaction tx = graphDb.beginTx();
try {
IndexManager indexManager = graphDb.index();
System.out.println(indexManager.forRelationships(persons));
System.out.println(indexManager.existsForNodes(persons)); //returns true
//Getting an error with the following code
System.out.println(indexManager.forNodes(persons));
tx.success();
} finally {
tx.finish();
graphDb.shutdown();
}
}
}
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Generating a directed acyclic graph
Hi Peter, Yes It would be nice to implement the paper fully, it's definitely possible to do using neo4j. . 1. Create a time expanded network 2. Run the Dijkstra to find all transfer patterns 3. Construct a directed acyclic graph from each stop using the transfer patterns 4. Running a time dependent Dijkstra search on the directed acyclic graphs, I used the path weight to keep track of time It is then possible to use the CostEvaluator to look up a direct-connection query a...@t - B based on the path weight. I have experimented with each step separately, except I'm not sure how to construct the directed acyclic graph's. I was thinking maybe the PatternMatcher might be a good way to construct the dag from the transfer patterns? cheers Paddy On Tue, Oct 19, 2010 at 10:04 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, are you planning to implement the paper fully? Sounds like a good match for my case too, modeling only the train network in Sweden right now. However, I would like to expand it later to more transport options. Well, seems everyone gets down the routing rabbit hole :) Cheers, /peter neubauer VP Product Development, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sun, Oct 17, 2010 at 9:55 PM, Paddy paddyf...@gmail.com wrote: Hi Peter, I've been trying to implement this paper: http://ad.informatik.uni-freiburg.de/papers/transferpatterns.pdf It's being used by google transit. I'm representing the network as a time expanded graph with each departure time at each stop as a stoptime node. Each stoptime node is connected to the parent stop node. Each route is connected at the stoptime level and walking connections are determined by using neo4j-spatial to find available transfers nearby. Time is represented in minutes 0-1440. The paper describes creating a time query graph lookup table, I was thinking of using lucene for the lookups. Then to find all transfer patterns, find all shortest paths from each stoptime node to all stops using a Dijkstra. Convert each path to a transfer pattern. Then to create a directed ascynic graph with these transfer patterns for every stop. Use a time dependent dijskra where costs are determined on the fly by a CostEvaluator using the time query lookup table. This will enable fast queries and also multiple paths. I'm not sure the best way to create these directed acyclic graph's from the transfer patterns. Most of the info I see has been about converting directed acyclic graph to tables, Not sure if there an algorigthm to construct a directed acyclic graph? Paddy On Sat, Oct 16, 2010 at 10:57 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, I am experimenting with connecting train routes to stations. Would your case be something similar? My approach right now is to first insert the stations and index them, then in a second pass insert the routes that reference the station names. Not super pretty but that is the data I have. Is yours something similar? /Peter On Sunday, October 17, 2010, Paddy paddyf...@gmail.com wrote: Hi, I would like to generate a directed acylic graph from a large list of paths. Each path begins from one single source node to multiple end nodes. Would this be a good use case for the pattern matching component? I couldn't find this functionality implemented, I'm not sure of the best way to approach this. Thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Generating a directed acyclic graph
Hi Peter, I've been trying to implement this paper: http://ad.informatik.uni-freiburg.de/papers/transferpatterns.pdf It's being used by google transit. I'm representing the network as a time expanded graph with each departure time at each stop as a stoptime node. Each stoptime node is connected to the parent stop node. Each route is connected at the stoptime level and walking connections are determined by using neo4j-spatial to find available transfers nearby. Time is represented in minutes 0-1440. The paper describes creating a time query graph lookup table, I was thinking of using lucene for the lookups. Then to find all transfer patterns, find all shortest paths from each stoptime node to all stops using a Dijkstra. Convert each path to a transfer pattern. Then to create a directed ascynic graph with these transfer patterns for every stop. Use a time dependent dijskra where costs are determined on the fly by a CostEvaluator using the time query lookup table. This will enable fast queries and also multiple paths. I'm not sure the best way to create these directed acyclic graph's from the transfer patterns. Most of the info I see has been about converting directed acyclic graph to tables, Not sure if there an algorigthm to construct a directed acyclic graph? Paddy On Sat, Oct 16, 2010 at 10:57 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, I am experimenting with connecting train routes to stations. Would your case be something similar? My approach right now is to first insert the stations and index them, then in a second pass insert the routes that reference the station names. Not super pretty but that is the data I have. Is yours something similar? /Peter On Sunday, October 17, 2010, Paddy paddyf...@gmail.com wrote: Hi, I would like to generate a directed acylic graph from a large list of paths. Each path begins from one single source node to multiple end nodes. Would this be a good use case for the pattern matching component? I couldn't find this functionality implemented, I'm not sure of the best way to approach this. Thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Generating a directed acyclic graph
Hi, I would like to generate a directed acylic graph from a large list of paths. Each path begins from one single source node to multiple end nodes. Would this be a good use case for the pattern matching component? I couldn't find this functionality implemented, I'm not sure of the best way to approach this. Thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How do I sort Lucene results by field value, and numerical range queries ?
Hi Andreas,
Yes it looks like you don't need to wrap it in a padded string.
I tried using myIndex.add(ndOne, time,1f); it will stills work.
thanks
Paddy
On Thu, Sep 23, 2010 at 12:37 AM, Andreas Ronge andreas.ro...@jayway.sewrote:
Hi Paddy
Thanks for the response.
But it would be nice to avoid wrapping integer or float values in
padded strings, as you described in your example:
Node ndOne = gds.createNode();
ndOne.setProperty(time, 1.3);
myIndex.add(ndOne, time, ndOne.getProperty(time) );
Instead I believe it's possible in Lucene 3.0 to index the time
property as a float. The question is if this feature is exposed in the
Neo4j Lucene API ?
Cheers
Andreas
On Thu, Sep 23, 2010 at 8:43 AM, Paddy paddyf...@gmail.com wrote:
Hi Andres,
Not sure about the use of sorting but I have previously tried some
numeric
range queries with the new indexProvider.
I've added my some test code i used below.
Cheers
Paddy
public class indexTest {
static GraphDatabaseService gds;
static IndexProvider provider;
@BeforeClass
public static void setup() {
gds = new EmbeddedGraphDatabase(data/neodb/neodb-tmp);
provider = new LuceneIndexProvider( gds );
}
@Test
public void fullTextIndex() {
Transaction tx = gds.beginTx();
try {
Node ndOne = gds.createNode();
ndOne.setProperty(time, 1.3);
Node ndTwo = gds.createNode();
ndTwo.setProperty(time, 1.5);
Node ndThree = gds.createNode();
ndThree.setProperty(time, 2.0);
Node ndFour = gds.createNode();
ndFour.setProperty(time, 3.0);
Node ndFive = gds.createNode();
ndFive.setProperty(time, 5.0);
IndexNode myIndex = provider.nodeIndex( fulltext,
LuceneIndexProvider.FULLTEXT_CONFIG );
myIndex.add(ndOne, time, ndOne.getProperty(time) );
myIndex.add(ndTwo, time, ndTwo.getProperty(time) );
myIndex.add(ndThree,time, ndThree.getProperty(time) );
myIndex.add(ndFour, time, ndFour.getProperty(time) );
myIndex.add(ndFive, time, ndFive.getProperty(time) );
for ( Node searchHit : myIndex.query( time:[1.5 TO 4.3] ) )
{
System.out.println( Found + searchHit.toString() );
System.out.println(time + searchHit.getProperty(time));
}
tx.success();
} finally {
tx.finish();
}
}
}
On Wed, Sep 22, 2010 at 11:15 PM, Andreas Ronge andreas.ro...@jayway.se
wrote:
Hi
In the example
https://svn.neo4j.org/laboratory/components/lucene-index/trunk/src/test/java/org/neo4j/index/impl/lucene/TestLuceneIndex.java
I only see how to sort by Sort.RELEVANCE and Sort.INDEXORDER.
How do I sort ascending/ on different fields ?
Another related question, how does neo4j work with the new improved
numerical capabilities of lucene 3.0.
Example if I add an integer with
org.neo4j.graphdb.index.Index.add(entity, key, integer_value) will it
be index as a NumericField so that I (how?) can use
NumericRangeQueries ?
(In old lucene one had to convert integers to strings and pad it with
zeros)
Cheers
Andres
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] BatchInserter usage with neo4j-lucene-index
Hi Mattias,
i just had neo4j-lucene-index 0.1-SNAPSHOT in the CLASSPATH when i got the
first error
but when i include 1.2-SNAPSHOT 0.1-SNAPSHOT and query the graph created
using:
IndexNode myIndex = provider.nodeIndex(fulltext,
LuceneIndexProvider.FULLTEXT_CONFIG);
for (Node searchHit : myIndex.query(Id:time)) {
}
i get an error:
Exception in thread main java.lang.IncompatibleClassChangeError: Class
org.neo4j.index.impl.SimpleIndexHits does not implement the requested
interface org.neo4j.graphdb.index.IndexHits
at neo4j.indexTest3.main(indexTest3.java:24)
I'm just not sure what i need to include in the CLASSPATH to insert and
index nodes using the BatchInserter,
which are then compatible to be queried using the new LuceneIndexProvider ?
thanks
Paddy
On Thu, Sep 23, 2010 at 4:45 PM, Mattias Persson
matt...@neotechnology.comwrote:
Your code doesn't use the new index framework, but there might be a
CLASSPATH issue where the neo4j-index 1.1 uses lucene 2.9.2 and
neo4j-lucene-index 0.1-SNAPSHOT uses lucene 3.0.1. If you've got 'em both
on
the classpath there might be problems, so please use the one or the other.
Or you could use neo4j-index 1.2-SNAPSHOT where this issue isn't a problem
anymore since it's updated to lucene 3.0.1 as well.
2010/9/24 Paddy paddyf...@gmail.com
Hi,
I got the following error when trying out the
neo4j-lucene-index 0.1-SNAPSHOT with the BatchInserter, it worked ok when
using the neo4j-index 1.1.
The problem is when i call: indexService.getSingleNode(Id, test);
should
i modify my code to use the new neo4j-lucene-index with
the the BatchInserter?
Thanks
Paddy
Exception in thread main java.lang.NoSuchMethodError:
org.apache.lucene.search.IndexSearcher.search(Lorg/apache/lucene/search/Query;)Lorg/apache/lucene/search/Hits;
at
org.neo4j.index.lucene.LuceneIndexBatchInserterImpl.getNodes(LuceneIndexBatchInserterImpl.java:239)
at
org.neo4j.index.lucene.LuceneIndexBatchInserterImpl.getSingleNode(LuceneIndexBatchInserterImpl.java:279)
at neo4j.indexTest2.main(indexTest2.java:42)
public class indexTest2 {
static GraphDatabaseService gds;
static LuceneFulltextIndexBatchInserter indexService;
static BatchInserter inserter;
public static void setup() {
inserter = new BatchInserterImpl(data/neodb/neodb-tmp,
BatchInserterImpl
.loadProperties(neo4j_config.props));
indexService = new LuceneFulltextIndexBatchInserter(inserter);
}
public static void main(String[] args) {
setup();
MapString, Object nodeProperties = new TreeMapString, Object();
nodeProperties.put(Id, test);
long node = inserter.createNode(nodeProperties);
indexService.index(node, Id, test);
long search = indexService.getSingleNode(Id, test);
System.out.println(Search + search);
inserter.shutdown();
indexService.optimize();
indexService.shutdown();
}
}
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship Check During Traversal
Hi ,
I'm trying to setup a Traversal in a time dependant graph
with multiple weighted connections between nodes representing
minutes. I want to only traverse the first relationship with a
value greater than the weight of the traversal's current position.
i.e if the path.weight()=100 only traverse the first relationship
with a departure time property =100
I have implemented a modified BranchSelector to identify the next
relationship to traverse depending on the path weight. But once i have
identified this relationship in BranchSelector.next(), how can
I return a new TraversalBranch using this node.
As i see the TraversalBranch is created in ExpansionSourceImpl using:
TraversalBranch next = new ExpansionSourceImpl( traverser, this,
depth + 1, node,traverser.description.expander, relationship );
Would I need to create a modified ExpansionSourceImpl?
Please let me know if i am going down the right path :-/
thanks
Paddy
On Sat, Sep 11, 2010 at 1:51 PM, Paddy paddyf...@gmail.com wrote:
Hi David,
Thanks for your help, I got it working using the following code. I tested
it on a small graph with the neo4j java-dijkstra example and it works :)
Cheers
Paddy
public static PruneEvaluator pruneAfterTransfer()
{
return new PruneEvaluator()
{
public boolean pruneAfter( Path path )
{
System.out.println(path + path);
int count=0;
if(path.lastRelationship().isType(RelationshipTypes.TRANSFER))
{
IterableRelationship relationships =
path.relationships();
for(Relationship relationship : relationships)
{
if(relationship.isType(RelationshipTypes.TRANSFER))
{
if (++count == 2) {
System.out.println(Breaking!!);
return true;
}
}
}
}
return false;
}
};
}
PruneEvaluator prunerAfterTransfer = pruneAfterTransfer();
private static final TraversalDescription TRAVERSAL =
Traversal.description().uniqueness(
Uniqueness.NONE ).prune(pruneAfterTransfer());
On Fri, Sep 10, 2010 at 10:11 PM, David Montag
david.mon...@neotechnology.com wrote:
Hi Paddy,
One idea is to prune the traversal by looking at whether the path so far
already has a transfer relationship or not. You would then do some kind of
filtering of the resulting paths, e.g. only accepting those with correct
end
nodes. I don't know if the computational complexity of this is acceptable
or
not though.
And I don't know if this answer was relevant or not. I hope it was :)
David
On Sat, Sep 11, 2010 at 4:09 AM, Paddy paddyf...@gmail.com wrote:
Hi just a quick question regarding the use of the PruneEvaluator
I was wondering what would be the best way to modify the
TraversalDescription in the Dijkstra
algorithm in order to prune a traversal when a branch has reached a
second
transfer relationship.
I want to avoid multiple transfers in a bus network.
If the graph is arranged as:
(stop:1) --bus (stop:2) --transfer (stop:3) --bus (stop:4) --transfer
(stop:5)
Is it possible to prune the traversal branch when the 2nd transfer
relationship is reached after (stop:4)
Could this be achieved using a PruneEvaluator? Or am I approaching this
the
wrong way?
thanks
Paddy
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Relationship Check During Traversal
Hi just a quick question regarding the use of the PruneEvaluator I was wondering what would be the best way to modify the TraversalDescription in the Dijkstra algorithm in order to prune a traversal when a branch has reached a second transfer relationship. I want to avoid multiple transfers in a bus network. If the graph is arranged as: (stop:1) --bus (stop:2) --transfer (stop:3) --bus (stop:4) --transfer (stop:5) Is it possible to prune the traversal branch when the 2nd transfer relationship is reached after (stop:4) Could this be achieved using a PruneEvaluator? Or am I approaching this the wrong way? thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] API Questions and a bit more
Hi Alex,
Yes neo4j and lucene is confusing to start but it is very good when combined.:
) It would be good to see if it could integrated, elasticsearch sounds good
Thanks
Paddy
On 5 Sep 2010, at 02:14, Alexandru Popescu
☀the.mindstorm.mailingl...@gmail.com wrote:
On Saturday, September 4, 2010, Paddy paddyf...@gmail.com wrote:
Hi alex,
Some interesting suggestions, As far as I'm aware the Neo4j reference node
is needed to display graphs in neoclipse, That is the only reason i would
use a reference node.
I don't want tosound like arueing, but if that's the only reason, then
it should definitely *not* be part of the API.
The last test that was failing should work if you use the
LuceneFulltextQueryIndexServicehttp://components.neo4j.org/neo4j-index/apidocs/org/neo4j/index/lucene/LuceneFulltextQueryIndexService.html
fulltextIndex = new LuceneFulltextQueryIndexService(gds);
This part is getting even more confusing now :(. There are 2 Lucene
IndexServices in the neo4j 1.1 distro and there's an additional one in
the components. Anyone could explain what each of them is offeringand
if they can be combined?
ElasticSearch looks really good. How would you integrate ElasticSearch with
neo4j?
I'm a newbie to Neo4j so I'm not sure I know the right answer to this
question.
As far as I read, there're probably two ways to integrate the two:
- using the same approach as the IndexService
- use the Neo4j events framework for automatically index/update ElasticSearch
(unfortunately I don't know much about the event framework to say if
this would work or not though)
Bests,
:- alex
cheers
Paddy
2010/9/3 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com
On Friday, September 3, 2010, rick.bullo...@burningskysoftware.com
wrote:
1. They are durable but not permanent, in the sense that if a node
is deleted, its ID will be re-used, unlike autoincremented keys in a
database, which are typically not re-used. Perhaps a poor choice of
words.
In fact if I'm reading this right, they are both durable and
permanent: as long as the node exist it will always be associated with
that ID. The only caveat is that IDs can be reused once their initial
node was purged from the system.
This is a very important aspect as I can imagine many systems that can
use a small subset of the existing nodes as entry points. Basically by
using the cached IDs you'll be able to get to these without the need
of using indexing/traversals.
3. Third parameter = value of the K-V pair you're using to index the
node.
I figured that out myself, but I still believe that the new method
I've suggested would be welcome. Real question is: how many time you
store a set of properties in the node, but want to index it by a
completely unrelated/not persistent value? I'd speculate that this
scenario is very very rare.
4. The reference node is merely one approach to a graph structure.
You can have any number of standalone nodes. Reasonable to allow
deleting the default reference node, though it might be a good idea to
make this a configurable option on DB creation.
If the reference node is not mandatory then why creating it by
default? If you take as a reference the most well known hierarchical
model, the FS, there it makes a lot of sense to have a root node
(which is undeletable). But as I read this and noticed from the tests,
the Neo4j reference node serves no purpose at all.
5/6. Haven't done much with Lucene yet, about to get started soon.
Please keep sharing your experiences. Considering whether or not to
use SOLR, also.
I want to keep things as simple as possible, so for my current
experiments I'll not look beyond what is already available in Neo4j.
This aside, if I'd be to look into using a 3rd party indexing tool, my
first option would be ElasticSearch (disclaimer: I do know the lead
developer of ElasticSearch and his experience in indexing tools).
I hope others will jump in and comment/answer on my suggestions and
questions.
Thanks,
:- alex
Original Message
Subject: [Neo4j] API Questions and a bit more
From: Alexandru_Popescu_â [1]the.mindstorm.mailingl...@gmail.com
Date: Fri, September 03, 2010 4:53 am
To: [2]u...@lists.neo4j.org
Hi all,
Last night I had some time to play with Neo4j (1.1) API. I do have a
couple of questions and comments that I'd like to share with you:
1. The documentation I've found mentions that `Node` IDs are not
permanent. I'm wondering why are IDs exposed them?
2. I was surprised to see a `Node`.delete() failing. The reason was it
had relationships. I think adding a method `Node`.delete(boolean
force) would
make code much easier. The method would automatically:
- remove all relationships
- clean up indexes
Note 1: I've been able to implement locally such a method in an
utility class and it seems
Re: [Neo4j] Using the REST neo4j
Hi Alex, I don't think theres such thing as a stupid question! Keep asking, it the only way to learn :) Did you get it working? Maybe the documentation needs to be expanded. I don't have much experience with the neo4j REST but maybe this would help? http://wiki.neo4j.org/content/Getting_Started_Guide#If_you_use_Maven http://blog.neo4j.org/2010/04/neo4j-rest-server-part1-get-it-going.html Paddy 2010/9/5 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com 2010/9/6 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com: Still on the neo4j-rest component. Is this the correct SVN info: https://svn.neo4j.org/laboratory/components/rest/ I don't seem to find the web.xml in there... As I'm not an expert, I'm wondering if someone could tell me how to enable the neo4j maven repository as the project doesn't seem to build without it. I guess in the end we should delete this thread... it seems like I'm too tired and asking only stupid questions. I wasn't expecting to see a main class instead of the webapp though :- alex thanks, :- alex 2010/9/5 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com: To answer my own question: it looks like deploying a jar including the relationship definitions and including this in wrapper.conf gives you access to the nodes. My bad, :- alex 2010/9/5 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com: Hi all, I know this will probably sound much more stupid than my first email to the list, but after thinking for a bit I'm not sure I understand how one would be using the pre-packaged neo4j-rest. I've used some groovy script to load some test data in a neo4j db. I'm pretty sure it is in there as my script is also checking on this. Then I'm starting the neo4j-rest and try to look under the node 0... nothing. I want to navigate from node 0 but there are no relationships listed. Maybe I can access directly a node as my script is listing the IDs out. Nothing... So, how exactly would one use neo4j-rest? :- alex ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] TransportDublin Route Planner Github Project
Yep sounds like a better way to go instead of running a big Dijkstra. Maybe
an all Paths algorithm?
would monday be good to skype? I'm working on getting the site live,
I have configured nginx to redirect from port 80 to jetty on port 8080
I asked a question on Stack Overflow about the problem i'm having with
Jetty,
http://stackoverflow.com/questions/3509675/problems-with-jetty-crashing-intermittently
.
I started jetty using : java -Xmx900m -server -jar start.jar stdout.txt 2
stderr.txt
After running for 30 minutes the Jetty server will shutdown,
it is not processing any requests, I can see in the In the stderr.txt log
that the Shutdown hook is being executed somehow :
2010-08-27 07:02:01.310:INFO::Shutdown hook executing
2010-08-27 07:02:01.310:INFO::Graceful shutdown
selectchannelconnec...@0.0.0.0:8080
2010-08-27 07:02:12.156:INFO::Graceful shutdown
org.mortbay.jetty.webapp.webappcont...@1d64c37
{/test-jndi,/opt/jetty-6.1.24/contexts/test-jndi.d}
2010-08-27 07:02:12.156:INFO::Graceful shutdown
org.mortbay.jetty.webapp.webappcont...@90cb03
{/,file:/opt/jetty-6.1.24/webapps/transportdublin/}
2010-08-27 07:02:12.156:INFO::Graceful shutdown
org.mortbay.jetty.webapp.webappcont...@d81784
{/,/opt/jetty-6.1.24/webapps/test}
2010-08-27 07:02:12.156:INFO::Graceful shutdown
org.mortbay.jetty.handler.contexthand...@1175422
{/javadoc,file:/opt/jetty-6.1.24/javadoc/}
2010-08-27 07:02:12.156:INFO::Graceful shutdown
org.mortbay.jetty.webapp.webappcont...@fde7ba
{/test-jaas,file:/opt/jetty-6.1.24/webapps/test-jaas/}
2010-08-27 07:02:12.156:INFO::Graceful shutdown
org.mortbay.jetty.webapp.webappcont...@178743b
{/cometd,file:/opt/jetty-6.1.24/webapps/cometd/}
2010-08-27 07:02:13.163:INFO::Stopped selectchannelconnec...@0.0.0.0:8080
2010-08-27
07:02:13.166:INFO:/:org.tuckey.web.filters.urlrewrite.UrlRewriteFilter INFO:
destroy called
2010-08-27 07:02:13.166:INFO:/:Destroying Spring FrameworkServlet 'Spring
MVC Dispatcher Servlet'
2010-08-27 07:02:13.232:INFO::Shutdown hook complete
I have installed http://wiki.eclipse.org/Jetty/Feature/Java-monitor which is
useful, when starting jetty using
java -Dcom.sun.management.jmxremote -jar start.jar etc/jetty-jmx.xml
etc/jetty.xml stdout.txt 2 stderr.txt
I can monitor the jetty server on their control panel
http://java-monitor.com
Would it be better to directly access Jconsole remotely? Or can i access
any java logs to investigate further?
Thanks
Paddy
On Thu, Aug 26, 2010 at 8:41 AM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Very cool Paddy!
I have been talking to Craig about this - we are thinking that, with
knowing a lot of information about your bus stops, there should be a
lot of possibilities to make the graph smarter than running a big
Dijkstra over everything. It might be worth to explore if you can
narrow down the number of possible routes somewhat before doing the
exhaustive traversals. Not sure on the details but maybe we could have
a Skype call on that next week?
Cheers,
/peter neubauer
COO and Sales, Neo Technology
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Thu, Aug 26, 2010 at 9:59 AM, Paddy paddyf...@gmail.com wrote:
To run the Dijkstra algorithm multiple times, I'm assuming it would it be
best to use a high CPU ec2 instance?
I have been checking Amazon and in US East i can run a Cluster Compute
which is a Quadruple Extra Large ec2 instance
33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core Nehalem
architecture)
23 GB memory,
1690 GB of instance storage
64-bit platform
I/O Performance: Very High (10 Gigabit Ethernet)
API name: cc1.4xlarge
Not bad for $1.60 per hour! Would I need to run the Dijkstra algorithm in
a
multi threaded app?
Any help would be great
It runs on CentOS:
Basic Cluster Instances HVM CentOS 5.4 (AMI Id: ami-7ea24a17)
Minimal CentOS 5.4, 64-bit architecture, and HVM-based virtualization for
use with Amazon EC2 Cluster Instances.
More info here:
http://www.allthingsdistributed.com/2010/07/cluster_compute_instance_amazon_ec2.html
Cheers
Paddy
On Wed, Aug 25, 2010 at 3:44 AM, Paddy paddyf...@gmail.com wrote:
Hi Mike,
I cced you to the neo4j list, I enjoyed the talk you gave at the nosql
summer Vancouver about running CouchDb in the cloud at www.Coudant.com.
More info about couchdb: https://nosqleast.com/2009/#speaker/
https://nosqleast.com/2009/#speaker/miller
miller https://nosqleast.com/2009/#speaker/miller
Hope you can make it back on the 24th of Sept for the talk on neo4j
Any more insight into what I would need to do to run a script to launch
multiple instances would be great.
Here is my github http://github.com/paddydub
Re: [Neo4j] TransportDublin Route Planner Github Project
Hi Mike, Thanks for your advise really appreciate it, Hope you had fun in North Carolina :) I have forwarded this to the neo4j mailing list too, hopefully get more opinions too, I have added my answers below, if anyone has any input please reply too. I will do a lot more research and testing to try answer your questions better, as i'm not sure which approach to take. On Fri, Aug 27, 2010 at 2:43 PM, Michael Miller m...@cloudant.com wrote: Hi Paddy, Sorry to be so slow on my reply, insane travel week for me! Just got back from North Carolina, and I can say that the deep south is a strange place! So: 1) you need to run the dykstra alg. 13.5M times on the neo4j db. Can you have multiple independent instances of the neo4j db or will all 13.5M runs be pulling from the same db? If possible could I launch 1000 ec2 nodes and somehow mount 1000 cloned ebs volumes containing the neo4j database to each ec2 node. If i could do this I could run the the shortest path algorithm for a selection of 15 stops as starting points on 1000 ec2 nodes as there is 15000 stops in total. 2) Will your results be written back into the same db? Or can they be combined at a later date? Yes the results could be combined at a later date, e.g saved to a json file and populated into a new database. Or maybe sent to a remote neo4j db via the REST API? Or save the results to a single shared ebs volume containing all the results from the elastic Mapreduce. 3) How long (how much time) does a single run take? Depending on which approach is taken, at the moment i'm not 100% sure which approach to use, I need to do more research. I will get back to you with benchmarks and estimates. 4) And is your dykstra alg. multi-threaded or not? I think it would be best to use a single threaded app for simplicity, I will look into other algorithms as Peter and Craig have suggested to make the graph smarter than running a big Dijkstra over everything. But their is lots of possibility's we just have to find the best approach...Thanks Again Depending on those answers I'd suggest different solutions. Let's assume that you can run the algorithm as a single core application in a relatively short ammount of time (say 1 sec). So if you need to do 13.5E6 runs at 1 sec each, then I would spread that over 1000 ec2 nodes (m1.small?) and probably use elastic Mapreduce.That would take a 4 hours and cost you $375, if I did the math right. Now things change totally if it's a highly multi-threaded app and is getting bottlenecked at the IO level because you have to talk to a single neo4j instance. In that case you want to go in series (instead of parallel) as fast as possible and on the biggest iron you can find. However, I hope that it is a purely parallel problem without IO limitiations, in which case elastic mapreduce is a wonderful way to get to large scale easily (use the streaming case). -M Thanks, -- Forwarded message -- From: Paddy paddyf...@gmail.com Date: Thu, Aug 26, 2010 at 12:59 AM Subject: Re: [Neo4j] TransportDublin Route Planner Github Project To: Neo4j user discussions user@lists.neo4j.org To run the Dijkstra algorithm multiple times, I'm assuming it would it be best to use a high CPU ec2 instance? I have been checking Amazon and in US East i can run a Cluster Compute which is a Quadruple Extra Large ec2 instance 33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core Nehalem architecture) 23 GB memory, 1690 GB of instance storage 64-bit platform I/O Performance: Very High (10 Gigabit Ethernet) API name: cc1.4xlarge Not bad for $1.60 per hour! Would I need to run the Dijkstra algorithm in a multi threaded app? Any help would be great It runs on CentOS: Basic Cluster Instances HVM CentOS 5.4 (AMI Id: ami-7ea24a17) Minimal CentOS 5.4, 64-bit architecture, and HVM-based virtualization for use with Amazon EC2 Cluster Instances. More info here: http://www.allthingsdistributed.com/2010/07/cluster_compute_instance_amazon_ec2.html Cheers Paddy On Wed, Aug 25, 2010 at 3:44 AM, Paddy paddyf...@gmail.com wrote: Hi Mike, I cced you to the neo4j list, I enjoyed the talk you gave at the nosql summer Vancouver about running CouchDb in the cloud at www.Coudant.com. More info about couchdb: https://nosqleast.com/2009/#speaker/miller Hope you can make it back on the 24th of Sept for the talk on neo4j Any more insight into what I would need to do to run a script to launch multiple instances would be great. Here is my github http://github.com/paddydub/TransportDublin and twitter :) http://twitter.com/TransportDublin Should i use cloudera, Hadoop? I don't have much experience with distributed computing. What sort of software would i need to use in Amazon? Any help would be great! I think it could be used on road networks and extended to the http://www.openstreetmap.org/ API. what do u think
Re: [Neo4j] TransportDublin Route Planner Github Project
To run the Dijkstra algorithm multiple times, I'm assuming it would it be
best to use a high CPU ec2 instance?
I have been checking Amazon and in US East i can run a Cluster Compute
which is a Quadruple Extra Large ec2 instance
33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core Nehalem
architecture)
23 GB memory,
1690 GB of instance storage
64-bit platform
I/O Performance: Very High (10 Gigabit Ethernet)
API name: cc1.4xlarge
Not bad for $1.60 per hour! Would I need to run the Dijkstra algorithm in a
multi threaded app?
Any help would be great
It runs on CentOS:
Basic Cluster Instances HVM CentOS 5.4 (AMI Id: ami-7ea24a17)
Minimal CentOS 5.4, 64-bit architecture, and HVM-based virtualization for
use with Amazon EC2 Cluster Instances.
More info here:
http://www.allthingsdistributed.com/2010/07/cluster_compute_instance_amazon_ec2.html
Cheers
Paddy
On Wed, Aug 25, 2010 at 3:44 AM, Paddy paddyf...@gmail.com wrote:
Hi Mike,
I cced you to the neo4j list, I enjoyed the talk you gave at the nosql
summer Vancouver about running CouchDb in the cloud at www.Coudant.com.
More info about couchdb:
https://nosqleast.com/2009/#speaker/https://nosqleast.com/2009/#speaker/miller
miller https://nosqleast.com/2009/#speaker/miller
Hope you can make it back on the 24th of Sept for the talk on neo4j
Any more insight into what I would need to do to run a script to launch
multiple instances would be great.
Here is my github http://github.com/paddydub/TransportDublin and twitter
:) http://twitter.com/TransportDublin
Should i use cloudera, Hadoop? I don't have much experience with
distributed computing.
What sort of software would i need to use in Amazon?
Any help would be great! I think it could be used on road networks and
extended to the http://www.openstreetmap.org/ API.
what do u think? :)
I uploaded a prepopulated sample Dublin Bus time dependent network modelled
in a Neo4j database with 1 million
nodes and 10 million relationships in a Weighted Graph
http://github.com/downloads/paddydub/TransportDublin/neo4j-db.zip
Also a Json File containing the Stop Information which is used to populate
the database with spatial data.
http://github.com/downloads/paddydub/TransportDublin/stops.json
I will upload the code to github that would be needed to run the Dijkstra
algorithm 13,5000 million times on the neo4j database and
to cache the results to store neo4j database as a WeightedGraph Path object
and path weight as the cost in minutes.
Possibly with multiple ec2 instances running the Dijkstra algorithm for
different stop departure combinations,
all writing to a linked EBS, Not sure if this is possible.
http://wiki.neo4j.org/content/Neo4j_in_the_Cloud
Or storing the results to a json file to be inserted using the neo4j
BatchInserteter.
http://wiki.neo4j.org/content/Batch_Insert
I posted the question below to the neo4j mailing list, what do you think?
Can it be done? :)
Also the http://algo2.iti.kit.edu/routeplanning.php might be of interest.
and Fast Route Planning with Peter Sanders:
http://www.youtube.com/watch?v=-0ErpE8tQbw
If we do not know the end arrival time we can pick all nodes that have
bus's arriving at that stop in the next 3-4 hours.
Then create a link to a new Stop Time temporary end node, which is used as
the end node in the Dijkstra algorithm.
To be really accurate there would need to be 15000*15000*60*24 start nodes
or what would be the best way to approach this???
I posted the question below to the neo4j mailing list, what do you think?
Can it be done? :)
If we do not know the end arrival time we can pick all nodes that have
bus's arriving at that stop in the next 3-4 hours.
Then create a link to a newly created node in the neo4j database and use
this temporary node as the end node in the Dijsktra algorithm.
.
for (Map.EntryString, Double neighbour : neighboursMap.entrySet()) {
String neighbourStopID = neighbour.getKey();
// Lucene Query: 110001109* represents 09* a LuceneFulltextQuery to
find all stoptimes connected with an hour value of 09
String query = 1 + neighbourStopID + 09*;
for (Node hit : indexService.getNodes(StopTime.STOPTIMEID,query)) {
StopTime neighbourStopTime = new StopTime(hit);
..First Draft
import ie.transportdublin.datastructure.json.Data;
import ie.transportdublin.datastructure.json.Data.Stop;
import ie.transportdublin.datastructure.neo4j.RelationshipTypes;
import ie.transportdublin.datastructure.neo4j.StopTime;
import ie.transportdublin.datastructure.neo4j.Waypoint;
import ie.transportdublin.mvc.ajax.pathfinder.SetupGraph;
import ie.transportdublin.mvc.ajax.routeplanner.RoutePlanner;
import java.io.FileReader;
import java.util.List;
import java.util.Map;
import org.joda.time.DateTime;
import org.neo4j.graphalgo.CommonEvaluators;
import org.neo4j.graphalgo.CostEvaluator;
import org.neo4j.graphalgo.EstimateEvaluator;
import org.neo4j.graphalgo.GraphAlgoFactory
Re: [Neo4j] TransportDublin Route Planner Github Project
stopsList2 = stopsList1;
long startTimer = System.currentTimeMillis();
for(Stop stop :stopsList1)
{
// Show it.
System.out.println(stop);
String stopId = stop.getStopId();
Expander relExpander = Traversal.expanderForTypes(
RelationshipTypes.BUS, Direction.OUTGOING);
PathFinderWeightedPath dijkstra = GraphAlgoFactory.dijkstra(relExpander,
costEval);
/* The dataset uses 15000 dublin bus stops nodes which are
stored in a json file. To find all routes at 9.00am on a
monday for every stop, the Dijkstra algorithm would have
to be run 15000 x 15000 = 225 million times.
To find all routes for all times it would need to be run
for every 1 minute period in a day, it would have to be run
13,500 million times.
e.g 225 million x 60 x 24 = 13,500 million
*/
final int count = 15000*15000*60*24;
WeightedPath path;
double weight;
while(--count!=0)
{
path = dijkstra.findSinglePath(
start.getUnderlyingNode(),
end.getUnderlyingNode());
weight = path.weight();
}
long endTimer = System.currentTimeMillis();
System.out.println(Dijkstra Time:: + (endTimer - startTimer)
+ milliseconds);
}
}
WDYT? I would appreciate any help!
Cheers,
Paddy
On Tue, Aug 24, 2010 at 12:14 AM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Interesting stuff Paddy!
I would love to explore the combination of precomputed routes and
Neo4j Spatial to do these calculations! Feel free to keep us updated
on this, maybe some of the work can be incorporated into Neo4j Spatial
at some time?
Cheers,
/peter neubauer
COO and Sales, Neo Technology
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Tue, Aug 24, 2010 at 12:15 AM, Paddy paddyf...@gmail.com wrote:
Hi,
I think a better approach might be to take advantage of Hadoop MapReduce
to
precompute all routes by running the Dijkstra algorithm
to find the best route for every possible combination.
E.g the dataset uses 15000 bus stops nodes
To find all routes at 9.00am on a monday for every stop, the Dijkstra
algorithm would have to be run 15000 x 15000 = 225 million times.
To find all routes for all times it would need to be run for every 1
minute
period in a day, it would have to be run 13,500 million times.
e.g 225 million x 60 x 24 = 13,500 million
Each precomputed route would be stored in the neo4j database, with the
total
route time and the path also stored in the database
as a Weighted Path of Bus stops and connections times
If the best route from Stop 100 to Stop 342 is to take a bus that leaves
stop 100 at 9.05 and takes 45 minutes to reach Stop 342,
the database will be arranged as below to factor in waiting times:
the shortest route from stop 100 to stop 342 at 9.00am takes 50 Minutes,
Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543
--(15mins)-
Stop 342
the shortest route from stop 100 to stop 342 at 9.01am takes 49 Minutes,
Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543
--(15mins)-
Stop 342
the shortest route from stop 100 to stop 342 at 9.02am takes 48 Minutes,
Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543
--(15mins)-
Stop 342
the shortest route from stop 100 to stop 342 at 9.03am takes 47 Minutes,
Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543
--(15mins)-
Stop 342
the shortest route from stop 100 to stop 342 at 9.04am takes 46 Minutes,
Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543
--(15mins)-
Stop 342
the shortest route from stop 100 to stop 342 at 9.05am takes 45 Minutes,
Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543
--(15mins)-
Stop 342
The routing information for each 13,5000 million possible connections
would
be stored in the database.
-Route Finding
-Use neo4j spatial to find all stops within walking distance of the start
and end coordinate
calculate the walking time to each stop in minutes
-Using the precalculated routing information and factoring in the time to
it
takes to walk to a nearby stop,
-This would allow the best route possible route to be found with 2 neo4j
spatial querys
and a quick calculation to find the shortest route from every possible
route
combination from nearby stops.
i.e.
calculate walking time to stop 100 using neo4j spatial at runtime = 2
minutes.
If the departure time is 9.00 and it takes 2 minutes to walk to stop
100,
it means querying the precomputed routes
database for the already discovered quickest route
route from stop 100 to stop 342 at 9.02 which we know takes 48 minutes
+
calculate walking time from stop 342
Re: [Neo4j] TransportDublin Route Planner Github Project
Hi, I think a better approach might be to take advantage of Hadoop MapReduce to precompute all routes by running the Dijkstra algorithm to find the best route for every possible combination. E.g the dataset uses 15000 bus stops nodes To find all routes at 9.00am on a monday for every stop, the Dijkstra algorithm would have to be run 15000 x 15000 = 225 million times. To find all routes for all times it would need to be run for every 1 minute period in a day, it would have to be run 13,500 million times. e.g 225 million x 60 x 24 = 13,500 million Each precomputed route would be stored in the neo4j database, with the total route time and the path also stored in the database as a Weighted Path of Bus stops and connections times If the best route from Stop 100 to Stop 342 is to take a bus that leaves stop 100 at 9.05 and takes 45 minutes to reach Stop 342, the database will be arranged as below to factor in waiting times: the shortest route from stop 100 to stop 342 at 9.00am takes 50 Minutes, Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543 --(15mins)- Stop 342 the shortest route from stop 100 to stop 342 at 9.01am takes 49 Minutes, Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543 --(15mins)- Stop 342 the shortest route from stop 100 to stop 342 at 9.02am takes 48 Minutes, Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543 --(15mins)- Stop 342 the shortest route from stop 100 to stop 342 at 9.03am takes 47 Minutes, Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543 --(15mins)- Stop 342 the shortest route from stop 100 to stop 342 at 9.04am takes 46 Minutes, Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543 --(15mins)- Stop 342 the shortest route from stop 100 to stop 342 at 9.05am takes 45 Minutes, Path = Stop 100 --(15mins)-- Stop 55 --(15mins)-- Stop 4543 --(15mins)- Stop 342 The routing information for each 13,5000 million possible connections would be stored in the database. -Route Finding -Use neo4j spatial to find all stops within walking distance of the start and end coordinate calculate the walking time to each stop in minutes -Using the precalculated routing information and factoring in the time to it takes to walk to a nearby stop, -This would allow the best route possible route to be found with 2 neo4j spatial querys and a quick calculation to find the shortest route from every possible route combination from nearby stops. i.e. calculate walking time to stop 100 using neo4j spatial at runtime = 2 minutes. If the departure time is 9.00 and it takes 2 minutes to walk to stop 100, it means querying the precomputed routes database for the already discovered quickest route route from stop 100 to stop 342 at 9.02 which we know takes 48 minutes + calculate walking time from stop 342 to the end coordinate using neo4j spatial at runtime = 4 minutes = Total time for this route is 54 minutes This should be repeated for all combinations of bus stops that are within walking distance of the start and end coordinate. Use the route with the lowest total time as this will be the quickest route possible . If the route found with the lowest total time is from stop 100 to stop 342 at 9.02 The database would be queried to return the route from stop 100 to stop 342 at 9.02 This will return a Weighted path object of Bus stops nodes needed to travel to and connection times , latitude and longitude properties of the bus stops are used to populate the route on the Map. If the route uses multiple bus or train connections, the waiting times for connections is stored in the weighted path I think this could be achieved somehow a using hadoop MapReduce setup, maybe with multiple ec2 instances writing to a shared ec2 ebs neo4j database or routing results being stored locally to a json file an then populated to the database using the BatchInserter This would speed things up considerably as the A* algorightm would not have to be run for each routing request at runtime in the web app. Also the Dijkstra algorithm will find the best route as it is more accurate than the A* algorithm. I have no idea how long it would take to run the Dijkstra algorithm 13,5000 million times. If say it takes on average 1 second to run Dijkstra, it will take 156250 Days on one machine , does anyone have an ideas or experience with Hadoop or MapReduce or what would be the best way to go about this or if it is even possible. I think varations of this method could be useful for many different applications of route planning with Dijkstra. Also an interesting video by Google on how they represent graphs and on using MapReduce with graphs for Dijkstra and PageRank: Parallel Graph Algorithms with MapReduce http://www.youtube.com/watch?v=BT-piFBP4fE Also the MapReduce paper is a good read http://labs.google.com/papers/mapreduce-osdi04.pdf Thanks Paddy On Thu, Aug 19, 2010 at 11:17 PM, Peter Neubauer peter.neuba...@jayway.comwrote: Paddy, depending on your traffic
Re: [Neo4j] Optimizing neo4j for traversal in Spring
just tested it out and it works perfectly using the following settings in the spring app-config.xml , I can see right away it makes a big difference for traversals :) bean id=configuration class=org.neo4j.kernel.EmbeddedGraphDatabase factory-method=loadConfigurations constructor-arg value=neo4j_config.props/ /bean bean id=graphDbService class=org.neo4j.kernel.EmbeddedGraphDatabase init-method=enableRemoteShell destroy-method=shutdown constructor-arg index=0 value=data/neo4j-db/ constructor-arg index=1 ref=configuration / /bean On Thu, Aug 12, 2010 at 7:46 AM, Anders Nawroth and...@neotechnology.comwrote: For reference, there's a Stackoverflow thread on this topic as well: http://stackoverflow.com/questions/3466437/ An answer has been posted today, but I didn't try it out though. /anders On 07/15/2010 01:28 PM, Paddy wrote: hi all, I would like to integrate the optimizing neo4j for traversal settings in Spring using the example from the imdb spring app, the graphDbService is configured the in app-config.xml: bean id=graphDbService class=org.neo4j.kernel.EmbeddedGraphDatabase init-method=enableRemoteShell destroy-method=shutdown constructor-arg index=0 value=/home/neo/var/neo4j-db/ /bean I want to use the optimizing for traversals example settings, how can the configurations for the neo4j_config.props file be set if the GraphDatabaseService is injected by Spring? http://wiki.neo4j.org/content/Configuration_Settings#Optimizing_for_traversals_example The Configuration Settings wiki outlines the setup as : MapString,String configuration = EmbeddedGraphDatabase. loadConfigurations( neo4j_config.props ); GraphDatabaseService graphDb = new EmbeddedGraphDatabase( my-neo4j-db/, configuration ); but with the spring imdb example it is and auto-wired using the @Autowired annotation, can these settings be configured in spring?: @Autowired private GraphDatabaseService graphDbService; Thanks a lot Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Use of LuceneFulltextIndexBatchInserter
I posted a question on MathOverflow http://mathoverflow.net/questions/35563/best-algorithm-to-use-for-a-bus-train-transport-route-planner and I emailed the authors of the slides, hopefully we can get some more insights into the bidirectional A* or the best algorithm to use. A lot of research has been done in this field, this describes a A* algorithm with landmarks: : http://www.optimization-online.org/DB_HTML/2008/11/2154.html Also the Dijkstra’s Algorithm On-Line: An Empirical Case Study from Public Railroad Transport paper also looks good cheers Paddy On Fri, Aug 13, 2010 at 10:06 PM, Paddy paddyf...@gmail.com wrote: Thanks Peter, Sounds good, Yes that would be a great addition to the graph-algo! I would love to contribute but I'm not sure where to begin, it looks pretty complicated. Also I noticed the ExperimentalAStar in org.neo4j.graphalgo.impl.path . What is the difference between AStar.java and the ExperimentalAStar.java ? found some very interesting slides too regarding a bidirectional A*: Bidirectional A* on Time-Dependent Graphs http://ctw08.dti.unimi.it/slides/B5/B5-1-Nannicini.pdf It describes the European road network as: roughly 18M nodes, 42M arcs. and A very good implementation of Dijkstra’s algorithm explores on average 9M nodes and takes more than 6 seconds (european road network): it’s too long! http://www.cs.princeton.edu/courses/archive/spr06/cos423/Handouts/EPP%20shortest%20path%20algorithms.pdf- page 4 Thanks Paddy On Fri, Aug 13, 2010 at 2:21 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, apart from tweaking the system, we have been thinking about a t that would start from both ends and route. Not sure how advanced that would be but I think it could come in handy. If you have time later on, that might be an interesting place to investigate some contribution to the graph-algo package :) Overall, the field of routing algos seems to be a very wide one, so I think once things are working at your end, there is a whole lot regarding better suitable algos taking into account special context and constraints that could be done ... Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 11:14 PM, Paddy paddyf...@gmail.com wrote: Hi Peter, Nearly there :) I'm going to make a few changes to try increase the speed of the a*star route finder, as the current graph is 1.2gb with 1million nodes and 10 million relationships. - reducing the database size by only including bus routes between 7am to 1pm on Mondays instead of every bus route for every day - configuring the neo4j_config.props for traversal and injecting it in the spring bean - creating less relationships by adding less walking connections between stops e.g only add connections for stops within 50m this will reduce the size of the db and hopefully this will speed up the path finding and as it can take 5-10 seconds for long routes with the current graph. I will let you know Cheers Paddy On Thu, Aug 12, 2010 at 11:27 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Glad you solved it! Btw, is the site now officially online? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 4:38 AM, Paddy paddyf...@gmail.com wrote: Hi, so I just needed to use :) : LuceneFulltextIndexBatchInserter index = new LuceneFulltextIndexBatchInserter( inserter ); Good batch insert example @ https://svn.neo4j.org/components/index/trunk/src/test/java/org/neo4j/index/lucene/TestBatchInsert.java Paddy On Wed, Aug 11, 2010 at 8:47 PM, Paddy paddyf...@gmail.com wrote: Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https
[Neo4j] Neo4j spatial Shapefile Importer
Hi
using the latest neo4j-spatial 0.1-SNAPSHOT i currently get an error on:
spatialService.createLayer( stops_layer);
Should i be using the ShapeFile Importer instead to create a layer?:
importer.importShapefile(roads.shp, layer_roads);
What open source software would you recommend to view Shapefiles and to
create Shapefiles from a list of coordinates ?
As i currently import spatial data stored in a text file.
public class ImportLayer2 {
private static final String DB_PATH = target/neo4j-db12;
public static void main(String[] args) throws Exception {
GraphDatabaseService graphDbService = new
EmbeddedGraphDatabase(DB_PATH);
try {
SpatialDatabaseService spatialService = new
SpatialDatabaseService(graphDbService);
spatialService.createLayer( stops_layer);
} finally {
graphDbService.shutdown();
}
}
}
i get the following error:
Exception in thread main java.lang.RuntimeException
at org.neo4j.gis.spatial.DefaultLayer.makeLayer(DefaultLayer.java:242)
at
org.neo4j.gis.spatial.SpatialDatabaseService.createLayer(SpatialDatabaseService.java:173)
at
org.neo4j.gis.spatial.SpatialDatabaseService.createLayer(SpatialDatabaseService.java:164)
at spatial.neo4j.ImportLayer2.main(ImportLayer2.java:18)
Caused by: java.lang.InstantiationException: org.neo4j.gis.spatial.Layer
at java.lang.Class.newInstance0(Class.java:340)
at java.lang.Class.newInstance(Class.java:308)
at org.neo4j.gis.spatial.DefaultLayer.makeLayer(DefaultLayer.java:248)
at org.neo4j.gis.spatial.DefaultLayer.makeLayer(DefaultLayer.java:240)
... 3 more
Cheers
Paddy
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Use of LuceneFulltextIndexBatchInserter
Hi Peter, Nearly there :) I'm going to make a few changes to try increase the speed of the a*star route finder, as the current graph is 1.2gb with 1million nodes and 10 million relationships. - reducing the database size by only including bus routes between 7am to 1pm on Mondays instead of every bus route for every day - configuring the neo4j_config.props for traversal and injecting it in the spring bean - creating less relationships by adding less walking connections between stops e.g only add connections for stops within 50m this will reduce the size of the db and hopefully this will speed up the path finding and as it can take 5-10 seconds for long routes with the current graph. I will let you know Cheers Paddy On Thu, Aug 12, 2010 at 11:27 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Glad you solved it! Btw, is the site now officially online? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 4:38 AM, Paddy paddyf...@gmail.com wrote: Hi, so I just needed to use :) : LuceneFulltextIndexBatchInserter index = new LuceneFulltextIndexBatchInserter( inserter ); Good batch insert example @ https://svn.neo4j.org/components/index/trunk/src/test/java/org/neo4j/index/lucene/TestBatchInsert.java Paddy On Wed, Aug 11, 2010 at 8:47 PM, Paddy paddyf...@gmail.com wrote: Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Use of LuceneFulltextIndexBatchInserter
Hi, so I just needed to use :) : LuceneFulltextIndexBatchInserter index = new LuceneFulltextIndexBatchInserter( inserter ); Good batch insert example @ https://svn.neo4j.org/components/index/trunk/src/test/java/org/neo4j/index/lucene/TestBatchInsert.java Paddy On Wed, Aug 11, 2010 at 8:47 PM, Paddy paddyf...@gmail.com wrote: Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Use of LuceneFulltextIndexBatchInserter
Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] TransportDublin Route Planner Github Project
thanks peter that looks a lot better. I'm working on a Neo4j Ajax combination to display only routes on the map. Ajax and Neo4j is a really powerful combination, it is useful feature to display the neo4j database property's, With a jQuery LiveSearch feature Spring Mvc-Ajax combined with neo4j luceneFullTextQuery, a database can be queried on the fly. What is the best hosting and memory settings for my kind of app? I have tested on a amazon ec2 small instance, but it can take from 0.1 to 10 seconds to find a route depending on distance .Can I replicate the database to allow multiple simultanous requests? Thanks Paddy On Tue, Jul 27, 2010 at 7:02 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, I took the freedom to format the wiki a bit and put a maven profile into the pom.xml in order to download and expand the data automatically if you run mvn -P import install jetty:run http://github.com/peterneubauer/TransportDublin Is that ok? Would it be possible to do the import automatically to, so we could extract the timetable data live somewhere? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sun, Jul 25, 2010 at 11:02 AM, Paddy paddyf...@gmail.com wrote: I have added a Quick Install Guide on the wiki if anyone would like to try: Download source : 1. Download latest zip source file from: http://github.com/paddydub/TransportDublin/archives/master 2. Extract zip contents into C:\dev\transportdublin\ Download database: 3. Download a prepopulated graph database from neo4j-db.ziphttp://www.transportdublin.ie/neo4j/neo4j-db.zip 4. Extract the neo4j-db.zip file to folder: C:\dev\transportdublin\data\ 5. cd to C:\dev\transportdublin\ Launch Jetty server 6. From the command line type: mvn jetty:run 7. Point your browser to location http://localhost:8080/transportdublin/routeplanner 8. Click on two locations on the map to generate a route Thanks Paddy On Fri, Jul 23, 2010 at 4:53 PM, Paddy paddyf...@gmail.com wrote: Hi, I have updated the wiki with screen shots and information http://wiki.github.com/paddydub/TransportDublin/ and I have uploaded my code and bus stop data sql script. Any suggestions or recommendations would be appreciated. I'm currently populating my graph from a mysql database, I'm working next on implementing the BatchInserter next to speed up the graph setup process Thanks Paddy On Wed, Jul 21, 2010 at 3:26 PM, Anders Nawroth and...@neotechnology.comwrote: Hi Paddy! Some interesting stuff you're working on there! I'd like to write a bit about the differences in neo4j and sql and why neo4j it is a perfect solution for route planning systems, do you think a wiki would be the best option to display the pics? I think the Github wiki of the project good be a good place to put the article. Images can be added to the source repo (just remember to use the raw version of the images as img src) or can be uploaded as downloads of the project. When your writings are in place, it should of course be linked from the Neo4j wiki. WDYT? /anders I will be uploading the code today and tomorrow, just making some last minute changes and writing some documentation. Cheers Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] TransportDublin Route Planner Github Project
I have added a Quick Install Guide on the wiki if anyone would like to try: Download source : 1. Download latest zip source file from: http://github.com/paddydub/TransportDublin/archives/master 2. Extract zip contents into C:\dev\transportdublin\ Download database: 3. Download a prepopulated graph database from neo4j-db.ziphttp://www.transportdublin.ie/neo4j/neo4j-db.zip 4. Extract the neo4j-db.zip file to folder: C:\dev\transportdublin\data\ 5. cd to C:\dev\transportdublin\ Launch Jetty server 6. From the command line type: mvn jetty:run 7. Point your browser to location http://localhost:8080/transportdublin/routeplanner 8. Click on two locations on the map to generate a route Thanks Paddy On Fri, Jul 23, 2010 at 4:53 PM, Paddy paddyf...@gmail.com wrote: Hi, I have updated the wiki with screen shots and information http://wiki.github.com/paddydub/TransportDublin/ and I have uploaded my code and bus stop data sql script. Any suggestions or recommendations would be appreciated. I'm currently populating my graph from a mysql database, I'm working next on implementing the BatchInserter next to speed up the graph setup process Thanks Paddy On Wed, Jul 21, 2010 at 3:26 PM, Anders Nawroth and...@neotechnology.comwrote: Hi Paddy! Some interesting stuff you're working on there! I'd like to write a bit about the differences in neo4j and sql and why neo4j it is a perfect solution for route planning systems, do you think a wiki would be the best option to display the pics? I think the Github wiki of the project good be a good place to put the article. Images can be added to the source repo (just remember to use the raw version of the images as img src) or can be uploaded as downloads of the project. When your writings are in place, it should of course be linked from the Neo4j wiki. WDYT? /anders I will be uploading the code today and tomorrow, just making some last minute changes and writing some documentation. Cheers Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] TransportDublin Route Planner Github Project
Hi, I have updated the wiki with screen shots and information http://wiki.github.com/paddydub/TransportDublin/ and I have uploaded my code and bus stop data sql script. Any suggestions or recommendations would be appreciated. I'm currently populating my graph from a mysql database, I'm working next on implementing the BatchInserter next to speed up the graph setup process Thanks Paddy On Wed, Jul 21, 2010 at 3:26 PM, Anders Nawroth and...@neotechnology.comwrote: Hi Paddy! Some interesting stuff you're working on there! I'd like to write a bit about the differences in neo4j and sql and why neo4j it is a perfect solution for route planning systems, do you think a wiki would be the best option to display the pics? I think the Github wiki of the project good be a good place to put the article. Images can be added to the source repo (just remember to use the raw version of the images as img src) or can be uploaded as downloads of the project. When your writings are in place, it should of course be linked from the Neo4j wiki. WDYT? /anders I will be uploading the code today and tomorrow, just making some last minute changes and writing some documentation. Cheers Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] TransportDublin Route Planner Github Project
Hi , I have a github repositry setup @ http://github.com/paddydub/TransportDublin I'm working on a site: transportdublin.ie which is a Dublin Public Transport Route Planner which is built using Neo4j , Google Maps API v3, Spring 3.0 MVC-AJAX, with JQuery for directions panel layout and AJAX and Javascript parsed JSON to display the polyline and markers. I have uploaded some screenshots: Route Planner Screenshot http://img838.imageshack.us/img838/8676/websitescreenshot.jpg Neo4j representation of a Bus Route http://img204.imageshack.us/img204/9110/neoclipsebustimetablesc.png SQL format used to populate the graph http://img835.imageshack.us/img835/6426/sqldiagram.png http://img231.imageshack.us/img231/5421/sqlstops.png http://img375.imageshack.us/img375/7087/sqlstoptimes.png http://img62.imageshack.us/img62/6079/sqltimetable.png I'd like to write a bit about the differences in neo4j and sql and why neo4j it is a perfect solution for route planning systems, do you think a wiki would be the best option to display the pics? I will be uploading the code today and tomorrow, just making some last minute changes and writing some documentation. Cheers Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] spatial-distance-query-example
Hi I didn't get a chance to finish re-factoring the project today, i will have it ready Tuesday as I'd like to write some documentation as well and make the code cleaner Thanks Paddy On Thu, Jul 15, 2010 at 11:38 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, great, we will take a look at your code. Craig mentioned that you might use the GeoTool suite to do distance queries, which Davide has integrated with Neo4j. We can see how that works out. Would love to fork that code int neo4j-examples in order to show other how to use the GIS features. Thanks for sharing the project so we can improve things! Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 skype:+46704106975?call LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Jul 16, 2010 at 3:55 AM, Paddy paddyf...@gmail.com wrote: Hi guys, I've added a neo4j spatial-distance-query-example on github to get to know how to use github, heres the link: http://github.com/paddydub/spatial-distance-query-example I'm going to add the transportdublin project i'm working on next to github. I'm refactoring the code and making some small changes, hopefully you guys can have a look and see what you think, and re factor or reuse the the code if you think it could be better. I have some small changes to make and i will upload it tomorrow thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j with Amazon EC2 Setup
hi, Thanks for the advice, Would it be better to use Amazon S3 or EBS storage to mount the neo4j database? thanks Paddy On Thu, Jul 8, 2010 at 6:24 AM, Dave butlerdi butle...@gmail.com wrote: We have just started putting the NK stuff together as they recently moved to V4 and it is quite a curve for us. (Also not much time due to August release). Have been using the NK stuff for 8 years or so and it a real good platform for handling cache, execution scheduling et al. Real smart guys over there. Over the years we have written many accessors (XMPP, JXTA, JINI (Spaces) and many other protocols as well as ASN1(PER) libraries to replace XML in messaging. Now just doing the Wave, Neo4j and Allegrograph stuff at present. As soon as we have something that is readable we will send a message. You will be able to install from the NK installer and the jars (as is common in NK will contain scripts, docs, code ...). On 8 July 2010 15:18, Peter Neubauer peter.neuba...@neotechnology.com wrote: Dave, sounds very interesting to have a netkernel based REST support for Neo4j. Would be great to get some details on that - is the code accessible somewhere to learn more about it? This might be an interesting programming model even for others ... Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Jul 8, 2010 at 2:59 PM, Dave butlerdi butle...@gmail.com wrote: Yep, that is fine. Just using Neo for an RDF store (dense tripple). It works ok but we need to do a lot more work, so still using Allegrograph more (3/4). OWL2 and Reasoner support for dynamic reasoning is really cool for us. . At present we are witing a NetKernel Module for Neo that will give us a very stable and fast neo Rest service as well as provide easy scripting of apps (DPML, Groovy JS, Ruby, PHP, Python ...). Will post when we get done (mid August most likeley). At present Neo is used to store RDF ised messages from barcode and RFID readers in GS1 formats , events are then passed to Robots which handle messaging and update Google Waves. It is all M2M except for dashboard. Sail and all really need better docs. We have worked most of it out and when we finish we will do some docs for internal use. You may also find some use for them. On 8 July 2010 14:41, Peter Neubauer peter.neuba...@neotechnology.com wrote: Dave, you working on the http://pharm2phork.org project? Seems awesome! Are you using the Neo4j RDF components in there? Would be nice to get some feedback on your experience in working with them! I added the project to http://wiki.neo4j.org/content/Neo4j_In_The_Wild#Other if that is ok? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Jul 8, 2010 at 2:30 PM, Dave butlerdi butle...@gmail.com wrote: There was a Cent OS AMI which contained Jetty in the EU West zone. Or else just use a Tomcat instance like ami-45e7002c on US East. Both were fine with Neo4j. You can also just embed jetty and neo into a wrapper and run. On 8 July 2010 14:22, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, yes, this is about the things to consider. I think you should be fine. You might need to tweak some filesystem stuff and the memory mapping settings later, according to the Performance Guidelines, http://wiki.neo4j.org/content/Neo4j_Performance_Guide . Let us know how things progress for you! Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Jul 8, 2010 at 3:25 AM, Paddy paddyf...@gmail.com wrote: Hi, I'm trying to configure neo4j to run on an amazon ec2 instance. This is my first time using ec2, I'd like to deploy a neo4j web app war file to Jetty. So far I have been reading
Re: [Neo4j] Neo4j with Amazon EC2 Setup
also can i load the configurations from the neo4j_config.props file if the GraphDatabaseService is injected by Spring? The Configuration Settings wiki outlines the setup as : MapString,String configuration = EmbeddedGraphDatabase.loadConfigurations( neo4j_config.props ); GraphDatabaseService graphDb = new EmbeddedGraphDatabase( my-neo4j-db/, configuration ); my graphDbService is configured the in app-config.xml: bean id=graphDbService class=org.neo4j.kernel.EmbeddedGraphDatabase init-method=enableRemoteShell destroy-method=shutdown constructor-arg index=0 value=/home/neo/var/neo4j-db/ /bean and auto-wired using the @Autowired annotation: @Autowired private GraphDatabaseService graphDbService; Thanks a lot Paddy On Sat, Jul 10, 2010 at 3:38 PM, Paddy paddyf...@gmail.com wrote: hi, Thanks for the advice, Would it be better to use Amazon S3 or EBS storage to mount the neo4j database? thanks Paddy On Thu, Jul 8, 2010 at 6:24 AM, Dave butlerdi butle...@gmail.com wrote: We have just started putting the NK stuff together as they recently moved to V4 and it is quite a curve for us. (Also not much time due to August release). Have been using the NK stuff for 8 years or so and it a real good platform for handling cache, execution scheduling et al. Real smart guys over there. Over the years we have written many accessors (XMPP, JXTA, JINI (Spaces) and many other protocols as well as ASN1(PER) libraries to replace XML in messaging. Now just doing the Wave, Neo4j and Allegrograph stuff at present. As soon as we have something that is readable we will send a message. You will be able to install from the NK installer and the jars (as is common in NK will contain scripts, docs, code ...). On 8 July 2010 15:18, Peter Neubauer peter.neuba...@neotechnology.com wrote: Dave, sounds very interesting to have a netkernel based REST support for Neo4j. Would be great to get some details on that - is the code accessible somewhere to learn more about it? This might be an interesting programming model even for others ... Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Jul 8, 2010 at 2:59 PM, Dave butlerdi butle...@gmail.com wrote: Yep, that is fine. Just using Neo for an RDF store (dense tripple). It works ok but we need to do a lot more work, so still using Allegrograph more (3/4). OWL2 and Reasoner support for dynamic reasoning is really cool for us. . At present we are witing a NetKernel Module for Neo that will give us a very stable and fast neo Rest service as well as provide easy scripting of apps (DPML, Groovy JS, Ruby, PHP, Python ...). Will post when we get done (mid August most likeley). At present Neo is used to store RDF ised messages from barcode and RFID readers in GS1 formats , events are then passed to Robots which handle messaging and update Google Waves. It is all M2M except for dashboard. Sail and all really need better docs. We have worked most of it out and when we finish we will do some docs for internal use. You may also find some use for them. On 8 July 2010 14:41, Peter Neubauer peter.neuba...@neotechnology.com wrote: Dave, you working on the http://pharm2phork.org project? Seems awesome! Are you using the Neo4j RDF components in there? Would be nice to get some feedback on your experience in working with them! I added the project to http://wiki.neo4j.org/content/Neo4j_In_The_Wild#Other if that is ok? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Jul 8, 2010 at 2:30 PM, Dave butlerdi butle...@gmail.com wrote: There was a Cent OS AMI which contained Jetty in the EU West zone. Or else just use a Tomcat instance like ami-45e7002c on US East. Both were fine with Neo4j. You can also just embed jetty and neo into a wrapper and run. On 8 July 2010 14:22, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, yes, this is about the things to consider. I think you should be fine. You might need to tweak some filesystem stuff and the memory mapping settings later, according to the Performance Guidelines, http
[Neo4j] Neo4j with Amazon EC2 Setup
Hi, I'm trying to configure neo4j to run on an amazon ec2 instance. This is my first time using ec2, I'd like to deploy a neo4j web app war file to Jetty. So far I have been reading the following tutorials on how to setup Jetty on ec2. If anyone has any previous experience deploying a neo4j app to amazon ec2 please let me know if I'm going in the right direction. 1. Create a fedora AMI with Jetty using the following tutorial http://docs.codehaus.org/display/JETTY/Jetty+on+Amazon+Elastic+Compute+Cloud+%28Amazon+EC2%29++Tutorial 2. make sure it is formatted it with ext3 http://wiki.neo4j.org/content/Neo4j_in_the_Cloud 3. deploy the war file to jetty http://www.enavigo.com/2008/08/29/deploying-a-web-application-to-jetty Thanks, Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] LuceneFulltextIndexService searching example
sorry should of seen that, thanks for your help
Paddy
On Wed, Jun 30, 2010 at 4:27 AM, Mattias Persson
matt...@neotechnology.comwrote:
As the example states you must use the LuceneFulltextQueryIndexService (not
the LuceneFulltextIndexService)!
2010/6/30 Paddy paddyf...@gmail.com
I have been trying out the example at:
http://wiki.neo4j.org/content/Indexing_with_IndexService
I'm trying to implement LuceneFulltextIndexService searching
I made this test app, just copying and pasting from the example.
The results I'm getting are different from the results expected from the
wiki example.
What am i doing wrong? I'm using the latest 1.1 snapshot.
OUTPUT:
1: nodes size 0
2: nodes size 1
3: nodes size 1
4: nodes size 2
5: nodes size 0
6: nodes size 0
7: nodes size 0
8: nodes size 0
package test;
import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.Transaction;
import org.neo4j.index.IndexHits;
import org.neo4j.index.IndexService;
import org.neo4j.index.lucene.LuceneFulltextIndexService;
import org.neo4j.kernel.EmbeddedGraphDatabase;
public class LuceneTest {
private static GraphDatabaseService graphDb;
private static IndexService index;
public static void main( final String[] args ){
graphDb = new EmbeddedGraphDatabase(target/neo4j-db-lucene6);
index= new LuceneFulltextIndexService(graphDb);
Transaction tx = graphDb.beginTx();
try
{
Node andy = graphDb.createNode();
Node larry = graphDb.createNode();
andy.setProperty( name, Andy Wachowski );
andy.setProperty( title, Director );
larry.setProperty( name, Larry Wachowski );
larry.setProperty( title, Director );
index.index( andy, name, andy.getProperty( name ) );
index.index( andy, title, andy.getProperty( title ) );
index.index( larry, name, larry.getProperty( name ) );
index.index( larry, title, larry.getProperty( title ) );
IndexHitsNode nodes = index.getNodes( name, wachow*
andy
); // -- andy and larry
System.out.println(1: nodes size + nodes.size());
nodes = index.getNodes( name, Andy ); // -- andy
System.out.println(2: nodes size + nodes.size());
nodes = index.getNodes( name, andy ); // -- andy
System.out.println(3: nodes size + nodes.size());
nodes = index.getNodes( name, wachowski ); // -- andy and
larry
System.out.println(4: nodes size + nodes.size());
nodes = index.getNodes( name, +wachow* +larry ); // --
larry
System.out.println(5: nodes size + nodes.size());
nodes = index.getNodes( name, andy AND larry ); // --
System.out.println(6: nodes size + nodes.size());
nodes = index.getNodes( name, andy OR larry ); // -- andy
and larry
System.out.println(7: nodes size + nodes.size());
nodes = index.getNodes( name, Wachowski AND larry ); //
--
larry
System.out.println(8: nodes size + nodes.size());
tx.success();
}
finally
{
tx.finish();
graphDb.shutdown();
}
}
}
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] DefaultExpander.java replacement?
Hi, DefaultExpander.java was removed from the latest build https://trac.neo4j.org/changeset/4590 How can i get the example from github working without the DefaultExpander ? http://github.com/neo4j-examples/java-astar-routing DefaultExpander relExpander = new DefaultExpander(); relExpander.add( RelationshipTypes.ROAD, Direction.BOTH ); AStar sp = new AStar( graphDb, relExpander, costEval, estimateEval ); Path path = sp.findSinglePath( NYC.getUnderlyingNode(), SF.getUnderlyingNode() ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] bus timetable with neo4j
Hey Guys, need a bit more help with this one I'm trying implement the bus timetable into the graph and also allow for walking distances between stops to be calculated Every journey for each route is stored with time data, the cost between each node is the travel time in minutes between the stops. Multiple bus stops can be at the same coordinate Input: start coordinate, end coordinate and time e.g 8:50am 1. Firstly find all stops within walking distance of start coordinate end coordinate using Neo4j spatial query 2. Add connections from the start coordinate to all stops within walking distance that have a bus departure time 8.50am and with walking distance in minutes as the cost 3. Add connections from the end coordinate to all stops with walking distance in minutes as the cost 1. Add relations for stops on same route and run. Stops on the same route but different run cannot connect example bus routes: Route 1, Run 1: (stop: 01)-(10)-(stop:02)-(15)-(stop:03)-(15)-(stop:04)-(30)-(stop:05) 09:00am 09:10am 09:25am09:40 10:10 Route 1, Run 2: (stop: 01)-(10)-(stop:02)-(15)-(stop:03)-(15)-(stop:04)-(30)-(stop:05) 09:30am 09:40am 09:55am10:10 10:40 Route 1, Run 3: (stop: 01)-(10)-(stop:02)-(15)-(stop:03)-(15)-(stop:04)-(30)-(stop:05) 10:00am 10:10am 10:25am10:40 11:10 Route 2, Run 1: (stop: 01)-(20)-(stop:02)-(15)-(stop:03)-(30)-(stop:04)-(30)-(stop:05) 08:30am 08:50am 09:15am09:40 10:10 Route 2, Run 2: (stop: 01)-(20)-(stop:02)-(15)-(stop:03)-(30)-(stop:04)-(30)-(stop:05) 09:20am 09:40am 09:55am10:25 10:55 4.Stops that share the same coordinates can connect if a bus leaves after the current stops arrival time E.g 1: Bus (Route 1, Run 1, stop: 01) can connect to any stop at the same coordinate with a time 9.00am if it is on different route 5.Nearby stops can connect: e.g if there is a nearby stop with departure time= 9:40am and the current stop arrival time is 9:30. They can connect if the walking distance is less than 10 min The arrival time would be calculated by adding the the total cost to the start time please let me know if you have any thoughts or any other ways to do it. On Wed, Jun 9, 2010 at 1:57 AM, Mattias Persson matt...@neotechnology.comwrote: For each bus stop node you could have a relationship to that route's timetable, represented by a separate timetable node. So all nodes for a route could just have a relationship to that route's timetable node where the actual timetable is stored, minimizing redundant information in your graph. A cool use of A*, I might add. 2010/6/9 Paddy paddyf...@gmail.com: I have nodes representing bus stops, and relations representing bus connections with cost values representing minutes it takes for the bus to reach each stop. I'm currently using the A Star algorithm to find the quickest route for the bus routes. what is the best way to integrate the timetable data with a neo4j graph. Route 117: (stop: 01) -mins:3 - (stop:02) -mins:6 - (stop:03) -mins:3 - (stop:04) -mins:3 - (stop:05) Route 203: (stop: 01) -mins:10 - (stop:08) -mins:6 - (stop:09) -mins:3 - (stop:11) Route 302: (stop: 03) -mins:9 - (stop:08) -mins:6.5 - (stop:06) -mins:3.3 - (stop:10) -mins:7 - (stop:01) A trip could be on multiple bus routes e.g a user can get bus route 1 and change to bus route 3 at any stop to get the quickest route There are relationships between stops within walking distance, the cost is represented by walking time in minutes. The timetable is currently stored in SQL as: Route, Day, Stop #, Time 117, Monday, 1, 9:00 117, Monday, 2, 9:03 117, Monday, 3, 9:09 117, Monday, 4, 9:12 117, Monday, 5, 9:15 .. . 117, Monday, 1, 9:30 117, Monday, 2, 9:33 .. . 117, Monday, 1, 10:50 117, Monday, 2, 10:53 .. . If i have the timetable data as a property field for each node, can i use this data in the astar algorithm? Any help would be great Thanks ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j spatial within distance query
Thanks
I got it working using Davide's example. I can find all bus stops within 1km
radius of a coordinate.
For example,
User form input:
Start - longitude= 53.352788, latitude= -6.264396
End - longitude=53.352788, latitude=-6.264396
1 - To generate a list containing bus stops within 1km of the start and the
end coordinates
Point refPoint = layer.getGeometryFactory().createPoint(new
Coordinate(longitude, latitude));
Search search = new SearchPointsWithinOrthodromicDistance(refPoint, 1);
reader.executeSearch(search);
ListSpatialDatabaseRecord stopsWithin1kmList =
resultssearch.getResults();
Result containing a list of bus stops within 1km with coordinates
{StopID : 1101100038, lat : 53.352788, lng : -6.264396}
{StopID : 1100300027, lat : 53.352788, lng : -6.264396}
{StopID : 1100401034, lat : 53.353443, lng : -6.265253}
{StopID : 1100400029, lat : 53.353443, lng : -6.265253}
2- The Astar shortest path algorithm which can find the quickest route
between two bus stops from dublin bus network stored in a neo4j database
Node start = index.getSingleNode( Waypoint.STOPID, 1103700013 );
Node end = index.getSingleNode( Waypoint.STOPID, 1103700017 );
WeightedPath path = astar.findSinglePath( start, end );
Returns a path of BusStopID's: 1103700013, 1103700014, 1103700015,
1103700016, 1103700017
PathToString:
:(5241)--BUS,5136]--(5242)--BUS,5137]--(5243)--BUS,5138]--(5244)--BUS,5145]--(5251)
Cost: 20
3 - Calculate distances from start and end node for each stop within 1km,
start - 1101100038 = 0.05km - walking time = 1 minute
start - 1100300027 = 0.15km- walking time = 3 minute
start - 1100401034 = 0.23km- walking time = 4 minute
start - 1100400029 = 0.28km- walking time = 5 minute
endNode - 1101100038 = 0.25km - walking time = 5 minute
endNode - 1100300027, = 0.15km - walking time = 3 minute
4 - I want to combine the spatial and Astar algorigthms to include walking
distance times into the Astar algorightm
-the user picks these coordinates by selecting a point on a Google Map
-Input start and end coordinates from a json with using Ajax/Spring MVC
Using neo4j a route is calculated and returned to the browser in a JSON
and displayed on a Map, with route information
I can see two ways of doing it, and I would like some advise which would
be best suited.
//Create a start and end node in the database using the start and the
end coordinates
Start - longitude= 53.352788, latitude= -6.264396
End - longitude=53.55274, latitude=-6.164396
String startCoordinateId = 53.352788,-6.264396;
String endStopId = 53.55274,-6.164396;
//Insert a startNode and endNode into the database with the
(lat,lng) as id
Node startNode = graph.makeNode(startCoordinateId);
startNode.setProperty(stop, startCoordinateId);
indexService.index(startNode, stop, startNode.getProperty(stop));
Node endNode = graph.makeNode(endCoordinateId);
endNode.setProperty(stop, endCoordinateId);
indexService.index(endNode, stop, endNode.getProperty(stop));
Add relationships to each bus stop within walking distance
startNode - 1101100038, cost: 1
startNode - 1100300027, cost: 3
startNode - 1100401034, cost: 4
startNode - 1100400029, cost: 5
..
endNode - 1101100038, cost: 1
endNode - 1100300027, cost: 3
//Calculate path from startCoordinate to endCoordinate
Node start = index.getSingleNode( Waypoint.STOPID, startCoordinateId);
Node end = index.getSingleNode( Waypoint.STOPID, endCoordinateId );
WeightedPath path = astar.findSinglePath( startNode, endNode );
Another option would be to calculate the route from multiple start and end
points, that adds the walking distance time to each start/end point as a
cost factor.
As I would prefer not to insert any data into the graph when calculating the
route so it is read only.
I would also add timetable data to find the shortest route time and also
fare cost information at a later date.
I would appreciate any help, thanks
Paddy
On Sat, Jun 5, 2010 at 12:09 PM, Davide dav...@davidesavazzi.net wrote:
On Fri, Jun 4, 2010 at 04:41, Paddy paddyf...@gmail.com wrote:
I have been looking at the neo4j spatial project but there is not much
docs,
is it possible to do this using neo4j spatial or lucene query?
It's possible with both. If you use Latitude / Longitude coordinates
you have to keep in mind that to calculate the distance between two
points you can't use the Euclidean distance but some more complicated
formula:
http://en.wikipedia.org/wiki/Geographical_distance#Spherical-surface_formulae
http://www.movable-type.co.uk/scripts/latlong-db.html
I've just committed a Search class on github that should implement
what you're looking for.
I've made a little test, loading MaxMind Cities database, and
searching within n km of my
[Neo4j] bus timetable with neo4j
I have nodes representing bus stops, and relations representing bus connections with cost values representing minutes it takes for the bus to reach each stop. I'm currently using the A Star algorithm to find the quickest route for the bus routes. what is the best way to integrate the timetable data with a neo4j graph. Route 117: (stop: 01) -mins:3 - (stop:02) -mins:6 - (stop:03) -mins:3 - (stop:04) -mins:3 - (stop:05) Route 203: (stop: 01) -mins:10 - (stop:08) -mins:6 - (stop:09) -mins:3 - (stop:11) Route 302: (stop: 03) -mins:9 - (stop:08) -mins:6.5 - (stop:06) -mins:3.3 - (stop:10) -mins:7 - (stop:01) A trip could be on multiple bus routes e.g a user can get bus route 1 and change to bus route 3 at any stop to get the quickest route There are relationships between stops within walking distance, the cost is represented by walking time in minutes. The timetable is currently stored in SQL as: Route, Day, Stop #, Time 117, Monday, 1, 9:00 117, Monday, 2, 9:03 117, Monday, 3, 9:09 117, Monday, 4, 9:12 117, Monday, 5, 9:15 .. . 117, Monday, 1, 9:30 117, Monday, 2, 9:33 .. . 117, Monday, 1, 10:50 117, Monday, 2, 10:53 .. . If i have the timetable data as a property field for each node, can i use this data in the astar algorithm? Any help would be great Thanks ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4j spatial within distance query
I would like to store coordinates as nodes in neo4j to do simple querys For example a list of 10,000 nodes: Node[1] - lon=-74.007124, lat=40.714550 Node[2] - lon=-94.626824, lat=39.113380 Node[3] - lon=-105.937406, lat=35.691543 Node[4] - lon=-122.329439, lat=47.603560 Node[5] - lon=-122.420139, lat=37.779600 .. .. I would like to find all nodes within 10km of the following coordinate :(lon=-120.42, lat=37.76) I have been looking at the neo4j spatial project but there is not much docs, is it possible to do this using neo4j spatial or lucene query? Thanks Patrick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
