Re: [Neo4j] Use of LuceneFulltextIndexBatchInserter
I posted a question on MathOverflow http://mathoverflow.net/questions/35563/best-algorithm-to-use-for-a-bus-train-transport-route-planner and I emailed the authors of the slides, hopefully we can get some more insights into the bidirectional A* or the best algorithm to use. A lot of research has been done in this field, this describes a A* algorithm with landmarks: : http://www.optimization-online.org/DB_HTML/2008/11/2154.html Also the Dijkstra’s Algorithm On-Line: An Empirical Case Study from Public Railroad Transport paper also looks good cheers Paddy On Fri, Aug 13, 2010 at 10:06 PM, Paddy paddyf...@gmail.com wrote: Thanks Peter, Sounds good, Yes that would be a great addition to the graph-algo! I would love to contribute but I'm not sure where to begin, it looks pretty complicated. Also I noticed the ExperimentalAStar in org.neo4j.graphalgo.impl.path . What is the difference between AStar.java and the ExperimentalAStar.java ? found some very interesting slides too regarding a bidirectional A*: Bidirectional A* on Time-Dependent Graphs http://ctw08.dti.unimi.it/slides/B5/B5-1-Nannicini.pdf It describes the European road network as: roughly 18M nodes, 42M arcs. and A very good implementation of Dijkstra’s algorithm explores on average 9M nodes and takes more than 6 seconds (european road network): it’s too long! http://www.cs.princeton.edu/courses/archive/spr06/cos423/Handouts/EPP%20shortest%20path%20algorithms.pdf- page 4 Thanks Paddy On Fri, Aug 13, 2010 at 2:21 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Paddy, apart from tweaking the system, we have been thinking about a t that would start from both ends and route. Not sure how advanced that would be but I think it could come in handy. If you have time later on, that might be an interesting place to investigate some contribution to the graph-algo package :) Overall, the field of routing algos seems to be a very wide one, so I think once things are working at your end, there is a whole lot regarding better suitable algos taking into account special context and constraints that could be done ... Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 11:14 PM, Paddy paddyf...@gmail.com wrote: Hi Peter, Nearly there :) I'm going to make a few changes to try increase the speed of the a*star route finder, as the current graph is 1.2gb with 1million nodes and 10 million relationships. - reducing the database size by only including bus routes between 7am to 1pm on Mondays instead of every bus route for every day - configuring the neo4j_config.props for traversal and injecting it in the spring bean - creating less relationships by adding less walking connections between stops e.g only add connections for stops within 50m this will reduce the size of the db and hopefully this will speed up the path finding and as it can take 5-10 seconds for long routes with the current graph. I will let you know Cheers Paddy On Thu, Aug 12, 2010 at 11:27 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Glad you solved it! Btw, is the site now officially online? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 4:38 AM, Paddy paddyf...@gmail.com wrote: Hi, so I just needed to use :) : LuceneFulltextIndexBatchInserter index = new LuceneFulltextIndexBatchInserter( inserter ); Good batch insert example @ https://svn.neo4j.org/components/index/trunk/src/test/java/org/neo4j/index/lucene/TestBatchInsert.java Paddy On Wed, Aug 11, 2010 at 8:47 PM, Paddy paddyf...@gmail.com wrote: Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org
Re: [Neo4j] Use of LuceneFulltextIndexBatchInserter
Glad you solved it! Btw, is the site now officially online? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 4:38 AM, Paddy paddyf...@gmail.com wrote: Hi, so I just needed to use :) : LuceneFulltextIndexBatchInserter index = new LuceneFulltextIndexBatchInserter( inserter ); Good batch insert example @ https://svn.neo4j.org/components/index/trunk/src/test/java/org/neo4j/index/lucene/TestBatchInsert.java Paddy On Wed, Aug 11, 2010 at 8:47 PM, Paddy paddyf...@gmail.com wrote: Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Use of LuceneFulltextIndexBatchInserter
Hi Peter, Nearly there :) I'm going to make a few changes to try increase the speed of the a*star route finder, as the current graph is 1.2gb with 1million nodes and 10 million relationships. - reducing the database size by only including bus routes between 7am to 1pm on Mondays instead of every bus route for every day - configuring the neo4j_config.props for traversal and injecting it in the spring bean - creating less relationships by adding less walking connections between stops e.g only add connections for stops within 50m this will reduce the size of the db and hopefully this will speed up the path finding and as it can take 5-10 seconds for long routes with the current graph. I will let you know Cheers Paddy On Thu, Aug 12, 2010 at 11:27 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Glad you solved it! Btw, is the site now officially online? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 4:38 AM, Paddy paddyf...@gmail.com wrote: Hi, so I just needed to use :) : LuceneFulltextIndexBatchInserter index = new LuceneFulltextIndexBatchInserter( inserter ); Good batch insert example @ https://svn.neo4j.org/components/index/trunk/src/test/java/org/neo4j/index/lucene/TestBatchInsert.java Paddy On Wed, Aug 11, 2010 at 8:47 PM, Paddy paddyf...@gmail.com wrote: Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Use of LuceneFulltextIndexBatchInserter
Paddy, apart from tweaking the system, we have been thinking about a double-sided A* that would start from both ends and route. Not sure how advanced that would be but I think it could come in handy. If you have time later on, that might be an interesting place to investigate some contribution to the graph-algo package :) Overall, the field of routing algos seems to be a very wide one, so I think once things are working at your end, there is a whole lot regarding better suitable algos taking into account special context and constraints that could be done ... Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 11:14 PM, Paddy paddyf...@gmail.com wrote: Hi Peter, Nearly there :) I'm going to make a few changes to try increase the speed of the a*star route finder, as the current graph is 1.2gb with 1million nodes and 10 million relationships. - reducing the database size by only including bus routes between 7am to 1pm on Mondays instead of every bus route for every day - configuring the neo4j_config.props for traversal and injecting it in the spring bean - creating less relationships by adding less walking connections between stops e.g only add connections for stops within 50m this will reduce the size of the db and hopefully this will speed up the path finding and as it can take 5-10 seconds for long routes with the current graph. I will let you know Cheers Paddy On Thu, Aug 12, 2010 at 11:27 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Glad you solved it! Btw, is the site now officially online? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Aug 13, 2010 at 4:38 AM, Paddy paddyf...@gmail.com wrote: Hi, so I just needed to use :) : LuceneFulltextIndexBatchInserter index = new LuceneFulltextIndexBatchInserter( inserter ); Good batch insert example @ https://svn.neo4j.org/components/index/trunk/src/test/java/org/neo4j/index/lucene/TestBatchInsert.java Paddy On Wed, Aug 11, 2010 at 8:47 PM, Paddy paddyf...@gmail.com wrote: Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Use of LuceneFulltextIndexBatchInserter
Hi, so I just needed to use :) : LuceneFulltextIndexBatchInserter index = new LuceneFulltextIndexBatchInserter( inserter ); Good batch insert example @ https://svn.neo4j.org/components/index/trunk/src/test/java/org/neo4j/index/lucene/TestBatchInsert.java Paddy On Wed, Aug 11, 2010 at 8:47 PM, Paddy paddyf...@gmail.com wrote: Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Use of LuceneFulltextIndexBatchInserter
Hi, I would like to use the LuceneFulltextIndexBatchInserter instead of the LuceneIndexBatchInserter to allow fulltext lucene indexing. How can I initialize the LuceneFulltextIndexBatchInserter ? the example on http://wiki.neo4j.org/content/Batch_Insert gives an example for the LuceneIndexBatchInserter but not for LuceneFulltextIndexBatchInserter LuceneIndexBatchInserter indexService = new LuceneIndexBatchInserterImpl( inserter ); thanks Paddy ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
