[Neo4j] Achim retweeted a nice edge weight visualization
Looks really good, especially the last, interactive one. http://worrydream.com/VisualizingEdgeWeights/ Cheers Michael ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] finding all shortest paths between one node and all other nodes in a large scale databse
Yes, you can probably do this thing in one traversal. Shortest path will give you the shortest path(s) between two given nodes, but are interested in any path, right? And you can find paths to several different end nodes in one traversal. Just specify an Evaluator which knows about that, or let loose a breadth-first traversal and filter your paths afterwards. 2011/5/11 Michael Hunger michael.hun...@neotechnology.com Hey JueiTing, I'm not sure if Hadoop is needed here. What is the current performance characteristics for the shortest path you are using? You could take a decent machine and just fire up, e.g. blocks of 10k node pairs to a ThreadPoolExecutor with cores*2 threads. Each of those tasks only has to return the sum of the path lengths (and you know the block size) so you can sum the whole thing up onto a long and divide it by the number of pairs processed at the end? Perhaps instead of brute force looping one and a half times over all nodes it is perhaps better to do a breath first traversal over all nodes (regardless of relationships, just outgoing rels) and returning unique paths from the traverser and calculating the path lenghts from the start nodes to each _connected_ node on the paths (start node - path[0..n].nodes[1..n]) Cheers Michael Am 11.05.2011 um 21:12 schrieb Peter Neubauer: Hi JueiTing, I think this is a typical case for a massive Map/Reduce job. I am thinking of combining Hadoop works with replicas of the graph and then do the computation. I believe Paddy Fitzgerald has been working with these approaches and can give some feedback. Of course, given the size of the graph, that might prove a problem. OTOH, if there are no modifications during the computation, you could run the calculations on read-only databases from the same store. Would that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, May 11, 2011 at 12:47 AM, 翁瑞廷 wshir...@gmail.com wrote: Hi, I'm trying to use Neo4j graph database to store a large social network(more than 5,000,000 nodes) for academic research. I need to compute the separation degree(path length) between any two nodes in the graph then get the average degree of whole database. The solution I'm using use now is archieved by executing API GraphAlgoFactory.shortestPath, but it means I need to execute (n*(n-1))/2 times to get all path length. I don't think it's a very good idea :( So, I'm wondering that is there any function which can assign one node and whole DB as Input, and return the paths or only path lengths between the node and all other nodes in the graph as Output? Just like Dijkstra algorithm did. I know there is a Dijkstra function in the API, but it only return the path between two nodes. It'll be really helpful for me if there is any function can return all the result in one computation(even the whole result array computing by Dijkstra) Or does anyone has better idea to deal with this issue? I tried to implement Dijkstra by myself, but it was impossible to declare such big 2D array in the program...(OutOfMemoryException) If there is any advice, I'll really appreciate it. Thanks. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] finding all shortest paths between one node and all other nodes in a large scale databse
Thanks for all your response, Here is the size of the grapth db: NodesSize - 100,000 97MB 200,000 182MB 300,000 267MB ... 5,000,000 expect 5GB I've tried to use 5 virtual machines, each one has 2 cores and 1G memory, Running 2 threads on each VM. Nodes Time -- 50,000 20mins 150,00040hrs Obviously, when the amount of node increases, the spending time of executing GraphAlgoFactory.shortestPath increases heavily and non-linearly. It's hard to estimate how many machines I need if I want to deal with 5,000,000 nodes or even 10,000,000, I think Map/Reduce will be one solution for me, and I'll try to use BFS traversal which may reduce some duplicate procedures. Nice to discuss with you, Thanks. 2011/5/12 Michael Hunger michael.hun...@neotechnology.com Hey JueiTing, I'm not sure if Hadoop is needed here. What is the current performance characteristics for the shortest path you are using? You could take a decent machine and just fire up, e.g. blocks of 10k node pairs to a ThreadPoolExecutor with cores*2 threads. Each of those tasks only has to return the sum of the path lengths (and you know the block size) so you can sum the whole thing up onto a long and divide it by the number of pairs processed at the end? Perhaps instead of brute force looping one and a half times over all nodes it is perhaps better to do a breath first traversal over all nodes (regardless of relationships, just outgoing rels) and returning unique paths from the traverser and calculating the path lenghts from the start nodes to each _connected_ node on the paths (start node - path[0..n].nodes[1..n]) Cheers Michael Am 11.05.2011 um 21:12 schrieb Peter Neubauer: Hi JueiTing, I think this is a typical case for a massive Map/Reduce job. I am thinking of combining Hadoop works with replicas of the graph and then do the computation. I believe Paddy Fitzgerald has been working with these approaches and can give some feedback. Of course, given the size of the graph, that might prove a problem. OTOH, if there are no modifications during the computation, you could run the calculations on read-only databases from the same store. Would that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, May 11, 2011 at 12:47 AM, 翁瑞廷 wshir...@gmail.com wrote: Hi, I'm trying to use Neo4j graph database to store a large social network(more than 5,000,000 nodes) for academic research. I need to compute the separation degree(path length) between any two nodes in the graph then get the average degree of whole database. The solution I'm using use now is archieved by executing API GraphAlgoFactory.shortestPath, but it means I need to execute (n*(n-1))/2 times to get all path length. I don't think it's a very good idea :( So, I'm wondering that is there any function which can assign one node and whole DB as Input, and return the paths or only path lengths between the node and all other nodes in the graph as Output? Just like Dijkstra algorithm did. I know there is a Dijkstra function in the API, but it only return the path between two nodes. It'll be really helpful for me if there is any function can return all the result in one computation(even the whole result array computing by Dijkstra) Or does anyone has better idea to deal with this issue? I tried to implement Dijkstra by myself, but it was impossible to declare such big 2D array in the program...(OutOfMemoryException) If there is any advice, I'll really appreciate it. Thanks. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Index Framework - Query Question
Could you perhaps create an index with a custom Analyzer which treats titles as fulltext and others as non-analyzed? 2011/5/11 Rick Bullotta rick.bullo...@thingworx.com Is there currently any way to have a composite index consisting of fulltext and non-analyzed (simple fields), and to query them in a single query statement? e.g. : title:Reloaded AND year:1999 In this case, I'd be using a Lucene Analyzer on the title, but indexing the primitive types without using the analyzer/norms/etc...I know how to do it with Lucene directly, but not clear if it can be done w/Neo4J. Thanks! Rick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Retrieving a node linked to the reference node takes forever
2011/5/11 Pablo Pareja ppar...@era7.com Hi, I just tested the same code with ~ 50.000 nodes in the DB and it works just fine. Why should it be different for scenarios with many more nodes? Actually, even though there are millions of nodes they are not directly related at all with the one where the program gets stuck. In fact there only are ~ 50 nodes linked to the reference node and at that point there isn't any other relationship liked to that node. Sounds weird. How much heap have you given your JVM? Have you supplied custom memory-mapping settings to neo4j? I was wondering, could it be possible that the cause of this issue would be trying to get the node id (long value) ? How's that implemented? Could there be any performance issues in the case of retrieving the id of a node whenever there are already a lot of nodes in the DB? Retreiving a node, if in the cache is in essence a HashMap lookup. If it needs to be loaded its an O(1) operation since the id tells neo4j the exact location in the nodestore file to get it from. So no matter how many nodes there are in the database a getNodeById is always O(1)... apart from that the file system/OS doesn't have that part of the file cached a.t.m. and by then if will be slower of course. Cheers, Pablo Pareja On Tue, May 10, 2011 at 11:54 AM, Pablo Pareja ppar...@era7.com wrote: Hi, I don't want to bother you all but are there any news/insights about this? Don't hesitate to tell me any other test I can carry out so that we can get more information about what could be going on. Cheers, Pablo Pareja On Mon, May 9, 2011 at 12:34 PM, Pablo Pareja ppar...@era7.com wrote: Ok, I'm running iostat command and this is what I'm getting so far: *(First results)* avg-cpu: %user %nice %system %iowait %steal %idle 0.050.000.030.420.30 99.20 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvdap11.8964.9814.25 161562 35424 xvdg 8.7096.3821.01 239634 52232 *(After ~ 10 minutes with the program stuck)* Linux 2.6.35.11-83.9.amzn1.x86_64 (ip-10-230-37-212)09/05/11 _x86_64_(4 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 0.070.050.213.020.26 96.39 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvdap13.8380.9515.04 227594 42280 xvdg107.06 880.5118.582475482 52232 On Mon, May 9, 2011 at 10:17 AM, Mattias Persson matt...@neotechnology.com wrote: So I'm assuming there's a high I/O load when this happens? It seems to be loading up one or more persistence windows (a part of the database) into memory. Could you verify that high I/O load with the iostat command (if you're on linux) or any process/system monitor? 2011/5/6 Pablo Pareja ppar...@era7.com There you go: Full thread dump Java HotSpot(TM) 64-Bit Server VM (19.0-b09 mixed mode): Low Memory Detector daemon prio=10 tid=0x7fa5ac0b7000 nid=0x526 runnable [0x] java.lang.Thread.State: RUNNABLE CompilerThread1 daemon prio=10 tid=0x7fa5ac0b5000 nid=0x525 waiting on condition [0x] java.lang.Thread.State: RUNNABLE CompilerThread0 daemon prio=10 tid=0x7fa5ac0b2000 nid=0x524 waiting on condition java.lang.Thread.State: RUNNABLE Signal Dispatcher daemon prio=10 tid=0x7fa5ac0b nid=0x523 waiting on condition [0x] java.lang.Thread.State: RUNNABLE Finalizer daemon prio=10 tid=0x7fa5ac091800 nid=0x522 in Object.wait() [0x7fa5a20a2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on 0x000103b4afb0 (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118) - locked 0x000103b4afb0 (a java.lang.ref.ReferenceQueue$Lock) [ec2-user@ip-10-230-55-104 bio4j_volume]$ at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) Reference Handler daemon prio=10 tid=0x7fa5ac08f800 nid=0x521 in Object.wait() [0x7fa5a21a3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on 0x000103b4af48 (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:485) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) - locked 0x000103b4af48 (a java.lang.ref.Reference$Lock) main prio=10 tid=0x7fa5ac007000 nid=0x51b runnable [0x7fa5b06cd000]
[Neo4j] Error while creating EXE from JAR and Runnable JAR
Was creating an exe by exporting JAR and Runnable JARs thro eclipse.. Using Launch4j to do that. and having proper manifest files with the class path too.. But i get the following error when i try to run the created exe file. Caused by: java.Lang. ClassNotFoundException: org.neo4j.graphdb.GraphDatabaseService at java.net.URLClassLoader$1.run at java.security.AccessController.doPrivileged at java.net.URLClassLoader.findClass at java.Land.ClassLoader.loadClass at sun.misc.Launcher$AppClassLoader.loadClass at java.Lang.ClassLoader.loadClass Could not find the main class: package name.Class name is there a solution to this.?? i had actually created an exe earlier which was working perfectly fine.. Plz help! Thanks. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Error while creating EXE from JAR and Runnable JAR
Could you supply how your generated classpath looks like? 2011/5/12 Abhishek AS abhishek@gmail.com Was creating an exe by exporting JAR and Runnable JARs thro eclipse.. Using Launch4j to do that. and having proper manifest files with the class path too.. But i get the following error when i try to run the created exe file. Caused by: java.Lang. ClassNotFoundException: org.neo4j.graphdb.GraphDatabaseService at java.net.URLClassLoader$1.run at java.security.AccessController.doPrivileged at java.net.URLClassLoader.findClass at java.Land.ClassLoader.loadClass at sun.misc.Launcher$AppClassLoader.loadClass at java.Lang.ClassLoader.loadClass Could not find the main class: package name.Class name is there a solution to this.?? i had actually created an exe earlier which was working perfectly fine.. Plz help! Thanks. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] User Digest, Vol 50, Issue 44
The classpath seems to be just a dot under the class path label. Message: 4 Date: Thu, 12 May 2011 11:02:30 +0200 From: Mattias Persson matt...@neotechnology.com Subject: Re: [Neo4j] Error while creating EXE from JAR and Runnable JAR To: Neo4j user discussions user@lists.neo4j.org Message-ID: BANLkTik8Xry8bj4eJ_Y=nhd3umdc07f...@mail.gmail.com Content-Type: text/plain; charset=UTF-8 Could you supply how your generated classpath looks like? 2011/5/12 Abhishek AS abhishek@gmail.com Was creating an exe by exporting JAR and Runnable JARs thro eclipse.. Using Launch4j to do that. and having proper manifest files with the class path too.. But i get the following error when i try to run the created exe file. Caused by: java.Lang. ClassNotFoundException: org.neo4j.graphdb.GraphDatabaseService at java.net.URLClassLoader$1.run at java.security.AccessController.doPrivileged at java.net.URLClassLoader.findClass at java.Land.ClassLoader.loadClass at sun.misc.Launcher$AppClassLoader.loadClass at java.Lang.ClassLoader.loadClass Could not find the main class: package name.Class name is there a solution to this.?? i had actually created an exe earlier which was working perfectly fine.. Plz help! Thanks. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com -- ___ User mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user End of User Digest, Vol 50, Issue 44 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] finding all shortest paths between one node and all other nodes in a large scale databse
in the gt; gt;gt; graph gt; gt;gt; as Output? Just like Dijkstra algorithm did. gt; gt;gt; I know there is a Dijkstra function in the API, but it only return the gt; path gt; gt;gt; between two nodes. gt; gt;gt; It'll be really helpful for me if there is any function can return all gt; the gt; gt;gt; result in one computation(even the whole result array computing by gt; gt;gt; Dijkstra) gt; gt;gt; gt; gt;gt; Or does anyone has better idea to deal with this issue? gt; gt;gt; I tried to implement Dijkstra by myself, but it was impossible to gt; declare gt; gt;gt; such big 2D array in the program...(OutOfMemoryException) gt; gt;gt; gt; gt;gt; If there is any advice, I'll really appreciate it. gt; gt;gt; gt; gt;gt; Thanks. gt; gt;gt; ___ gt; gt;gt; Neo4j mailing list gt; gt;gt; User@lists.neo4j.org gt; gt;gt; https://lists.neo4j.org/mailman/listinfo/user gt; gt;gt; gt; gt; ___ gt; gt; Neo4j mailing list gt; gt; User@lists.neo4j.org gt; gt; https://lists.neo4j.org/mailman/listinfo/user gt; gt; ___ gt; Neo4j mailing list gt; User@lists.neo4j.org gt; https://lists.neo4j.org/mailman/listinfo/user gt; ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Caselle da 1GB, trasmetti allegati fino a 3GB e in piu' IMAP, POP3 e SMTP autenticato? GRATIS solo con Email.it: http://www.email.it/f Sponsor: Per il ponte del 2 giugno scegli le offerte speciali dei Riccione Family Hotels! Bimbi gratis, spiaggia, baby menu, miniclub, parchi divertimento Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=11447d=20110512 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Error while creating EXE from JAR and Runnable JAR
Could you please share your complete project including the detailed steps you are executing to generate the exe. (e.g. via dropbox) With only fragments of the information we can only guess at most. And with the shared project perhaps also other mailing-list users can help as well. Thanks so much Michael Am 12.05.2011 um 11:02 schrieb Mattias Persson: Could you supply how your generated classpath looks like? 2011/5/12 Abhishek AS abhishek@gmail.com Was creating an exe by exporting JAR and Runnable JARs thro eclipse.. Using Launch4j to do that. and having proper manifest files with the class path too.. But i get the following error when i try to run the created exe file. Caused by: java.Lang. ClassNotFoundException: org.neo4j.graphdb.GraphDatabaseService at java.net.URLClassLoader$1.run at java.security.AccessController.doPrivileged at java.net.URLClassLoader.findClass at java.Land.ClassLoader.loadClass at sun.misc.Launcher$AppClassLoader.loadClass at java.Lang.ClassLoader.loadClass Could not find the main class: package name.Class name is there a solution to this.?? i had actually created an exe earlier which was working perfectly fine.. Plz help! Thanks. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Gwt + eclipse + Neo4j screencast + testings
hej rene, Right now I would be happy about some best practices to design a software backend using neo4j. You know in MySQL it is kind of straight forward how to build software around the data base. In neo4j a new way of thinking starts. I haven't really looked for it yet. But i guess there won't be anything out there anyway. for a web-context it its is better start the embedded graph-db only once. in your screencast you will do this every request. please try using a context-listener to start / shutdown the embedded db in a clean way. so the access-times in the gwt-servlet will really rock! http://www.mail-archive.com/user@lists.neo4j.org/msg08350.html cheers /thomas I was a little confused that there was a wiki and the other docs but that didn't really bother me. most the time I use google to search for resources anyway. Best regards René 2011/5/9 Peter Neubauer peter.neuba...@neotechnology.com Rene, if you don't mind asking - what are the most valuable docs for you? We are trying to migrate from the wiki.neo4j.org to the generated and tested code snippets and edited documentation at docs.neo4j.org - it would be great to see what users find valuable to keep improving on to get started with Neo4j! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. 2011/5/9 René Pickhardt r.pickha...@googlemail.com Hey Peter, Since I want to redesign the backend of my social networking site for my PhD thesis and I am almost convinced that Neo4j is the way to go I will most certainly work with your graph data base over the next couple months and get to know it better. Since you guys made it open source (great!) I will try to give something back by continuing to blog about it. Once I get to know neo4j better and really start migrating my current data base to it and running the social networking site with neo4j as a backend I am sure to be able to give you guys more valuable feedback. Besides many questions which I can only solve by reading the docs I am satisfied right now. In my very own interest I wish you guys good luck with neo4j and of course many paying customers (-: best regards René 2011/5/9 Peter Neubauer peter.neuba...@neotechnology.com You rock it Rene! Nice summary - please let us know how we can make your experience even more smooth and what you you think should be there for you to have an even better experience? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. 2011/5/9 René Pickhardt r.pickha...@googlemail.com Hey everyone, Peter asked me to post this to the list. So here you go: Screencast explaining how to set up neo4j in eclipse and GWT: http://www.rene-pickhardt.de/how-to-combine-neo4j-with-gwt-and-eclipse/ A short blog article with a sum up of my first testing on using neo4j on the friendship graph (and some other edges between users) from the social networking site I run (~300'000 Edges): http://www.rene-pickhardt.de/neo4j-graph-database-vs-mysql/ Thanks to neotechnology for providing such a great tool! Regards René -- **www.twitter.com/renepickhardt Skype: rene.pickhardt www.rene-pickhardt.de http://www.beijing-china-blog.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- *Das Debütalbum meiner Band IN LEGEND erscheint am 20.5.2010! Video gucken http://www.youtube.com/watch?v=aLesGMA0-rM und bei AMAZON vorbestellen http://goo.gl/3H0Fm (Danke!) * -- mobile: +49 (0)176 6433 2481 Skype: +49 (0)6131 / 4958926 Skype: rene.pickhardt www.rene-pickhardt.de http://www.beijing-china-blog.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- *Das Debütalbum meiner Band IN LEGEND erscheint am 20.5.2010! Video gucken http://www.youtube.com/watch?v=aLesGMA0-rM und bei AMAZON vorbestellen http://goo.gl/3H0Fm (Danke!) * -- mobile: +49 (0)176 6433 2481 Skype: +49 (0)6131
Re: [Neo4j] Gwt + eclipse + Neo4j screencast + testings
Hey Thomas, thank you for your advice! I was reading the discussion with John and already aware of the fact that this is the way to go. I already implemented it and of course it is running much faster (-: I hope you don't mind if I include this peace of work in an upcomming blogpost in which I will also share a very basic gwt + neo4j application. greetz René 2011/5/12 Thomas Baum thomas.b...@neopersistence.com hej rene, Right now I would be happy about some best practices to design a software backend using neo4j. You know in MySQL it is kind of straight forward how to build software around the data base. In neo4j a new way of thinking starts. I haven't really looked for it yet. But i guess there won't be anything out there anyway. for a web-context it its is better start the embedded graph-db only once. in your screencast you will do this every request. please try using a context-listener to start / shutdown the embedded db in a clean way. so the access-times in the gwt-servlet will really rock! http://www.mail-archive.com/user@lists.neo4j.org/msg08350.html cheers /thomas I was a little confused that there was a wiki and the other docs but that didn't really bother me. most the time I use google to search for resources anyway. Best regards René 2011/5/9 Peter Neubauer peter.neuba...@neotechnology.com Rene, if you don't mind asking - what are the most valuable docs for you? We are trying to migrate from the wiki.neo4j.org to the generated and tested code snippets and edited documentation at docs.neo4j.org - it would be great to see what users find valuable to keep improving on to get started with Neo4j! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. 2011/5/9 René Pickhardt r.pickha...@googlemail.com Hey Peter, Since I want to redesign the backend of my social networking site for my PhD thesis and I am almost convinced that Neo4j is the way to go I will most certainly work with your graph data base over the next couple months and get to know it better. Since you guys made it open source (great!) I will try to give something back by continuing to blog about it. Once I get to know neo4j better and really start migrating my current data base to it and running the social networking site with neo4j as a backend I am sure to be able to give you guys more valuable feedback. Besides many questions which I can only solve by reading the docs I am satisfied right now. In my very own interest I wish you guys good luck with neo4j and of course many paying customers (-: best regards René 2011/5/9 Peter Neubauer peter.neuba...@neotechnology.com You rock it Rene! Nice summary - please let us know how we can make your experience even more smooth and what you you think should be there for you to have an even better experience? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. 2011/5/9 René Pickhardt r.pickha...@googlemail.com Hey everyone, Peter asked me to post this to the list. So here you go: Screencast explaining how to set up neo4j in eclipse and GWT: http://www.rene-pickhardt.de/how-to-combine-neo4j-with-gwt-and-eclipse/ A short blog article with a sum up of my first testing on using neo4j on the friendship graph (and some other edges between users) from the social networking site I run (~300'000 Edges): http://www.rene-pickhardt.de/neo4j-graph-database-vs-mysql/ Thanks to neotechnology for providing such a great tool! Regards René -- **www.twitter.com/renepickhardt Skype: rene.pickhardt www.rene-pickhardt.de http://www.beijing-china-blog.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- *Das Debütalbum meiner Band IN LEGEND erscheint am 20.5.2010! Video gucken http://www.youtube.com/watch?v=aLesGMA0-rM und bei AMAZON vorbestellen http://goo.gl/3H0Fm (Danke!) * --
[Neo4j] Neo4j on AWS
Hello, I was wondering if anyone had any suggestions on how best to deploy a Neo4j application to Amazon Web Services. Initially spinning up a single EC2 instance with an EmbeddedGraphDatabase will do, but that solution does not scale. Ideally, I would be able to use ElasticBeanstalk to manage and auto-scale my application servers and have them connect to a single Neo4j server instance. Thoughts? Best, Maarten ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Spring Data Graph Load Time Weaving
Maarten, sorry was away with the family. Yes I get the emails but will respond this one to the ML - content of general interest. I'd say writing a client against the low level rest API is about twice the work and much worse performance wise, because: - you make a lot more requests (.e.g 1+n selects) - transaction scope == 1 request - you transfer much more data than you need To optimize using the REST API you have to write all your database access in terms of traversals (then you automatically transfer a multiple of the data you need). Any modification affects just a single element (property, node, relationship), indexing counts extra. Also index lookup (for instance to assert that data is not inserted twice (also for duplicate relationships) costs a lot more. These were at least my experiences with the domains I worked with against the server. The difference to SQL / HSQLDB is that we have no query language _yet_ which allows for fine grained filtering, traversals and projections. I.e. ways to get exactly that back from the DB what you want, not less not more. Same goes for updates, in SQL you have the DML (insert, update, delete) which also works with large datasets and batching data. We don't have that yet either. That's why creating a domain level protocol that covers the use-cases you want to talk about with the server, e.g. * fetch me all friends that have a xxx offering, or * update all the status information of this list of people, or * give me the recommendations for buying new products for the friends of this guy. speaking in domain language, with a granularity that fits your demands (and is not too fine grained and not to coarse grained) and which transports only that over the wire that you need in your app while being highly performant because it talks directly to the embedded graphdb. You can even use the same domain model (or in this case rather presentation-model) in the server extension and your app-server (grails) that is then just serialized accordingly and transported to the view and controller-layer (i.e. your app-server). This maximizes throughput, creates the correct transaction boundaries and minimizes latency. If you want to you can create this domain level protocol even as a RESTful interface that also other clients/consumers could talk to the Neo4j-Server extensions (e.g. javascript consuming the JSON or ruby or whatever). Regarding HA, yes you can embed Neo4j in your application servers and have all those instances form a cluster that is automatically synced. Being HA it works with transactions and concurrency, no worries. But you're right updates to the other slaves are eventual consistent. That's why you would employ sticky routes so that the user that just wrote information to a slave is routed again to that slave for reading. (See also the Neo4j - manual: http://docs.neo4j.org/chunked/milestone/ha-architecture.html) HTH Michael Am 12.05.2011 um 17:34 schrieb Maarten Haubrich: Sorry for the spam, but I'm not sure if you are getting emails from my maar...@caregaroo.com email address... Thanks for the quick reply, Michael! sorry, but I don't quite understand. What do you mean by a clean domain layer protocol that talks in your domain + use cases are server extensions? Would I write a separate application that runs an embedded neo and abstracts the database away and then have my clients (the actual web servers) use that proprietary protocol to get the data? That seems like a lot of extra work. From an operations standpoint I would rather just deploy a server and have it behave in the same manner as the embedded version (like HSQLDB) - but I understand that things are not quite as simple as that :) Also, it would be helpful if you could expand a bit on the issues with the REST API, since I have just read snippets here and there in the mailing list. Regarding the enterprise version: So with this version I can have my database embedded in each of the application servers and neo will handle keeping all instances in sync for me. How does that work in terms of transactions and concurrency? Is all the data replicated immediately, or is it more like the eventual consistency in Amazon SimpleDb. Thanks a lot for your time. Best, Maarten On 2011-05-12, at 7:25 AM, Michael Hunger michael.hun...@neotechnology.com wrote: Maarten, Cluster does not necessarily mean server. Any Neo4j enterprise instance can run in clustered (HA) mode (embedded or not), server or embedded depends on how you want to access the db - in process (part of your app-server) or out of process - neo4j-REST protocol. There are no other things, but just the REST API as an issue. Which you wouldn't use anyway as a clean domain layer protocol that talks in your domain + use-cases as a server extension is the way to go. Cheers Michael Am 12.05.2011 um 16:15 schrieb Maarten Haubrich:
Re: [Neo4j] Neo4j EC2 AMI image available
Daniel, I would put some Apache server in front of it and secure it on URL patterns, see http://docs.neo4j.org/chunked/snapshot/operations-security.html for a simplicstic example. Would that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, May 11, 2011 at 10:57 PM, Daniel Gasienica dan...@gasienica.chwrote: This is great news! What is the simplest way to secure a Neo4j instance on AWS? Daniel On Tue, May 10, 2011 at 06:29, Peter Neubauer peter.neuba...@neotechnology.com wrote: Hi all, Jussi Heinonen from OpenCredo has created a Neo4j Server EC2 image to try out things with. Well done, thanks for the effort Jussi! Read more abotu it and feel free to give feedback - http://jussiheinonen.blogspot.com/2011/05/neo4j-graph-database-server-image-in.html Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Index Framework - Query Question
Hi, Mattias. I'll definitely give it a try. Ultimately, I'd like to achieve the following scenario (and maybe you can give us some guidance on how to implement it): As a use case: we want to index the node associated with a collaboration entry (that has a timestamp, tag(s), location, textual content, and potentially some numeric values as properties) on a combination of what is currently many different types of indexes in Neo - fulltext, timeline, spatial, and plain (key/value field(s)).We'd like to search/query on any combination of those elements. We've been considering a few options: - use relationships (that wouldn't meet our performance/concurrency requirements) - use the current index framework with separate indexes, query multiple times, and do the set comparison/intersection in our code - bypass Neo's indexing model altogether and go directly to Lucene, though we'd lose the transaction capabilities - hack the Neo code for the various Lucene index implementations and create our own composite index index type that supports fulltext fields, geospatial fields, simple data types (string, number, boolean), timeline, and multi-valued fields I like option #4, but we'd definitely need some help and input to build it in a way that wasn't too fragile from version to version of Neo. I think it is a capability that would be broadly useful for a lot of Neo users, though, and would fit nicely with some of the work Anders is doing in a Neo query language/syntax. I do think the ability to have more control at the field level could lead to some cool capabilities and significant optimizations both in storage and retrieval. I also think you've managed to work around the transactioning issues, so it would be foolish to re-invent the wheel there. The basic idea would be a composite index that perhaps had a new add method which accepted a Field definition instead of just a simple string field name. The Lucene index implementation could leverage this to exploit a lot of the power of Lucene to meet the overall goal of a very flexible indexing/query subsystem for Neo4J. Additionally, the Field definition for full-text fields could have an analyzer associated with it, rather than an Analyzer at the index level only. If you would be up for it, maybe we come over to Sweden for a couple days and hack #4 together, or we could do it virtually? Thoughts? Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Mattias Persson Sent: Thursday, May 12, 2011 3:55 AM To: Neo4j user discussions Subject: Re: [Neo4j] Index Framework - Query Question Could you perhaps create an index with a custom Analyzer which treats titles as fulltext and others as non-analyzed? 2011/5/11 Rick Bullotta rick.bullo...@thingworx.com Is there currently any way to have a composite index consisting of fulltext and non-analyzed (simple fields), and to query them in a single query statement? e.g. : title:Reloaded AND year:1999 In this case, I'd be using a Lucene Analyzer on the title, but indexing the primitive types without using the analyzer/norms/etc...I know how to do it with Lucene directly, but not clear if it can be done w/Neo4J. Thanks! Rick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j EC2 AMI image available
Thanks, Peter. I've found that piece of documentation before but Jussi just outlined an even more straightforward way of securing Neo4j on AWS: http://jussiheinonen.blogspot.com/2011/05/neo4j-graph-database-server-image-in.html?showComment=1305213734091#c177328379806909558 –Daniel On Thu, May 12, 2011 at 11:31, Peter Neubauer peter.neuba...@neotechnology.com wrote: Daniel, I would put some Apache server in front of it and secure it on URL patterns, see http://docs.neo4j.org/chunked/snapshot/operations-security.html for a simplicstic example. Would that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, May 11, 2011 at 10:57 PM, Daniel Gasienica dan...@gasienica.ch wrote: This is great news! What is the simplest way to secure a Neo4j instance on AWS? Daniel On Tue, May 10, 2011 at 06:29, Peter Neubauer peter.neuba...@neotechnology.com wrote: Hi all, Jussi Heinonen from OpenCredo has created a Neo4j Server EC2 image to try out things with. Well done, thanks for the effort Jussi! Read more abotu it and feel free to give feedback - http://jussiheinonen.blogspot.com/2011/05/neo4j-graph-database-server-image-in.html Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j EC2 AMI image available
That's indeed a nice and simple approach. It would be nice if admins could access the web interface from a browser though. Would that be possible using the Amazon security approach? Aseem On Thu, May 12, 2011 at 7:00 PM, Daniel Gasienica dan...@gasienica.chwrote: Thanks, Peter. I've found that piece of documentation before but Jussi just outlined an even more straightforward way of securing Neo4j on AWS: http://jussiheinonen.blogspot.com/2011/05/neo4j-graph-database-server-image-in.html?showComment=1305213734091#c177328379806909558 –Daniel On Thu, May 12, 2011 at 11:31, Peter Neubauer peter.neuba...@neotechnology.com wrote: Daniel, I would put some Apache server in front of it and secure it on URL patterns, see http://docs.neo4j.org/chunked/snapshot/operations-security.html for a simplicstic example. Would that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, May 11, 2011 at 10:57 PM, Daniel Gasienica dan...@gasienica.ch wrote: This is great news! What is the simplest way to secure a Neo4j instance on AWS? Daniel On Tue, May 10, 2011 at 06:29, Peter Neubauer peter.neuba...@neotechnology.com wrote: Hi all, Jussi Heinonen from OpenCredo has created a Neo4j Server EC2 image to try out things with. Well done, thanks for the effort Jussi! Read more abotu it and feel free to give feedback - http://jussiheinonen.blogspot.com/2011/05/neo4j-graph-database-server-image-in.html Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
