[Neo] Neo4j RESTful traversal API
HI there. Loving Neo4j, but been putting off using because was no * straightforward* way to use from PHP. I even started writing my own RESTful API, but never found the time to finish it off! However, I am excited that there is now a (semi?) official one in the works, even if it is early days. I have started a PHP library for accessing neo4j via the REST API... I will share when its ready. I can see how most of the functionality works from the Java functional tests and the application.wadl index. However, there is no mention of any way to access neo4j's traversal features. I was wondering how this is planned to work? In my API I had something like: node/1/traverse?relationship_type=KNOWS Which would traverse from node 1 along relationship type 'KNOWS' etc... However, it started to get complex very quickly as you need separate return and stop evaluators. I suppose, the most beautiful method would to to traverse via Gremlin: /node/1/outE/i...@age27] (obviously URI encoded) That would be dead sexy. However, I know the gremlin guys seem a little reluctant to offer a RESTful API (they seem to be stuck in their Java ways)... Whats the plan? Cheers Alastair ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Neo4j RESTful traversal API
Alastair, that sounds very cool. I'm very interested in getting Neo4j to workwith PHP as well. And like you I've had to put development onto the back burner for a while. I'd like to help somehow but with your specific question about traversal, I'm not sure I can be of much help. I don't know if you've considered the other alternatives. I've added quite a bit of info to the Neo-Wiki about using PHP. Maybe you have seen it already. I'd give you the direct link but the neo4j website is currently down :( Mainly I did some testing to show that it is feasible to access ne4j from PHP using javabridge. I thionk Peter has developed this further (there is a post somewhere in the mailing list). Javabride is one alternative. there are two others. One the RESTful API and another would be direct integration into the PHP language similar to the way MongoDB is integrated. I wanted to experiment with both of theses but haven't gotten around to it. I'm glad your'e working in this direction also. Andy Potter On Monday 22 March 2010 10:12:38 Alastair James wrote: HI there. Loving Neo4j, but been putting off using because was no * straightforward* way to use from PHP. I even started writing my own RESTful API, but never found the time to finish it off! However, I am excited that there is now a (semi?) official one in the works, even if it is early days. I have started a PHP library for accessing neo4j via the REST API... I will share when its ready. I can see how most of the functionality works from the Java functional tests and the application.wadl index. However, there is no mention of any way to access neo4j's traversal features. I was wondering how this is planned to work? In my API I had something like: node/1/traverse?relationship_type=KNOWS Which would traverse from node 1 along relationship type 'KNOWS' etc... However, it started to get complex very quickly as you need separate return and stop evaluators. I suppose, the most beautiful method would to to traverse via Gremlin: /node/1/outE/i...@age27] (obviously URI encoded) That would be dead sexy. However, I know the gremlin guys seem a little reluctant to offer a RESTful API (they seem to be stuck in their Java ways)... Whats the plan? Cheers Alastair ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Neo4j RESTful traversal API
Guys, we are working on getting the site up again - our provider changed servers on us! Sorry for the inconvenience! Regarding REST, we are back on it starting this week, shortly Mattias will outline our way forward and ask for input from you all, so tune in and contribute! Good to see some early testers emerging here :) Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.tinkerpop.com - Processing for Internet-scale graphs. http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party. On Mon, Mar 22, 2010 at 10:36 AM, andrew.pot...@gmx.net wrote: Alastair, that sounds very cool. I'm very interested in getting Neo4j to workwith PHP as well. And like you I've had to put development onto the back burner for a while. I'd like to help somehow but with your specific question about traversal, I'm not sure I can be of much help. I don't know if you've considered the other alternatives. I've added quite a bit of info to the Neo-Wiki about using PHP. Maybe you have seen it already. I'd give you the direct link but the neo4j website is currently down :( Mainly I did some testing to show that it is feasible to access ne4j from PHP using javabridge. I thionk Peter has developed this further (there is a post somewhere in the mailing list). Javabride is one alternative. there are two others. One the RESTful API and another would be direct integration into the PHP language similar to the way MongoDB is integrated. I wanted to experiment with both of theses but haven't gotten around to it. I'm glad your'e working in this direction also. Andy Potter On Monday 22 March 2010 10:12:38 Alastair James wrote: HI there. Loving Neo4j, but been putting off using because was no * straightforward* way to use from PHP. I even started writing my own RESTful API, but never found the time to finish it off! However, I am excited that there is now a (semi?) official one in the works, even if it is early days. I have started a PHP library for accessing neo4j via the REST API... I will share when its ready. I can see how most of the functionality works from the Java functional tests and the application.wadl index. However, there is no mention of any way to access neo4j's traversal features. I was wondering how this is planned to work? In my API I had something like: node/1/traverse?relationship_type=KNOWS Which would traverse from node 1 along relationship type 'KNOWS' etc... However, it started to get complex very quickly as you need separate return and stop evaluators. I suppose, the most beautiful method would to to traverse via Gremlin: /node/1/outE/i...@age27] (obviously URI encoded) That would be dead sexy. However, I know the gremlin guys seem a little reluctant to offer a RESTful API (they seem to be stuck in their Java ways)... Whats the plan? Cheers Alastair ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Unable to memory map
Hi, We have seen this message before emitted as a warning from Neo4j. Are you seing this as a warning as well, or are you getting an exception thrown to your application code? It's hard to deal with these errors since nio only throws IOException, and not any more semantic information than that, I believe we deal with all cases by issuing a warning and then falling back to another method of performing the same operation, but if you are getting exceptions we need to resolve it. If you are indeed getting exceptions, some code that triggers it would be very helpful. Cheers, Tobias On Wed, Mar 17, 2010 at 1:47 PM, Marc Preddie mpred...@gmail.com wrote: Hi, I've look at the mailing list and found 1 similar situation, but no real solution. So I was hoping someone could shed some light on this. I seem to have an issue with neo4j being able to use memory mapped files. I've run my service on Win XP 64bit, Mac OSX Snow Leopard 10.6.2 and Centos 5.x 64bit and always get the same error when launching. I'm using APOC 1.0 and have a DB of approx 600M. In my neo config I allocate about 5M more for each type of file than the actual file size (I've tried multiple different settings). On each machine I also leave at least 1.5G for the OS and have at least 2.5G heap for the Java process. I'm also using the classes EmbeddedReadOnlyGraphDatabase and LuceneReadOnlyIndexService to access and browse DB. Neo config neostore.nodestore.db.mapped_memory=10M neostore.relationshipstore.db.mapped_memory=110M neostore.propertystore.db.mapped_memory=85M neostore.propertystore.db.index.mapped_memory=10M neostore.propertystore.db.index.keys.mapped_memory=10M neostore.propertystore.db.strings.mapped_memory=320M neostore.propertystore.db.arrays.mapped_memory=10M Here is the error org.neo4j.kernel.impl.nioneo.store.MappedMemException: Unable to map pos=3005872 recordSize=33 totalSize=1153416 at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.init(MappedPersistenceWindow.java:59) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.allocateNewWindow(PersistenceWindowPool.java:530) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.refreshBricks(PersistenceWindowPool.java:430) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:122) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:459) at org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:248) at org.neo4j.kernel.impl.nioneo.xa.NeoReadTransaction.getMoreRelationships(NeoReadTransaction.java:103) at org.neo4j.kernel.impl.nioneo.xa.NioNeoDbPersistenceSource$ReadOnlyResourceConnection.getMoreRelationships(NioNeoDbPersistenceSource.java:275) at org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:93) at org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:585) at org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:332) at org.neo4j.kernel.impl.core.NodeImpl.ensureFullRelationships(NodeImpl.java:320) at org.neo4j.kernel.impl.core.NodeImpl.getAllRelationshipsOfType(NodeImpl.java:129) at org.neo4j.kernel.impl.core.NodeImpl.getSingleRelationship(NodeImpl.java:179) at org.neo4j.kernel.impl.core.NodeProxy.getSingleRelationship(NodeProxy.java:98) Caused by: java.io.IOException: Access is denied at sun.nio.ch.FileChannelImpl.truncate0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:728) at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.init(MappedPersistenceWindow.java:53) ... 78 more Regards Marc ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Problems installing neo4j.py
No, but I haven't tried especially hard either. Quick check now, the contents of jython2.5/Lib/site-packages/classes is: geronimo-jta_1.1_spec.jar junit.jar lucene_core.jar neo4j-commons.jar neo4j-index.jar neo4j-kernel.jar neo4j-remote-graphdb.jar import works fine, its not until i actually do something that i get errors, so it seems to find the installed wrappers, jython site-packages is added to my pythonpath. I haven't installed any pure java version of Neo4j. I do have a jvm. I am however using python 2.6 instead of 2.5, Due to the lack of packaged python2.5 for opensuse. The working instance i have of neo4j.py is using Jpype/Cpython method on an Ubuntu based machine, haven't tried the other way around (jython on ubuntu, Jpype on suse). btw I just realized that I'm probably switching back and forth using my work mail and private mail in this and other discussions, hope that doesn't cause problems. On Mon, Mar 22, 2010 at 2:47 PM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: Did you make any progress on this? I haven't been able to reproduce the problem. On Tue, Mar 16, 2010 at 11:01 PM, Erik Ask ask.e...@gmail.com wrote: I'm away from the computer in question now, but I did have that directory, and it did contain files. I've also tried installing to a directory in my home directory, i.e. no sudo involved, and everything seemed to install just fine. But still no backend when actually running. Is there a list of the files needed to be in that directory On Tue, Mar 16, 2010 at 9:43 PM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: Your previous error indicated that the dependencies directory could not be created during the setup, this is where all the java libraries gets downloaded to. Could you make sure that your Neo4j python install directory (seems to be /usr/local/jython/Lib/site-packages/neo4j in your case) contains a directory called classes, and that it contains the necessary jar-files? Since this directory contains jar-files and not class-files, it will not work as CLASSPATH, so your CLASSPATH setting does not add anything. The Neo4j python bindings will automatically discover the jar files in this directory and add them to sys.path in Jython, so you don't have to manage the CLASSPATH on your own. -Tobias On Tue, Mar 16, 2010 at 4:50 PM, Erik Ask erik...@maths.lth.se wrote: Tobias Ivarsson wrote: It looks very strange. Which version of Jython are you using? On Tue, Mar 16, 2010 at 4:12 PM, Erik Ask ask.e...@gmail.com wrote: Hello, I'm having trouble installing neo4j.py sudo jython setup.py install root's password: warning: /root/.jython is a directory, not a file *sys-package-mgr*: processing new jar, '/usr/local/jython/jython.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/resources.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/rt.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/jsse.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/jce.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/charsets.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/ext/localedata.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/ext/sunpkcs11.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/ext/dnsns.jar' *sys-package-mgr*: processing new jar, '/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/ext/sunjce_provider.jar' Traceback (most recent call last): File setup.py, line 146, in module main() File setup.py, line 104, in main for dependency,include in pom.download_dependencies(dependencies_path): File /usr/matematik/erikask/python/neo4j-python/pom.py, line 67, in download_dependencies os.mkdir(destination) File /usr/local/jython/Lib/os.py, line 309, in mkdir raise OSError(err, msg, path) OSError: [Errno 0] couldn't make directory: 'dependencies' Does this look familiar to anyone?? ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user 2.5.1 Just installed it from their homepage I was actually able to finish the installation by rerunning it without sudo, checking what directories weren't created (/usr/local/jython/Lib/site-packages/neo4j in this case) crating them, sudo chmod a+w. I'm guessing it relates to the system setup I have with some discs being nfs mounted with other root accounts (although none of those were involved in this so I'm not sure) Now I'm getting another error instead (while running a script): Traceback (most recent call last):
Re: [Neo] Neo4j RESTful traversal API
Btw, the wiki etc is up again - sorry for the inconvenience! Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.tinkerpop.com - Processing for Internet-scale graphs. http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party. On Mon, Mar 22, 2010 at 10:59 AM, Peter Neubauer neubauer.pe...@gmail.com wrote: Guys, we are working on getting the site up again - our provider changed servers on us! Sorry for the inconvenience! Regarding REST, we are back on it starting this week, shortly Mattias will outline our way forward and ask for input from you all, so tune in and contribute! Good to see some early testers emerging here :) Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.tinkerpop.com - Processing for Internet-scale graphs. http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party. On Mon, Mar 22, 2010 at 10:36 AM, andrew.pot...@gmx.net wrote: Alastair, that sounds very cool. I'm very interested in getting Neo4j to workwith PHP as well. And like you I've had to put development onto the back burner for a while. I'd like to help somehow but with your specific question about traversal, I'm not sure I can be of much help. I don't know if you've considered the other alternatives. I've added quite a bit of info to the Neo-Wiki about using PHP. Maybe you have seen it already. I'd give you the direct link but the neo4j website is currently down :( Mainly I did some testing to show that it is feasible to access ne4j from PHP using javabridge. I thionk Peter has developed this further (there is a post somewhere in the mailing list). Javabride is one alternative. there are two others. One the RESTful API and another would be direct integration into the PHP language similar to the way MongoDB is integrated. I wanted to experiment with both of theses but haven't gotten around to it. I'm glad your'e working in this direction also. Andy Potter On Monday 22 March 2010 10:12:38 Alastair James wrote: HI there. Loving Neo4j, but been putting off using because was no * straightforward* way to use from PHP. I even started writing my own RESTful API, but never found the time to finish it off! However, I am excited that there is now a (semi?) official one in the works, even if it is early days. I have started a PHP library for accessing neo4j via the REST API... I will share when its ready. I can see how most of the functionality works from the Java functional tests and the application.wadl index. However, there is no mention of any way to access neo4j's traversal features. I was wondering how this is planned to work? In my API I had something like: node/1/traverse?relationship_type=KNOWS Which would traverse from node 1 along relationship type 'KNOWS' etc... However, it started to get complex very quickly as you need separate return and stop evaluators. I suppose, the most beautiful method would to to traverse via Gremlin: /node/1/outE/i...@age27] (obviously URI encoded) That would be dead sexy. However, I know the gremlin guys seem a little reluctant to offer a RESTful API (they seem to be stuck in their Java ways)... Whats the plan? Cheers Alastair ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Neo4j RESTful traversal API
Cheers Peter. Al On 22 March 2010 15:26, Peter Neubauer neubauer.pe...@gmail.com wrote: Btw, the wiki etc is up again - sorry for the inconvenience! Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.tinkerpop.com - Processing for Internet-scale graphs. http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party. On Mon, Mar 22, 2010 at 10:59 AM, Peter Neubauer neubauer.pe...@gmail.com wrote: Guys, we are working on getting the site up again - our provider changed servers on us! Sorry for the inconvenience! Regarding REST, we are back on it starting this week, shortly Mattias will outline our way forward and ask for input from you all, so tune in and contribute! Good to see some early testers emerging here :) Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.tinkerpop.com - Processing for Internet-scale graphs. http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party. On Mon, Mar 22, 2010 at 10:36 AM, andrew.pot...@gmx.net wrote: Alastair, that sounds very cool. I'm very interested in getting Neo4j to workwith PHP as well. And like you I've had to put development onto the back burner for a while. I'd like to help somehow but with your specific question about traversal, I'm not sure I can be of much help. I don't know if you've considered the other alternatives. I've added quite a bit of info to the Neo-Wiki about using PHP. Maybe you have seen it already. I'd give you the direct link but the neo4j website is currently down :( Mainly I did some testing to show that it is feasible to access ne4j from PHP using javabridge. I thionk Peter has developed this further (there is a post somewhere in the mailing list). Javabride is one alternative. there are two others. One the RESTful API and another would be direct integration into the PHP language similar to the way MongoDB is integrated. I wanted to experiment with both of theses but haven't gotten around to it. I'm glad your'e working in this direction also. Andy Potter On Monday 22 March 2010 10:12:38 Alastair James wrote: HI there. Loving Neo4j, but been putting off using because was no * straightforward* way to use from PHP. I even started writing my own RESTful API, but never found the time to finish it off! However, I am excited that there is now a (semi?) official one in the works, even if it is early days. I have started a PHP library for accessing neo4j via the REST API... I will share when its ready. I can see how most of the functionality works from the Java functional tests and the application.wadl index. However, there is no mention of any way to access neo4j's traversal features. I was wondering how this is planned to work? In my API I had something like: node/1/traverse?relationship_type=KNOWS Which would traverse from node 1 along relationship type 'KNOWS' etc... However, it started to get complex very quickly as you need separate return and stop evaluators. I suppose, the most beautiful method would to to traverse via Gremlin: /node/1/outE/i...@age27] (obviously URI encoded) That would be dead sexy. However, I know the gremlin guys seem a little reluctant to offer a RESTful API (they seem to be stuck in their Java ways)... Whats the plan? Cheers Alastair ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Dr Alastair James CTO James Publishing Ltd. http://www.linkedin.com/pub/3/914/163 www.worldreviewer.com WINNER Travolution Awards Best Travel Information Website 2009 WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008 WINNER Travolution Awards Best New Online Travel Company 2008 WINNER Travel Weekly Magellan Award 2008 WINNER Yahoo! Finds of the Year 2007 Noli nothis permittere te terere! ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Indexing on Doubles in Neo4j
Hello all, I'm working on an application where it would be nice to perform lookups on a graph database based on real-valued properties. For example, if I have a social network, and have assigned real-valued centrality measures to each node, I'd like to be able to choose all vertices whose centrality measure is greater than some threshold. I see that the Timeline index service offers this for integer-valued properties. Is there something similar (or in the pipeline) for doing the same with real-valued properties? Is there an easy way to adapt one of the current indexing utilities to do this (besides multiplying by 10^n for sufficiently large n and then rounding)? Thanks, Alex D'Amour Harvard Institute for Quantitative Social Science ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Indexing on Doubles in Neo4j
Maybe this? http://components.neo4j.org/neo4j-graph-algo/apidocs/org/neo4j/graphalgo/cen trality/package-summary.html -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Alex D'Amour Sent: Monday, March 22, 2010 5:06 PM To: Neo user discussions Subject: [Neo] Indexing on Doubles in Neo4j Hello all, I'm working on an application where it would be nice to perform lookups on a graph database based on real-valued properties. For example, if I have a social network, and have assigned real-valued centrality measures to each node, I'd like to be able to choose all vertices whose centrality measure is greater than some threshold. I see that the Timeline index service offers this for integer-valued properties. Is there something similar (or in the pipeline) for doing the same with real-valued properties? Is there an easy way to adapt one of the current indexing utilities to do this (besides multiplying by 10^n for sufficiently large n and then rounding)? Thanks, Alex D'Amour Harvard Institute for Quantitative Social Science ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Unable to memory map
Hi, Previous to the 1.0 release, there was just a warning message, now with the 1.0 release there is a warning message, along with an exception describing the warning. Everything still works, but we would really like and need the performance boost that memory mapping may provide. I'll try to get a small sample (code and db) to you in the next day or so. Regards, Marc On Mon, Mar 22, 2010 at 10:24 AM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: Hi, We have seen this message before emitted as a warning from Neo4j. Are you seing this as a warning as well, or are you getting an exception thrown to your application code? It's hard to deal with these errors since nio only throws IOException, and not any more semantic information than that, I believe we deal with all cases by issuing a warning and then falling back to another method of performing the same operation, but if you are getting exceptions we need to resolve it. If you are indeed getting exceptions, some code that triggers it would be very helpful. Cheers, Tobias On Wed, Mar 17, 2010 at 1:47 PM, Marc Preddie mpred...@gmail.com wrote: Hi, I've look at the mailing list and found 1 similar situation, but no real solution. So I was hoping someone could shed some light on this. I seem to have an issue with neo4j being able to use memory mapped files. I've run my service on Win XP 64bit, Mac OSX Snow Leopard 10.6.2 and Centos 5.x 64bit and always get the same error when launching. I'm using APOC 1.0 and have a DB of approx 600M. In my neo config I allocate about 5M more for each type of file than the actual file size (I've tried multiple different settings). On each machine I also leave at least 1.5G for the OS and have at least 2.5G heap for the Java process. I'm also using the classes EmbeddedReadOnlyGraphDatabase and LuceneReadOnlyIndexService to access and browse DB. Neo config neostore.nodestore.db.mapped_memory=10M neostore.relationshipstore.db.mapped_memory=110M neostore.propertystore.db.mapped_memory=85M neostore.propertystore.db.index.mapped_memory=10M neostore.propertystore.db.index.keys.mapped_memory=10M neostore.propertystore.db.strings.mapped_memory=320M neostore.propertystore.db.arrays.mapped_memory=10M Here is the error org.neo4j.kernel.impl.nioneo.store.MappedMemException: Unable to map pos=3005872 recordSize=33 totalSize=1153416 at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.init(MappedPersistenceWindow.java:59) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.allocateNewWindow(PersistenceWindowPool.java:530) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.refreshBricks(PersistenceWindowPool.java:430) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:122) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:459) at org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:248) at org.neo4j.kernel.impl.nioneo.xa.NeoReadTransaction.getMoreRelationships(NeoReadTransaction.java:103) at org.neo4j.kernel.impl.nioneo.xa.NioNeoDbPersistenceSource$ReadOnlyResourceConnection.getMoreRelationships(NioNeoDbPersistenceSource.java:275) at org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:93) at org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:585) at org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:332) at org.neo4j.kernel.impl.core.NodeImpl.ensureFullRelationships(NodeImpl.java:320) at org.neo4j.kernel.impl.core.NodeImpl.getAllRelationshipsOfType(NodeImpl.java:129) at org.neo4j.kernel.impl.core.NodeImpl.getSingleRelationship(NodeImpl.java:179) at org.neo4j.kernel.impl.core.NodeProxy.getSingleRelationship(NodeProxy.java:98) Caused by: java.io.IOException: Access is denied at sun.nio.ch.FileChannelImpl.truncate0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:728) at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.init(MappedPersistenceWindow.java:53) ... 78 more Regards Marc ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Something fun to try in neo4j : collatz conjecture
On Fri, Mar 19, 2010 at 10:20 AM, Laurent Laborde kerdez...@gmail.com wrote: thank you peter. Following the various link, i found http://www.cytoscape.org/ Look very promising !! i'll try this weekend :) 3 days later, i'm still populating neo4j like a crazy, trying to compute the collatz conjecture up to 100 millions. i'm at 63.5 millions, but it's now much much slower than at the beginning. probably because of the insane numbers of index.getSingleNode on lucene on a growing index. i'm expecting something like a billion of node when it will reach 100 millions. the index is 6GB and the base is 9GB. The cpu usage is now ~10% instead of 25%, i have a grand total of 9+6 GB of data in the neo4j drectory with only 8GB on ram on my windows 7 i'm now (random) IO bound. My SATA velociraptor 10krpm is fighting at 1MB/s (not so bad; considering it's highly mixed R/W and purely random IO) time to buy a 32GB SSD :) -- Keru -- Laurent ker2x Laborde Sysadmin DBA at http://www.over-blog.com/ ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] displaying very large graph
A couple I've come across recently that I'd never seen mentioned before: http://gephi.org/ http://www.ondex.org/ Jonny On Thu, Mar 18, 2010 at 4:13 PM, Laurent Laborde kerdez...@gmail.comwrote: i found a page that link a lot of graph visualisation tools : http://www.mkbergman.com/414/large-scale-rdf-graph-visualization-tools/ i'll dig it this weekend :) -- Ker2x On Fri, Mar 19, 2010 at 12:07 AM, Laurent Laborde kerdez...@gmail.com wrote: Friendly greetings ! i wish to display very large graph. ( hundreds of thousands of node, or many millions of node if possible). - graphviz probably won't do it. - my processing app is 100% sure to explode i found that, but didn't tried it yet : http://bioinformatics.icmb.utexas.edu/lgl/ Large Graph Layout the gallery is impressive, with huge graph and millions of edges. But it seems that i have to provide the coordinate of the nodes/edges, which is exactly my problem. i don't know the math/algorithm to compute the best way to display graphs. my processing code use simple physics rule (spring and node repulsion) but it's very very very slow :( Any tought ? Thank you :) -- Laurent ker2x Laborde Sysadmin DBA at http://www.over-blog.com/ -- Laurent ker2x Laborde Sysadmin DBA at http://www.over-blog.com/ ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
