[Neo4j] All simple paths algorithm
Dear Sir / Madam, Let me introduce myself. I'm an IT engineer and currently working for an university project related to development of knowledge discovery and semantic associations evaluation system. Due to that I'm interested in principle of your algorithm's design (eo4j::Algo.shortest_paths - finds all shortest paths between two nodes.) Please let me know whether it is possible to get from you this algorithm's description (eventually pseudo-code). This knowledge and information will only be used for scientific purposes. Thank you in advance for prompt answer from your side. Best regards, Piotr Stapor ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] setProperty Exception
Hi John,
maybe you are trying to add a property that has a value of String[]
with at least one of the array members being null?. That would cause
the first exception. The second exception is most probably because you
try to success() and finish() your transaction even after the
exception.
For the first issue, always make sure that you are not adding null
values, not even as members of an array.
For the second issue, make sure that you have a try/catch block around
your transactional code that does tx.success() only when everything is
in place, with the catch block doing the tx.failure(). At the finally
block, just do tx.finish() - that should do it.
In code:
Transaction tx = db.beginTx();
try {
// do tx stuff here
tx.success();
}
catch (Exception e) {
tx.failure();
}
finally {
tx.finish();
}
Hope the above help. If your code does not conform to my assumptions
please share some more details.
cheers,
CG
On Thu, Aug 25, 2011 at 5:43 AM, John Howard johnyho...@gmail.com wrote:
I am using embedded 1.4.1
I get these 2 strange exceptions almost simultaneously when try to create a
new property.
java.lang.NullPointerException
at
org.neo4j.kernel.impl.nioneo.store.DynamicArrayStore.allocateFromString(DynamicArrayStore.java:142)
at
org.neo4j.kernel.impl.nioneo.store.DynamicArrayStore.allocateRecords(DynamicArrayStore.java:369)
at
org.neo4j.kernel.impl.nioneo.store.PropertyStore.allocateArrayRecords(PropertyStore.java:466)
at
org.neo4j.kernel.impl.nioneo.store.PropertyStore.encodeValue(PropertyStore.java:535)
at
org.neo4j.kernel.impl.nioneo.xa.WriteTransaction.nodeAddProperty(WriteTransaction.java:1314)
at
org.neo4j.kernel.impl.nioneo.xa.WriteTransaction.nodeAddProperty(WriteTransaction.java:1741)
at
org.neo4j.kernel.impl.persistence.PersistenceManager.nodeAddProperty(PersistenceManager.java:140)
at
org.neo4j.kernel.impl.core.NodeManager.nodeAddProperty(NodeManager.java:849)
at org.neo4j.kernel.impl.core.NodeImpl.addProperty(NodeImpl.java:101)
at org.neo4j.kernel.impl.core.Primitive.setProperty(Primitive.java:411)
at org.neo4j.kernel.impl.core.NodeProxy.setProperty(NodeProxy.java:120)
at graph.createProperty(graph.java:232)
and then
org.neo4j.graphdb.TransactionFailureException: Unable to commit transaction
at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:98)
Caused by: javax.transaction.RollbackException: Failed to commit,
transaction rolledback
at
org.neo4j.kernel.impl.transaction.TxManager.rollbackCommit(TxManager.java:811)
at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:645)
at
org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:109)
at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85)
... 40 more
TIA
John
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] All simple paths algorithm
Hi Piotr, Neo4j is an open source product. This means you're quite welcome to read the source of the algorithm and see how it works directly. The implementation for all of our shortest path algorithms can be found here: https://github.com/neo4j/community/tree/master/graph-algo/src/main/java/org/neo4j/graphalgo/impl/shortestpath Enjoy! Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] IMDB Error 503 in setup.html
thank you! it works :) franz -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/IMDB-Error-503-in-setup-html-tp3281457p3283655.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] IMDB Error 503 in setup.html
Glad to help! /peter On Thu, Aug 25, 2011 at 1:38 PM, elfranco f.ruff...@gmail.com wrote: thank you! it works :) franz -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/IMDB-Error-503-in-setup-html-tp3281457p3283655.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] All simple paths algorithm
2011/8/25 Jim Webber j...@neotechnology.com Hi Piotr, Neo4j is an open source product. This means you're quite welcome to read the source of the algorithm and see how it works directly. The implementation for all of our shortest path algorithms can be found here: https://github.com/neo4j/community/tree/master/graph-algo/src/main/java/org/neo4j/graphalgo/impl/shortestpath Our shortest path algorithm currently used by default is really here: https://github.com/neo4j/community/blob/master/graph-algo/src/main/java/org/neo4j/graphalgo/impl/path/ShortestPath.java Enjoy! Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4jPHP batch insert benchmarks
Josh, it might be that the parsing of the JSON load is taking up increasingly much time when you get big batches. At least that is my suspicion. Also, that might be the reason for the heap problems - basically the String parsing is taking over :/ Do you have any means of verifying that? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 25, 2011 at 6:15 AM, jadell josh.ad...@gmail.com wrote: Hey all, I've been working on adding batch support to http://github.com/jadell/Neo4jPHP Neo4jPHP . Here are the results of my latest benchmarks. First column is the number of nodes being inserted, second column is the average in seconds over 5 runs to insert that many nodes in a single batch, third column is the average in seconds over 5 runs to insert that many nodes one at a time: #nodes batchsingle 100 0 100 0.2 0.4 250 0 1 500 0.8 2 10001.4 4 25006 10.6 500023.2 21.2 1 91.6 40.4 It seems like batches win out until right around 5000 nodes at a time. I've profiled my code, and it seems like the time spent in PHP is roughly equivalent for batch vs. single. All the time difference is spent in a curl_exec call, talking to or waiting to hear back from the server. I tried going up to 10 nodes. Single insert handled this just fine, but the server kept returning a 500 Java Heap space error on the batch, even with 512M max heap. Benchmark script can be found here: http://gist.github.com/1169394 http://gist.github.com/1169394 Benchmarks were run on an 4 x 2.3GHz core Intel i7, 4G RAM, running Ubuntu 10.10. Neo4j server was run with out-of-the-box settings in a VM runnning Ubuntu 10.10 with 1 dedicated core and 1G RAM. I hope this is of interest to anyone. I'd love to get some feedback from anyone using Neo4j from PHP, with Neo4jPHP or any other library. -- Josh Adell -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4jPHP-batch-insert-benchmarks-tp3282984p3282984.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Performance Benchmarks Available
We are looking for neo4j performance benchmarks vs rdbms and nosql data stores; are there any available? Thanks raffi -- This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications. == ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Performance Benchmarks Available
Hi! Here's articles which make some kind of comparison: http://www.delicious.com/neo4j/articles+comparison If anyone else know about other articles on this topic, please tell me! /anders On 08/25/2011 03:15 PM, Basmajian, Raffi wrote: We are looking for neo4j performance benchmarks vs rdbms and nosql data stores; are there any available? Thanks raffi -- This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications. == ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4jPHP batch insert benchmarks
Hey Peter, I don't have any way of verifying on the server side, other than measuring the time it takes for curl_exec to return a response. On the client side I can see that PHP's json_encode/json_decode functions are taking less than .5% of the total run time, even with a batch size of 1. During one of my 10 node attempts, I printed out the server response of the 500 Heap space error. It seemed like the last method in the stack trace was dealing with a Deserializer class or method. I will try again and capture the stack trace output to post here. Thanks, -- Josh Adell -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4jPHP-batch-insert-benchmarks-tp3282984p3283926.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Performance Benchmarks Available
Hi, I did not find any useful performance benchmarks at that link, just articles about how and why graph-style data bases perform better than SQL-based data stores. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Anders Nawroth Sent: Thursday, August 25, 2011 9:29 AM To: Neo4j user discussions Subject: Re: [Neo4j] Performance Benchmarks Available Hi! Here's articles which make some kind of comparison: http://www.delicious.com/neo4j/articles+comparison If anyone else know about other articles on this topic, please tell me! /anders On 08/25/2011 03:15 PM, Basmajian, Raffi wrote: We are looking for neo4j performance benchmarks vs rdbms and nosql data stores; are there any available? Thanks raffi -- This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications. == ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications. == ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Test and try/catch block
If you're using the standard try/catch/finally idiom in Neo4j then consider using Nat Pryce/Steve Freeman's transactor pattern. We have an example of this in the neo4j server code here: https://github.com/neo4j/community/blob/master/server/src/functionaltest/java/org/neo4j/server/helpers/Transactor.java It's not always appropriate, but it the general case it can keep your code a bit terser and more intent-ful. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Performance Benchmarks Available
Hi Raffi, There are no standard TPC-like benchmarks for Neo4j. However, for even modest data sets, Neo4j can be orders of magnitude faster than some SQL-oriented databases primarily because it avoids join pain (no sets) for connected data. There are a few slides in the tutorial deck about this here (warning, large file): https://github.com/jimwebber/neo4j-tutorial/blob/master/presentation/A%20Programmatic%20Introduction%20to%20Neo4j.pptx And there's a video of me talking about this stuff here (see around 25:40): http://skillsmatter.com/podcast/java-jee/cypher HTH, Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4jPHP batch insert benchmarks
The heap space stuff would make sense I think, because we currently deserialize and serialize in-place, keeping the whole thing in memory. Would be interesting to see if we could implement a setup that can stream the deserialization/serialization, getting rid of the memory overhead.. You said you are using out-of-the-box settings for the server, I don't remember off the top of my head what the default heap size is, but you might want to try giving it more RAM. I'm gonna guess that's where performance dies.. I'll have to look at what is proper HTTP behavior, but there should be a way we could start streaming back the response as it is being calculated, as long as we can come up with a good way of aborting if something fails.. Doing that would mean we don't have to keep a hundred thousand requests and responses in memory, which would completely change the performance situation. Big thanks for taking the time to put this together! /jake On Thu, Aug 25, 2011 at 3:52 PM, jadell josh.ad...@gmail.com wrote: Hey Peter, I don't have any way of verifying on the server side, other than measuring the time it takes for curl_exec to return a response. On the client side I can see that PHP's json_encode/json_decode functions are taking less than .5% of the total run time, even with a batch size of 1. During one of my 10 node attempts, I printed out the server response of the 500 Heap space error. It seemed like the last method in the stack trace was dealing with a Deserializer class or method. I will try again and capture the stack trace output to post here. Thanks, -- Josh Adell -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4jPHP-batch-insert-benchmarks-tp3282984p3283926.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4jPHP batch insert benchmarks
Hey Josh, You can validate what Peter's suggesting by setting a small heap when you run the server. If you edit conf/neo4j-wrapper.conf you can override the property for heap size with something like this: wrapper.java.maxmemory=1 Then you should (in theory) be able to see the batch operation fail much earlier if it is the JSON components barfing. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Executing arbitrary code through REST (was: Specifying best first order of traverse over REST API)
Guys, with the custom sorting in Lucene and this thread coming up all the time, I took the time to document the execution of arbitrary Groovy and thus, Java calls through REST. In the example below, there are calls to Neo4j APIs, Gremlin stuff and custom sorting using Lucene classes, and return of a Neo4j search hit object. You can do all this in a Neo4j Server plugin, but if you need to, this is an example on how to do it with only REST. http://docs.neo4j.org/chunked/snapshot/gremlin-plugin.html#rest-api-send-an-arbitrary-groovy-script---lucene-sorting Hope that helps for future reference! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 25, 2011 at 1:00 AM, Matt Luongo m...@scholr.ly wrote: +1, we could really use that. Client-side sorting sucks. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Aug 24, 2011 at 4:43 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I've just spent a bunch of time reading into how one can control the ordering of a traverse beyond simple breadth first or depth first. More precisely, even when breadth first, how one can control *which* neighbors are traversed first. (It matters less in which order they're traversed vs. which order they're returned if you're returning all results, since you can just sort on the client. But it matters a lot if you want to use the paged traverser, since you're then only returning the first results.) I've learned that this is doable from Java by writing your own BranchSelector implementation: http://components.neo4j.org/neo4j/1.4.1/apidocs/ I've found the built-in implementations, e.g. the pre-order breadth-first and depth-first: https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderBreadthFirstSelector.java https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderDepthFirstSelector.java To achieve a custom best first, Igor Dovgiy for example shared that he modeled his implementation of the breadth-first selector, except just using a PriorityQueue instead of a regular Queue. My question is: is there any way to specify this sort of thing over the REST API instead of having to write a plugin? If not, does that sound like a reasonable feature request? I really just want something simple: nodes ordered by some timestamp property. It's killing us that we can't do this today. We might just have to look into writing this as a plugin... Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Specifying best first order of traverse over REST API
See the other mail for a pointer :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 25, 2011 at 1:00 AM, Matt Luongo m...@scholr.ly wrote: +1, we could really use that. Client-side sorting sucks. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Aug 24, 2011 at 4:43 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I've just spent a bunch of time reading into how one can control the ordering of a traverse beyond simple breadth first or depth first. More precisely, even when breadth first, how one can control *which* neighbors are traversed first. (It matters less in which order they're traversed vs. which order they're returned if you're returning all results, since you can just sort on the client. But it matters a lot if you want to use the paged traverser, since you're then only returning the first results.) I've learned that this is doable from Java by writing your own BranchSelector implementation: http://components.neo4j.org/neo4j/1.4.1/apidocs/ I've found the built-in implementations, e.g. the pre-order breadth-first and depth-first: https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderBreadthFirstSelector.java https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderDepthFirstSelector.java To achieve a custom best first, Igor Dovgiy for example shared that he modeled his implementation of the breadth-first selector, except just using a PriorityQueue instead of a regular Queue. My question is: is there any way to specify this sort of thing over the REST API instead of having to write a plugin? If not, does that sound like a reasonable feature request? I really just want something simple: nodes ordered by some timestamp property. It's killing us that we can't do this today. We might just have to look into writing this as a plugin... Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4jPHP batch insert benchmarks
Jim, When I was running into the issue, I set the maxmemory=256 and can confirm that it took much longer to fail, but it did fail in the same way. I didn't think of setting it smaller than the default, but I suspect you are correct. I'll try it that way when I attempt to generate the stack trace later so that I don't have to wait several minutes for it to fail. -- Josh -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4jPHP-batch-insert-benchmarks-tp3282984p3284382.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Specifying best first order of traverse over REST API
Thanks Peter, we'll look into Gremlin. =) But I'll push back a bit and say it would still ask for a feature like this to be a first-class feature of the REST API. If my app is e.g. in Python, it's not super API-friendly for me to have to write Java-ish code to achieve what I want. I'd get it if what I was asking for was an uncommon edge case request. But for quite a while, this has been the one biggest pain point of Neo4j for us: how to return a meaningful subset of traverse results instead of all results. Having a paged traverser doesn't help at all if you can neither specify the order nor know what the default ordering is based on (I asked a while back and was told it's unspecified). (And again, by order here I mean beyond just breadth first.) I'm all for simple, minimalistic and elegant APIs. I'm also all for being scenario-driven, and solving problems rather than building arbitrary features. So perhaps one solution here would be to introduce a new built-in best first order that's just like Djikstra's shortest-path API: you just specify a cost property (or function) for nodes and/or relationships. This would definitely solve our problem, and, I suspect, most people's problems in this regard. It wouldn't be a bulky feature that doesn't fit in (it would compliment Djikstra's very nicely), and it wouldn't require writing a complex Gremlin script to achieve this (I would think) common requirement. I'll get off the soapbox now. ;) Thanks for your consideration! Aseem On Thu, Aug 25, 2011 at 9:01 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: See the other mail for a pointer :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 25, 2011 at 1:00 AM, Matt Luongo m...@scholr.ly wrote: +1, we could really use that. Client-side sorting sucks. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Aug 24, 2011 at 4:43 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I've just spent a bunch of time reading into how one can control the ordering of a traverse beyond simple breadth first or depth first. More precisely, even when breadth first, how one can control *which* neighbors are traversed first. (It matters less in which order they're traversed vs. which order they're returned if you're returning all results, since you can just sort on the client. But it matters a lot if you want to use the paged traverser, since you're then only returning the first results.) I've learned that this is doable from Java by writing your own BranchSelector implementation: http://components.neo4j.org/neo4j/1.4.1/apidocs/ I've found the built-in implementations, e.g. the pre-order breadth-first and depth-first: https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderBreadthFirstSelector.java https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderDepthFirstSelector.java To achieve a custom best first, Igor Dovgiy for example shared that he modeled his implementation of the breadth-first selector, except just using a PriorityQueue instead of a regular Queue. My question is: is there any way to specify this sort of thing over the REST API instead of having to write a plugin? If not, does that sound like a reasonable feature request? I really just want something simple: nodes ordered by some timestamp property. It's killing us that we can't do this today. We might just have to look into writing this as a plugin... Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Specifying best first order of traverse over REST API
Aseem, If you can provide a nice suggestion on how exactly it would look, maybe I can implement it if the others agree? /peter Sent from my phone. On Aug 25, 2011 7:19 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Thanks Peter, we'll look into Gremlin. =) But I'll push back a bit and say it would still ask for a feature like this to be a first-class feature of the REST API. If my app is e.g. in Python, it's not super API-friendly for me to have to write Java-ish code to achieve what I want. I'd get it if what I was asking for was an uncommon edge case request. But for quite a while, this has been the one biggest pain point of Neo4j for us: how to return a meaningful subset of traverse results instead of all results. Having a paged traverser doesn't help at all if you can neither specify the order nor know what the default ordering is based on (I asked a while back and was told it's unspecified). (And again, by order here I mean beyond just breadth first.) I'm all for simple, minimalistic and elegant APIs. I'm also all for being scenario-driven, and solving problems rather than building arbitrary features. So perhaps one solution here would be to introduce a new built-in best first order that's just like Djikstra's shortest-path API: you just specify a cost property (or function) for nodes and/or relationships. This would definitely solve our problem, and, I suspect, most people's problems in this regard. It wouldn't be a bulky feature that doesn't fit in (it would compliment Djikstra's very nicely), and it wouldn't require writing a complex Gremlin script to achieve this (I would think) common requirement. I'll get off the soapbox now. ;) Thanks for your consideration! Aseem On Thu, Aug 25, 2011 at 9:01 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: See the other mail for a pointer :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 25, 2011 at 1:00 AM, Matt Luongo m...@scholr.ly wrote: +1, we could really use that. Client-side sorting sucks. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Aug 24, 2011 at 4:43 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I've just spent a bunch of time reading into how one can control the ordering of a traverse beyond simple breadth first or depth first. More precisely, even when breadth first, how one can control *which* neighbors are traversed first. (It matters less in which order they're traversed vs. which order they're returned if you're returning all results, since you can just sort on the client. But it matters a lot if you want to use the paged traverser, since you're then only returning the first results.) I've learned that this is doable from Java by writing your own BranchSelector implementation: http://components.neo4j.org/neo4j/1.4.1/apidocs/ I've found the built-in implementations, e.g. the pre-order breadth-first and depth-first: https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderBreadthFirstSelector.java https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderDepthFirstSelector.java To achieve a custom best first, Igor Dovgiy for example shared that he modeled his implementation of the breadth-first selector, except just using a PriorityQueue instead of a regular Queue. My question is: is there any way to specify this sort of thing over the REST API instead of having to write a plugin? If not, does that sound like a reasonable feature request? I really just want something simple: nodes ordered by some timestamp property. It's killing us that we can't do this today. We might just have to look into writing this as a plugin... Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Specifying best first order of traverse over REST API
Okay! Will think about this and get back to you soon. Peter, I gotta say -- your open-mindedness and willingness to help developers like me is awesome and much appreciated. Customer satisfaction, etc. =) Cheers, Aseem On Thu, Aug 25, 2011 at 11:12 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, If you can provide a nice suggestion on how exactly it would look, maybe I can implement it if the others agree? /peter Sent from my phone. On Aug 25, 2011 7:19 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Thanks Peter, we'll look into Gremlin. =) But I'll push back a bit and say it would still ask for a feature like this to be a first-class feature of the REST API. If my app is e.g. in Python, it's not super API-friendly for me to have to write Java-ish code to achieve what I want. I'd get it if what I was asking for was an uncommon edge case request. But for quite a while, this has been the one biggest pain point of Neo4j for us: how to return a meaningful subset of traverse results instead of all results. Having a paged traverser doesn't help at all if you can neither specify the order nor know what the default ordering is based on (I asked a while back and was told it's unspecified). (And again, by order here I mean beyond just breadth first.) I'm all for simple, minimalistic and elegant APIs. I'm also all for being scenario-driven, and solving problems rather than building arbitrary features. So perhaps one solution here would be to introduce a new built-in best first order that's just like Djikstra's shortest-path API: you just specify a cost property (or function) for nodes and/or relationships. This would definitely solve our problem, and, I suspect, most people's problems in this regard. It wouldn't be a bulky feature that doesn't fit in (it would compliment Djikstra's very nicely), and it wouldn't require writing a complex Gremlin script to achieve this (I would think) common requirement. I'll get off the soapbox now. ;) Thanks for your consideration! Aseem On Thu, Aug 25, 2011 at 9:01 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: See the other mail for a pointer :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 25, 2011 at 1:00 AM, Matt Luongo m...@scholr.ly wrote: +1, we could really use that. Client-side sorting sucks. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Aug 24, 2011 at 4:43 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I've just spent a bunch of time reading into how one can control the ordering of a traverse beyond simple breadth first or depth first. More precisely, even when breadth first, how one can control *which* neighbors are traversed first. (It matters less in which order they're traversed vs. which order they're returned if you're returning all results, since you can just sort on the client. But it matters a lot if you want to use the paged traverser, since you're then only returning the first results.) I've learned that this is doable from Java by writing your own BranchSelector implementation: http://components.neo4j.org/neo4j/1.4.1/apidocs/ I've found the built-in implementations, e.g. the pre-order breadth-first and depth-first: https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderBreadthFirstSelector.java https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderDepthFirstSelector.java To achieve a custom best first, Igor Dovgiy for example shared that he modeled his implementation of the breadth-first selector, except just using a PriorityQueue instead of a regular Queue. My question is: is there any way to specify this sort of thing over the REST API instead of having to write a plugin? If not, does that sound like a reasonable feature request? I really just want something simple: nodes ordered by some timestamp property. It's killing us that we can't do this today. We might just have to look into writing this as a plugin... Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Cypher: how would you query first- and second-degree friends?
Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: how would you query first- and second-degree friends?
Good and valid requests, Aseem. Consider them heard. We'll see how soon we can get it to you. Andrés On Thu, Aug 25, 2011 at 12:57 PM, Aseem Kishore aseem.kish...@gmail.comwrote: Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Wishlist: subgraphs over REST API
Sorry if I'm inundating the list w/ too many emails. =D I've been loving Cypher -- way more user-friendly and powerful than the REST API's traverse method -- but I'm finding even Cypher isn't optimized for queries where I really want to fetch a *subgraph*, not tabular data. I can give plenty of scenarios if that helps (I've even drawn diagrams!), but the theme in my queries is repeatedly that I need to understand the relationships between nodes in our graph, which means I want subgraphs. I understand that the traverse method in the REST API gives you the option of returning nodes, relationships, or paths. I'd like to propose a new return type: subgraph (or graph). Here's what I'm envisioning: the JSON contains an adjacency matrix of nodes and relationships -- referenced entirely by URLs -- and an additional map from URL to data (properties) for each referenced node and relationship. Here's an example format: https://gist.github.com/1171908 I'd *love* to be able to query and fetch subgraphs from the graph like this. That would be so awesome. I'd love thoughts and feedback. Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: how would you query first- and second-degree friends?
Thanks Andrés! Couple of random extensions to the thoughts above: - To be consistent, perhaps for optional matches, the results could always include a row where that optional match was ignored. That way, Elizabeth, null wouldn't be an inconsistency; *every* first-degree friend would have an [friend], null row. - And perhaps *that* row could be the one that has the full node data, and every subsequent row for the second optional match would just reference the URL/ID of the first-degree node. After chewing on this a lot more, though, I've realized that what I really want in this scenario is not tabular data, but a real subgraph. Hence my email just now. =) Aseem On Thu, Aug 25, 2011 at 1:34 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Good and valid requests, Aseem. Consider them heard. We'll see how soon we can get it to you. Andrés On Thu, Aug 25, 2011 at 12:57 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Server couldn't start after java.lang.OutOfMemoryError
Hey Jim, It happened again and I'm pretty sure there was no other server process running. Also this time when it crashed I saw this line in the console log (not seeing this previously): Could not load hsdis-.jnilib; library not loadable; PrintAssembly is disabled When I tried to restart, data/graph.db/message.log shows a lot of lines like the following and fail with the error message in my last email: Thu Aug 25 15:18:15 PDT 2011: Injected one phase commit, txId=11611 Thu Aug 25 15:18:15 PDT 2011: Injected one phase commit, txId=11612 Thu Aug 25 15:18:15 PDT 2011: Injected one phase commit, txId=11613 On Tue, Aug 23, 2011 at 6:23 PM, Jim Webber j...@neotechnology.com wrote: Hi Nuo, process [26266]... waiting for server to be ready. BAD. another server-process is running with [] Just to check before we dig deeper, are you certain you don't have another server process running? Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4jPHP batch insert benchmarks
I bumped the maxmemory up to 512 and ran a batch to create 10 nodes (repeated 10 times). After an average of 20 seconds, I always received the following response: HTTP/1.1 100 Continue HTTP/1.1 500 Java heap space Content-Type: text/html; charset=iso-8859-1 Cache-Control: must-revalidate,no-cache,no-store Content-Length: 4389 Server: Jetty(6.1.25) html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/ titleError 500 Java heap space/title /head body HTTP ERROR 500 pProblem accessing /db/data/batch. Reason: preJava heap space/pre/p Caused by: prejava.lang.OutOfMemoryError: Java heap space at java.util.HashMap.lt;initgt;(HashMap.java:209) at java.util.LinkedHashMap.lt;initgt;(LinkedHashMap.java:181) at org.codehaus.jackson.map.deser.UntypedObjectDeserializer.mapObject(UntypedObjectDeserializer.java:199) at org.codehaus.jackson.map.deser.UntypedObjectDeserializer.deserialize(UntypedObjectDeserializer.java:77) at org.codehaus.jackson.map.deser.UntypedObjectDeserializer.mapArray(UntypedObjectDeserializer.java:155) at org.codehaus.jackson.map.deser.UntypedObjectDeserializer.deserialize(UntypedObjectDeserializer.java:73) at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:1980) at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1271) at org.neo4j.server.rest.domain.JsonHelper.readJson(JsonHelper.java:54) at org.neo4j.server.rest.repr.formats.JsonFormat.readList(JsonFormat.java:101) at org.neo4j.server.rest.web.BatchOperationService.performBatchOperations(BatchOperationService.java:78) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:184) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:276) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:83) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:133) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:71) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1171) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1103) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1053) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1043) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:406) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:477) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:662) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) /pre hr //smallPowered by Jetty:///small/br/ br/ br/ br/ /body /html So it seems to be an issue with deserializing the JSON. Unfortunately, I am not familiar enough with Java's dev environment or tools to diagnose any further. Any ideas? -- Josh -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4jPHP-batch-insert-benchmarks-tp3282984p3285635.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
