[Neo4j] How to boost performance?

2011-11-23 Thread Vinicius Carvalho
Hi there, I've posted a few days ago about the POC I'm doing here at my company. I have some initial numbers and I'd like to ask for some help here in order to promote neo4j here in LMI Ericsson. I've loaded a mySQL db with a really simple entity, that pretty much only represents a node and

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Peter Neubauer
Vinicius, in order to cut down on the REST JSON overhead (which you don't have in the RDBMS case), maybe you could look at just counting the results, something like START n = node(3) MATCH n--()--(x) return count(x) And see what happens? Cheers, /peter neubauer GTalk:      neubauer.peter

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Vinicius Carvalho
Hi Peter, thanks, that indeed boosts to lower 10 ms, but we really need to access the nodes to perform the operations. I know that to be fair on a test, we should be running an embedded version of neo4j inside the appserver, after all the cache resides in the same JVM. But I'm not worried with

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Peter Neubauer
Vinicius, in real-world usages, you probably want to build a REST API that operates on domain and usecase level. Shuffling 6K nodes forth and back and resolve properties sounds not good to me, given the REST discovery overhead of the JSON representation. At the very least, you could do START n =

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Vinicius Carvalho
Peter, I agree with you, would not be the ideal scenario, I would never access a REST api that way, at minimum pagination should be there. But we have this really specific case, where we need to have (not whole domain) but at least part of it (power, bearing, gain, frequency) to perform some

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Michael Hunger
Vinicius, first: did you have any issues importing the data into Neo4j? second: your example used cypher which is not optimized for performance (yet!). This is in our plans for the next two releases of neo4j. So if you want to see the real performance of neo4j, please use the traversal

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Rick Bullotta
. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Vinicius Carvalho Sent: Wednesday, November 23, 2011 6:55 AM To: Neo4j user discussions Subject: Re: [Neo4j] How to boost performance? Hi Peter, thanks, that indeed boosts to lower 10 ms

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Vinicius Carvalho
Hi Michael, thanks. The data load was fine, I've used your script with the BathInserter. Memory footprint was really slow, I think the peak was 200mb of heap usage. I did something really retarded and left a logger.info, which slowed things a bit, but the process was really smooth. Many thanks on

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Michael Hunger
Please use EmbeddedGraphDatabase, EmbeddedReadOnlyGraphDatabase caches a snapshot of the data in its caches and doesn't get update-changes. Michael Am 23.11.2011 um 14:39 schrieb Vinicius Carvalho: Hi Michael, thanks. The data load was fine, I've used your script with the BathInserter.

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Vinicius Carvalho
But wouldn't it mean that I need to have exclusive lock on the db? I would like to keep the server running pointing at the same data directory. Regards On Wed, Nov 23, 2011 at 1:50 PM, Michael Hunger michael.hun...@neotechnology.com wrote: Please use EmbeddedGraphDatabase,

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Michael Hunger
Just make sure that it is just a snapshot of the data and doesn't update its caches. Otherwise you will run into synchronization issues. See also this thread and Tobias' explanations around it:

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Vinicius Carvalho
Tks, for this test it's just a readonly graph now, so I don't think I'll run into synchronization issues. As we proceed with tests, I do hope that we will have one day is a HA version of neo4j. And as Jim's said in that thread, use it for other to read the graph. Regards On Wed, Nov 23, 2011 at

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Vinicius Carvalho
Hi Michael, this is going to be a newbie question, so please forgive me: I've re ran the tests with your examples, and using a embedded database. First thing: Whooping FAST! Mind blowing :D - 5ms But ... I got different results, same time though which is great, proves the exact thing that

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Michael Hunger
You might run into uniqueness constraints. There are different uniqueness constraints that can be specified for a traversal, meaning how often is a node visited during one. (see here: http://components.neo4j.org/neo4j/1.6-SNAPSHOT/apidocs/org/neo4j/kernel/Uniqueness.html) You probably want to

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Vinicius Carvalho
Tks once again Michael. Adding the Direction.Outgoing nailed it, I have the same dataset as the ones using cypher On the first one using the traversaldescription, I've modified to: TraversalDescription traversalQuery =

Re: [Neo4j] How to boost performance?

2011-11-23 Thread Linan Wang
hi, i noticed that you were using db.getNodeById to retrieve the starting node. the performance of this method call is quite different from real world apps especially those have external unique id. basically you need to index the external id yourself and get it via index call. besides, i'd suggest