Re: [Neo4j] Performance DB with large datasets
Since the small graph works well and it looks like you are performing writes together with reads a possible cause could be OS writing out dirty pages to disk when it should not. Have a look at http://wiki.neo4j.org/content/Linux_Performance_Guide While running the test execute: #watch grep -A 1 dirty /proc/vmstat to check if the numbers for nr_dirty and nr_writeback increases. If that is the case you need to change the ratio settings as described in the guide. Regards, Johan On Tue, Dec 7, 2010 at 7:22 PM, Marius Kubatz marius.kub...@udo.edu wrote: Hi, there is still no difference in the performance, which is somewhat disturbing. I cant't see the allocation of nioneo memory mapping in the java process at all. It goes up to the heap size and then stops there. Marius 2010/12/7 Marius Kubatz marius.kub...@udo.edu: Hi Peter, thank you very much for your quick reply, unfortunately there is no messages.log, seems I have an older db version. I'm sending you the ls dump from the directory: total 5318580 11 active_tx_log 4096 lucene 4096 lucene-fulltext 27 neostore 9 neostore.id 34954011 neostore.nodestore.db 9 neostore.nodestore.db.id 1917225350 2 neostore.propertystore.db 133 neostore.propertystore.db.arrays 9 neostore.propertystore.db.arrays.id 190425 neostore.propertystore.db.id 10485 neostore.propertystore.db.index 9 neostore.propertystore.db.index.id 10449 neostore.propertystore.db.index.keys 9 neostore.propertystore.db.index.keys.id 2047597776 neostore.propertystore.db.strings 30790905 neostore.propertystore.db.strings.id 901093347 neostore.relationshipstore.db 149433 neostore.relationshipstore.db.id 20 neostore.relationshiptypestore.db 9 neostore.relationshiptypestore.db.id 215 neostore.relationshiptypestore.db.names 9 neostore.relationshiptypestore.db.names.id 2097160 nioneo_logical.log.1 4 nioneo_logical.log.active 88 tm_tx_log.1 29365 tm_tx_log.2 I have 3.848.862 nodes and 53.355.402 relationships in my graph. thus I created the following neo4j props file: neostore.nodestore.db.mapped_memory=30M neostore.relationshipstore.db.mapped_memory=1685M neostore.propertystore.db.mapped_memory=1000M neostore.propertystore.db.strings.mapped_memory=1000M neostore.propertystore.db.arrays.mapped_memory=0M I have 8GB Ram and gave JavaVM is running with -Xmx2048m , and the mapping should consume 4GB. Just started the experiment again the first run is traversing a neighborhood of : 124 nodes and 2.279.166 edges, so I'm very curious how this will end :) Thanks for your help! Regards Marius -- Programs must be written for people to read, and only incidentally for machines to execute. - Abelson Sussman, SICP, preface to the first edition ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4j REST API and JAXB
Hi all, is anyone using the REST interface and JAXB?
I have some problems with the traversal response of POST call to
/node/3/traverse/path which returns:
[ {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;, http://localhost:/node/1;
],
length : 1,
relationships : [ http://localhost:/relationship/6; ],
end : http://localhost:/node/1;
}, {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;, http://localhost:/node/2;
],
length : 1,
relationships : [ http://localhost:/relationship/2; ],
end : http://localhost:/node/2;
} ]
Does anyone know how to define the JAXB objects?
--
Christopher
twitter: @fakod
blog: http://blog.fakod.eu
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j REST API and JAXB
Christopher,
are you trying to build your own server, or are you using Neo4j Server?
The REST API is now part of the server component, we are right now
defining a mechanism to mount your own server extensions without
breaking the REST hypermedia of the existing API.
What are you trying to do?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Dec 8, 2010 at 3:52 PM, Christopher Schmidt
fakod...@googlemail.com wrote:
Hi all, is anyone using the REST interface and JAXB?
I have some problems with the traversal response of POST call to
/node/3/traverse/path which returns:
[ {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;, http://localhost:/node/1;
],
length : 1,
relationships : [ http://localhost:/relationship/6; ],
end : http://localhost:/node/1;
}, {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;, http://localhost:/node/2;
],
length : 1,
relationships : [ http://localhost:/relationship/2; ],
end : http://localhost:/node/2;
} ]
Does anyone know how to define the JAXB objects?
--
Christopher
twitter: @fakod
blog: http://blog.fakod.eu
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j REST API and JAXB
I have not tried it, but the scala-template project uses JAXB:
https://github.com/ept/neo4j-scala-template
The project contains a few working examples of Scala + JAXB. Perhaps it
will provide some further insights.
--
Ivan
On Wed, Dec 8, 2010 at 9:52 AM, Christopher Schmidt fakod...@googlemail.com
wrote:
Hi all, is anyone using the REST interface and JAXB?
I have some problems with the traversal response of POST call to
/node/3/traverse/path which returns:
[ {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;, http://localhost:/node/1
],
length : 1,
relationships : [ http://localhost:/relationship/6; ],
end : http://localhost:/node/1;
}, {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;, http://localhost:/node/2
],
length : 1,
relationships : [ http://localhost:/relationship/2; ],
end : http://localhost:/node/2;
} ]
Does anyone know how to define the JAXB objects?
--
Christopher
twitter: @fakod
blog: http://blog.fakod.eu
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j REST API and JAXB
I have a little spare time ;-) and I try to create a/another very smart
REST client DSL in Scala.
For testing and test data I am using the Neo4j Server from here:
http://blog.neo4j.org/2010/04/neo4j-rest-server-part1-get-it-going.html
Client library is Jersey, JSON marshaling and unmarshaling lib is JAXB. The
current client Scala code is:
rest {
implicit s =
val traversalPath = node/3/traverse/path.POST[JSONArray] =
PathRequest(order = depth first, max_depth = 4, uniqueness = node path)
println(Array length: + traversalPath.length) // prints 2
}
The above JSONArray works fine, but if I use Java objects
(as XmlRootElement) I get unmarshaling exceptions. It would be helpful If
there is someone out there who already created some JAXB objects for
Neo4j...
Christopher
On Wed, Dec 8, 2010 at 3:56 PM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Christopher,
are you trying to build your own server, or are you using Neo4j Server?
The REST API is now part of the server component, we are right now
defining a mechanism to mount your own server extensions without
breaking the REST hypermedia of the existing API.
What are you trying to do?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Dec 8, 2010 at 3:52 PM, Christopher Schmidt
fakod...@googlemail.com wrote:
Hi all, is anyone using the REST interface and JAXB?
I have some problems with the traversal response of POST call to
/node/3/traverse/path which returns:
[ {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;,
http://localhost:/node/1;
],
length : 1,
relationships : [ http://localhost:/relationship/6; ],
end : http://localhost:/node/1;
}, {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;,
http://localhost:/node/2;
],
length : 1,
relationships : [ http://localhost:/relationship/2; ],
end : http://localhost:/node/2;
} ]
Does anyone know how to define the JAXB objects?
--
Christopher
twitter: @fakod
blog: http://blog.fakod.eu
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Christopher
twitter: @fakod
blog: http://blog.fakod.eu
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j REST API and JAXB
Hi Ivan, there is only one JAXB object here:
https://github.com/ept/neo4j-scala-template/blob/master/src/main/scala/com/example/models/Moo.scala
But thanks anyway :-)
BTW: For this Rest GET index call:
rest {
implicit s =
val index = /index.GET[GetIndex]
}
the following JAXB objects do work fine:
@XmlAccessorType(XmlAccessType.FIELD)
@XmlRootElement
class GetIndex {
var node:Array[Node] = _
}
@XmlAccessorType(XmlAccessType.FIELD)
class Node {
var template:String = _
var `type`:String = _
}
The returned JSON object to unmarshal is:
{
node : [ {
template : http://localhost:/index/node/{key}/{value};,
type : lookup
} ]
}
On Wed, Dec 8, 2010 at 4:08 PM, ivan Brusic i...@brusic.com wrote:
I have not tried it, but the scala-template project uses JAXB:
https://github.com/ept/neo4j-scala-template
The project contains a few working examples of Scala + JAXB. Perhaps it
will provide some further insights.
--
Ivan
On Wed, Dec 8, 2010 at 9:52 AM, Christopher Schmidt
fakod...@googlemail.com
wrote:
Hi all, is anyone using the REST interface and JAXB?
I have some problems with the traversal response of POST call to
/node/3/traverse/path which returns:
[ {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;,
http://localhost:/node/1
],
length : 1,
relationships : [ http://localhost:/relationship/6; ],
end : http://localhost:/node/1;
}, {
start : http://localhost:/node/3;,
nodes : [ http://localhost:/node/3;,
http://localhost:/node/2
],
length : 1,
relationships : [ http://localhost:/relationship/2; ],
end : http://localhost:/node/2;
} ]
Does anyone know how to define the JAXB objects?
--
Christopher
twitter: @fakod
blog: http://blog.fakod.eu
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Christopher
twitter: @fakod
blog: http://blog.fakod.eu
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j REST API and JAXB
Hi Christopher, Although I can't immediately give you a solution to your JAXB question, I can give you another minor problem :-) The REST api is now merged with the Neo4j server and all current and future work is happening on the server component. It shouldn't involve any code changes for you, but can you switch to server to host your DB just so I know you're on the latest version? Jim PS - There might be a config change needed, the default port changed to 7474 compared to the old, deprecated REST api. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
