Re: [Neo4j] Batch uploading to REST server
I setup visualvm with a jmx connection to the server running neo4j I wasn't able to CPU profiling working, it had an error message saying getThreadCpuTime was expecting long but got something else. I do have thread and heap dumps though if those would be useful. When I was running the batch upload I saw that the CPU usage never went over one core ~15% total cpu usage. Also I noticed that when I did a batch job of 1.5k nodes, 3k relationships the network usage would spike and be consistent and then fall for the upload and then spike and fall again for the read back from the server. However when I did 15k nodes and 30k relationships the network would spike, fall and then have short regular spike of only a couple KB/s. Is there some internal paging going on with the REST server, why doesn't it transfer the file all at once and then process it? The 1.5k job took about 25 seconds, the 15k job took 1220 seconds. Below is the configuration I have for the server, the machine itself has 32GB of ram and 8 64bit cores: -XX:+DisableExplicitGC -XX:+UseParallelGC -XX:+UseNUMA -Xss1024m -Xms2048m -Xmx4056m Neo4j properties: neostore.nodestore.db.mapped_memory=300M neostore.relationshipstore.db.mapped_memory=600M neostore.propertystore.db.mapped_memory=600M neostore.propertystore.db.strings.mapped_memory=1024M neostore.propertystore.db.arrays.mapped_memory=1024M cache_type=strong use_memory_mapped_buffers=false -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Batch-uploading-to-REST-server-tp340p3510431.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Batch uploading to REST server
Not so much a script as just using unix command: POST http://foo.bar:7474/db/data/batch < neo_batch.txt File can be found at (this is only for a 100 node binary tree as I didn't want to post the 10k one due to size): https://github.com/vistorve/neo4j_batch/blob/master/neo_batch.txt -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Batch-uploading-to-REST-server-tp340p3478032.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Batch uploading to REST server
Hi, I was benchmarking some batch creation jobs to the REST server. I tried to load ~15k nodes, 15k relationships between them, and indexing all the nodes. This took in the range of 1 - 2 hours which to me seems to be very slow. Is there anything I can do in server configuration or the like to speed up the process? The 623KB file was POSTed to /db/data/batch. Server is community 1.5.M02. Thanks, Alex -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Batch-uploading-to-REST-server-tp340p340.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Error adding item to idexes using Neo4jrestclient python
Awesome, all is working now! Thanks Javier! -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Error-adding-item-to-idexes-using-Neo4jrestclient-python-tp3468029p3468531.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Error adding item to idexes using Neo4jrestclient python
neo4jrestclient 1.4.5 neo4j server I tried both latest snapshot (1.5-38-g831a83d-dirty) and the current milestone 1.5.M02 -Alex -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Error-adding-item-to-idexes-using-Neo4jrestclient-python-tp3468029p3468154.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Error adding item to idexes using Neo4jrestclient python
Hi, I can create nodes and indexes and traverse fine but when I try to add an item to an index i get the following error: Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.6/dist-packages/neo4jrestclient/client.py", line 1030, in __setitem__ ", data %s" % (request_url, url_ref)) neo4jrestclient.request.StatusException: Error [405]: Method Not Allowed. Specified method is invalid for this server.. Error requesting index with POST http://localhost:7474/db/data/index/node/seg_nodes/name/x , data http://localhost:7474/db/data/node/6 Any help you could provide would be great. Thanks, Alex -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Error-adding-item-to-idexes-using-Neo4jrestclient-python-tp3468029p3468029.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
