Also Klemens, it is a bit confusing that you share Java code but refer to server config?
Which Neo4j version do you use? The config seem to be for 2.1.x For 2.2.x there is dbms.pagecache.memory=8G I'd use cache_type=none Where and how do you run this code? Usually neo4j can update stuff in a multi-threaded-way really quickly. I can update 10M nodes in 20 seconds with 24 threads on a 6 core machine. You probably also want to batch it up a bit, i.e. doing 1000 to 10k nodes per transaction and job. If you have 8 cores you should use a threadpool of size 16 or 32 Michael > Am 16.10.2015 um 08:41 schrieb Klemens Engelbrechtsmüller > <[email protected]>: > > Hello! I use the neo4j java core api and want to update 10 million nodes. I > thought it will be better to do it with multithreading but the performance is > not that good (35 minutes for setting properties). > > To explain: Each node "Person" has at least one relation "POINTSREL" to a > "Point" node, which has the property "Points". I want to sum up the points > from the "Point" node and set it as property to the "Person" node. > > > > Here is my code: > > > > Transaction transaction = service.beginTx(); > ResourceIterator<Node> iterator = service.findNodes(Labels.person); > transaction.success(); > transaction.close(); > > ExecutorService executor = Executors.newFixedThreadPool(5); > > while(iterator.hasNext()){ > executor.execute(new MyJob(iterator.next())); > } > > //wait until all threads are done > executor.shutdown(); > > try { > executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS); > } catch (InterruptedException e) { > e.printStackTrace(); > } > > > And here the runnable class > > > > private class MyJob implements Runnable { > > private Node node; > > /* collect useful parameters in the constructor */ > public MyJob(Node node) { > this.node = node; > } > > public void run() { > Transaction transaction = service.beginTx(); > Iterable<org.neo4j.graphdb.Relationship> rel = > this.node.getRelationships(RelationType.POINTSREL, Direction.OUTGOING); > > double sum = 0; > for(org.neo4j.graphdb.Relationship entry : rel){ > try{ > sum += (Double)entry.getEndNode().getProperty("Points"); > } catch(Exception e){ > e.printStackTrace(); > } > } > this.node. double sum = 0; for(org.neo4j.graphdb.Relationship entry : > rel){ try{ sum += (Double)entry.getEndNode().getProperty("Points"); } > catch(Exception e){ e.printStackTrace(); } } this.node.setProperty("Sum", > sum); > > transaction.success(); > transaction.close(); > } > } > > > Is there a better (faster) way to do that? > > > > About my setting: AWS Instance with 8 CPUs and 32GB ram > > > > neo4j-wrapper.conf > > > > # Java Heap Size: by default the Java heap size is dynamically > # calculated based on available system resources. > # Uncomment these lines to set specific initial and maximum > # heap size in MB. > wrapper.java.initmemory=16000 > wrapper.java.maxmemory=16000 > > > neo4j.properties > > > > # The type of cache to use for nodes and relationships. > cache_type=soft > cache.memory_ratio=30.0 > neostore.nodestore.db.mapped_memory=2G > neostore.relationshipstore.db.mapped_memory=7G > neostore.propertystore.db.mapped_memory=2G > neostore.propertystore.db.strings.mapped_memory=2G > neostore.propertystore.db.arrays.mapped_memory=512M > > > Do you have any ideas? Thank you > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
