Thanks Michael, that makes it crystal clear (i am totally going for it :) )
On Thursday, February 6, 2014 5:09:46 PM UTC-5, Michael Hunger wrote: > > As the tx log is not forced to disk you might loose a few seconds of > inserted data when it crashes > > But the data on disk is still consistent > > Sent from mobile device > > Am 06.02.2014 um 20:06 schrieb Debajyoti Roy <[email protected]<javascript:> > >: > > ((GraphDatabaseAPI)neo4jGraph).tx().unforced().begin() is awesome but are > there any down sides to doing this? > > > > > On Tuesday, February 4, 2014 3:38:13 AM UTC-5, Michael Hunger wrote: >> >> No, it just indicates that the API might change in the future. >> >> Am 04.02.2014 um 09:16 schrieb Sotiris Beis <[email protected]>: >> >> I don't have blocks of data, I measure the insertion time of 1,000 nodes >> and their edges (which I call a block). I am doing that because I want to >> simulate the creation of a graph by single element insertion. >> >> This >> >> Transaction tx = ((GraphDatabaseAPI)neo4jGraph).tx().unforced().begin(); >> >> does the job, but the tx() function says it's deprecated. Is this going >> to be a problem? >> >> Sotiris >> >> >> On 02/03/2014 06:41 PM, Michael Hunger wrote: >> >> Why do you do individual inserts when you have blocks of data? >> >> You can often aggregate events on the application level to be inserted >> as a bigger batch. >> see: http://maxdemarzi.com/2013/09/05/scaling-writes/ >> >> Otherwise you can also release the force-write-to-log constraint and use >> >> Transaction tx = ((GraphDatabaseAPI)neo4jGraph).tx().unforced().begin(); >> >> Instead of neo4jGraph.beginTx(); >> >> >> Am 03.02.2014 um 15:40 schrieb Sotiris Beis <[email protected]>: >> >> In addition to my previous question is there any suggestion to improve >> the performance of the following code. I use this to simulate the needed >> time to create a graph if the graph is created by single insertions >> (incrementally not batch). So I commit every transaction and I measure the >> time that a block needs to be inserted. A block consists of a 1,000 nodes >> and their edges. Here is the code: >> >> import java.io.BufferedReader; >> import java.io.FileInputStream; >> import java.io.IOException; >> import java.io.InputStreamReader; >> import java.util.ArrayList; >> import java.util.List; >> >> import org.neo4j.graphdb.GraphDatabaseService; >> import org.neo4j.graphdb.Node; >> import org.neo4j.graphdb.RelationshipType; >> import org.neo4j.graphdb.Transaction; >> import org.neo4j.graphdb.factory.GraphDatabaseFactory; >> import org.neo4j.graphdb.factory.GraphDatabaseSetting; >> import org.neo4j.graphdb.index.Index; >> >> public class Neo4jSingleInsertion implements Insertion { >> >> public static String INSERTION_TIMES_OUTPUT_PATH = >> "data/neo4j.insertion.times"; >> >> private static int count; >> >> private GraphDatabaseService neo4jGraph = null; >> private Index<Node> nodeIndex = null; >> >> private static enum RelTypes implements RelationshipType { >> SIMILAR >> } >> >> public static void main(String args[]) { >> Neo4jSingleInsertion test = new Neo4jSingleInsertion(); >> test.startup("data/neo4j"); >> test.createGraph("data/enronEdges.txt"); >> test.shutdown(); >> } >> >> public void startup(String neo4jDBDir) { >> System.out.println("The Neo4j database is now starting . . . ."); >> neo4jGraph = new GraphDatabaseFactory().newEmbeddedDatabase( >> neo4jDBDir); >> nodeIndex = neo4jGraph.index().forNodes("nodes"); >> } >> >> public void shutdown() { >> System.out.println("The Neo4j database is now shuting down . . . >> ."); >> if(neo4jGraph != null) { >> neo4jGraph.shutdown(); >> nodeIndex = null; >> } >> } >> >> public void createGraph(String datasetDir) { >> count++; >> System.out.println("Incrementally creating the Neo4j database . >> . . ."); >> List<Double> insertionTimes = new ArrayList<Double>(); >> try { >> BufferedReader reader = new BufferedReader(new >> InputStreamReader(new FileInputStream(datasetDir))); >> String line; >> int nodesCounter = 0; >> int lineCounter = 1; >> Transaction tx = null; >> long start = System.currentTimeMillis(); >> long duration; >> while((line = reader.readLine()) != null) { >> if(lineCounter > 4) { >> String[] parts = line.split("\t"); >> >> Node srcNode = nodeIndex.get("nodeId", parts[0]). >> getSingle(); >> if(srcNode == null) { >> tx = neo4jGraph.beginTx(); >> srcNode = neo4jGraph.createNode(); >> srcNode.setProperty("nodeId", parts[0]); >> nodeIndex.add(srcNode, "nodeId", parts[0]); >> tx.success(); >> tx.finish(); >> nodesCounter++; >> } >> >> if(nodesCounter == 1000) { >> duration = System.currentTimeMillis() - start; >> insertionTimes.add((double) duration); >> nodesCounter = 0; >> start = System.currentTimeMillis(); >> } >> >> Node dstNode = nodeIndex.get("nodeId", parts[1]). >> getSingle(); >> if(dstNode == null) { >> tx = neo4jGraph.beginTx(); >> dstNode = neo4jGraph.createNode(); >> dstNode.setProperty("nodeId", parts[1]); >> nodeIndex.add(dstNode, "nodeId", parts[1]); >> tx.success(); >> tx.finish(); >> nodesCounter++; >> } >> >> tx = neo4jGraph.beginTx(); >> srcNode.createRelationshipTo(dstNode, RelTypes. >> SIMILAR); >> tx.success(); >> tx.finish(); >> >> if(nodesCounter == 1000) { >> duration = System.currentTimeMillis() - start; >> insertionTimes.add((double) duration); >> nodesCounter = 0; >> start = System.currentTimeMillis(); >> } >> } >> lineCounter++; >> } >> duration = System.currentTimeMillis() - start; >> insertionTimes.add((double) duration); >> reader.close(); >> } >> catch (IOException e) { >> e.printStackTrace(); >> } >> Utils utils = new Utils(); >> utils.writeTimes(insertionTimes, Neo4jSingleInsertion. >> INSERTION_TIMES_OUTPUT_PATH+"."+count); >> } >> >> } >> >> Thanks, >> Sotiris >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Neo4j" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/neo4j/uZqRgCBc9lg/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
