Jeff, added a link in the Wiki, http://wiki.neo4j.org/content/Neo4j_Performance_Guide#Write_performance . Any other place you think would be good to have this info in?
Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Jul 22, 2010 at 10:48 PM, Jeff Klann <jkl...@iupui.edu> wrote: > Yes, thanks that fixed it. I noticed this is documented at > http://wiki.neo4j.org/content/Transactions under "Big Transactions". Perhaps > there should be a link here from the performance guide. > > Thanks, > Jeff Klann > > On Thu, Jul 22, 2010 at 3:30 PM, Toby Matejovsky > <toby.matejov...@gmail.com>wrote: > >> I believe the issue is the size of the transaction, based on a similar >> question. Can you batch the inserts into transactions with about 30k per >> transaction? >> >> -- >> Toby Matejovsky >> >> >> On Thu, Jul 22, 2010 at 11:27 AM, Jeff Klann <jkl...@iupui.edu> wrote: >> >> > I'm stumped on this one. >> > >> > I'm getting the "fast write performance at first that slows to a crawl" >> > issue described in the performance guide, so I increased the Linux >> > dirty_page ratio (all the way up to 80%), turned of auto log rotation, >> and >> > increased the size of the memory mapped cache. This issue is still >> > happening >> > exactly as before. >> > >> > I've narrowed my problem to this: >> > *If I insert a lot of nodes with about 50 short string properties each, >> > the performance slows to a crawl at about 40,000 inserts (and it stays >> > slow)* ... however if I don't insert the properties the performance is >> > fine. >> > >> > What am I doing wrong? The machine currently has a small amount of RAM, >> but >> > I don't understand why that would impact pure insertion, and only after >> > thousands of inserts. (I don't read the properties back after adding >> them.) >> > I have not used BatchInserter because it is nice to have normal database >> > access for some parts of this database builder program I'm writing, but >> if >> > that's the only way I could refactor. Also all these inserts are within >> one >> > transaction (about 100k nodes per transaction) - do I need to split this >> > into smaller transactions? >> > >> > Thanks, >> > Jeff Klann >> > _______________________________________________ >> > Neo4j mailing list >> > User@lists.neo4j.org >> > https://lists.neo4j.org/mailman/listinfo/user >> > >> _______________________________________________ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
