Re: [Neo4j] Performance problem inserting nodes with many short string properties

2010-07-24 Thread Mattias Persson
2010/7/22 Jeff Klann jkl...@iupui.edu And in the meantime I'm rewriting some code to use the batch inserter, but the LuceneIndexBatchInserterImpl is not reading an index that already exists in the db! (I'm trying to use a pre-existing index to find parent nodes for the nodes I'm inserting.)

Re: [Neo4j] Performance problem inserting nodes with many short string properties

2010-07-23 Thread Jeff Klann
Great. Since you summarize the problem on that page and only mention small transactions, I'd think you might want to mention my problem too... that making a transaction *too large* also demolishes performance, because the transaction's on the heap. - Jeff Klann On Thu, Jul 22, 2010 at 5:19 PM,

Re: [Neo4j] Performance problem inserting nodes with many short string properties

2010-07-23 Thread Peter Neubauer
Ahh, thanks for the hint, updated :) Cheers, /peter neubauer COO and Sales, Neo Technology GTalk:      neubauer.peter Skype       peter.neubauer Phone       +46 704 106975 LinkedIn   http://www.linkedin.com/in/neubauer Twitter      http://twitter.com/peterneubauer http://www.neo4j.org         

Re: [Neo4j] Performance problem inserting nodes with many short string properties

2010-07-22 Thread Jeff Klann
And in the meantime I'm rewriting some code to use the batch inserter, but the LuceneIndexBatchInserterImpl is not reading an index that already exists in the db! (I'm trying to use a pre-existing index to find parent nodes for the nodes I'm inserting.) The shell verifies the index is in fact

Re: [Neo4j] Performance problem inserting nodes with many short string properties

2010-07-22 Thread Toby Matejovsky
I believe the issue is the size of the transaction, based on a similar question. Can you batch the inserts into transactions with about 30k per transaction? -- Toby Matejovsky On Thu, Jul 22, 2010 at 11:27 AM, Jeff Klann jkl...@iupui.edu wrote: I'm stumped on this one. I'm getting the fast

Re: [Neo4j] Performance problem inserting nodes with many short string properties

2010-07-22 Thread Jeff Klann
Yes, thanks that fixed it. I noticed this is documented at http://wiki.neo4j.org/content/Transactions under Big Transactions. Perhaps there should be a link here from the performance guide. Thanks, Jeff Klann On Thu, Jul 22, 2010 at 3:30 PM, Toby Matejovsky toby.matejov...@gmail.comwrote: I

Re: [Neo4j] Performance problem inserting nodes with many short string properties

2010-07-22 Thread Peter Neubauer
Jeff, added a link in the Wiki, http://wiki.neo4j.org/content/Neo4j_Performance_Guide#Write_performance . Any other place you think would be good to have this info in? Cheers, /peter neubauer COO and Sales, Neo Technology GTalk:      neubauer.peter Skype       peter.neubauer Phone       +46