Short answer is "maybe". ;-) There are some cases where the transaction is an "all or nothing" scenario, others where incremental commits are OK. Having the ability to do incremental autocommits would be useful, however. In a perfect world, it could be based on a "bucket" (e.g. XXX transactions), a time (each 30 seconds), or on a memory usage rule.
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Mattias Persson Sent: Friday, July 09, 2010 7:30 AM To: Neo4j user discussions Subject: Re: [Neo4j] OutOfMemory while populating large graph 2010/7/9 Marko Rodriguez <[email protected]> > Hi, > > > Would it actually be worth something to be able to begin a transaction > which > > auto-committs stuff every X write operation, like a batch inserter mode > > which can be used in normal EmbeddedGraphDatabase? Kind of like: > > > > graphDb.beginTx( Mode.BATCH_INSERT ) > > > > ...so that you can start such a transaction and then just insert data > > without having to care about restarting it now and then? > > Thats cool! Does that already exist? In my code (like others on the list it > seems) I have a counter++ that every 20,000 inserts (some made up number > that is not going to throw an OutOfMemory) commits and the reopens a new > transaction. Sorta sux. > No it doesn't, I just wrote stuff which I though someone could think of as useful. A cool thing with just telling it to do a batch insert mode transaction (not the actual commit interval) is that it could look at how much memory it had to play around with and commit whenever it would be the most efficient, even having the ability to change the limit on the fly if the memory suddenly ran out. > Thanks, > Marko. > > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [[email protected]] Hacker, Neo Technology www.neotechnology.com _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
