Great, so maybe neo4j-index should be updated to depend on Lucene 2.9.3. 2010/7/9 Bill Janssen <jans...@parc.com>
> Note that a couple of memory issues are fixed in Lucene 2.9.3. Leaking > when indexing big docs, and indolent reclamation of space from the > FieldCache. > > Bill > > Arijit Mukherjee <ariji...@gmail.com> wrote: > > > I've a similar problem. Although I'm not going out of memory yet, I can > see > > the heap constantly growing, and JProfiler says most of it is due to the > > Lucene indexing. And even if I do the commit after every X transactions, > > once the population is finished, the final commit is done, and the graph > db > > closed - the heap stays like that - almost full. An explicit gc will > clean > > up some part, but not fully. > > > > Arijit > > > > On 9 July 2010 17:00, Mattias Persson <matt...@neotechnology.com> wrote: > > > > > 2010/7/9 Marko Rodriguez <okramma...@gmail.com> > > > > > > > Hi, > > > > > > > > > Would it actually be worth something to be able to begin a > transaction > > > > which > > > > > auto-committs stuff every X write operation, like a batch inserter > mode > > > > > which can be used in normal EmbeddedGraphDatabase? Kind of like: > > > > > > > > > > graphDb.beginTx( Mode.BATCH_INSERT ) > > > > > > > > > > ...so that you can start such a transaction and then just insert > data > > > > > without having to care about restarting it now and then? > > > > > > > > Thats cool! Does that already exist? In my code (like others on the > list > > > it > > > > seems) I have a counter++ that every 20,000 inserts (some made up > number > > > > that is not going to throw an OutOfMemory) commits and the reopens a > new > > > > transaction. Sorta sux. > > > > > > > > > > No it doesn't, I just wrote stuff which I though someone could think of > as > > > useful. A cool thing with just telling it to do a batch insert mode > > > transaction (not the actual commit interval) is that it could look at > how > > > much memory it had to play around with and commit whenever it would be > the > > > most efficient, even having the ability to change the limit on the fly > if > > > the memory suddenly ran out. > > > > > > > > > > Thanks, > > > > Marko. > > > > > > > > _______________________________________________ > > > > Neo4j mailing list > > > > User@lists.neo4j.org > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > > > > > > -- > > > Mattias Persson, [matt...@neotechnology.com] > > > Hacker, Neo Technology > > > www.neotechnology.com > > > _______________________________________________ > > > Neo4j mailing list > > > User@lists.neo4j.org > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > -- > > "And when the night is cloudy, > > There is still a light that shines on me, > > Shine on until tomorrow, let it be." > > _______________________________________________ > > Neo4j mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
