2010/1/29 Symeon (Akis) Papadopoulos <[email protected]>: > >> I just fixed the bug and your benchmarks run fine w/o any modification >> on my machine, so they should run fine on your end too! >> > Thank you Mattias! > So, in the end it was a neo4j-index bug? Thus I assume we don't need to > modify our code (of course we'll take into consideration your previous > recommendations for using the BatchInserter, but for the sake of it, we > would also like to run the benchmark in transactional mode as well). Yep, there shouldn't be any need to modify your code at all. Yep, I understand if you'd like to benchmark the transactional mode as well! > >> The bug is just committed so the latest build will be available within >> an hour. I see that you aren't using maven, am I right? In that case >> you can download it from >> http://m2.neo4j.org/org/neo4j/neo4j-index/1.0-SNAPSHOT/ (look for .jar >> files around this date/time + 1 hour or something). Or you could check >> out the code and build it yourself: >> >> svn co https://svn.neo4j.org/components/index/trunk >> > Out of the two built jars of 29th Jan 2010, I guess we should use the > most recent one (neo4j-index-1.0-20100129.113202-3.jar), right? Hmm, I don't think it have been deployed yet... so wait a little longer for this change to be deployed. We have a little "lag" in deployed jar files, which is kind of annoying in situations like this :) > > >> You could update to the latest neo4j-kernel as well, but maybe that's >> not necessary though. >> > We'll first try out to update just the neo-index. > If we face more problems we'll proceed with the kernel update as well. > > Thanks again for the prompt response! It should be me thanking you, you gave me a complete test case triggering a bug... very nice > >> 2010/1/29 Mattias Persson <[email protected]>: >> >>> 2010/1/29 Symeon (Akis) Papadopoulos <[email protected]>: >>> >>>> Mattias Persson wrote: >>>> >>>>> Another problem I see is that you're having too granular transactions >>>>> which will slow down the insertion process quite a bit. Try grouping a >>>>> couple of thousands operations in one transaction and you'll see a >>>>> performance boost! >>>>> >>>>> FYI: I can trigger the problem you were having with lucene "too many >>>>> open files" issue. And I'm almost 100% sure that it will be resolved >>>>> if you increase the span of your transactions. >>>>> >>>>> >>>> That's what we are already doing through the class SimpleBatchTxManager >>>> (in package org.neo4j.util). We group read and write transactions in >>>> batches of some thousands. Isn't this correct? >>>> >>> You're right, sorry for missing that. >>> >>> However I just found the bug which causes this... hang on, I'll fix it >>> in an hour or two! >>> >>>> _______________________________________________ >>>> Neo mailing list >>>> [email protected] >>>> https://lists.neo4j.org/mailman/listinfo/user >>>> >>>> >>> >>> -- >>> Mattias Persson, [[email protected]] >>> Neo Technology, www.neotechnology.com >>> >>> >> >> >> >> > > _______________________________________________ > Neo mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user >
-- Mattias Persson, [[email protected]] Neo Technology, www.neotechnology.com _______________________________________________ Neo mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
