Did you configure the heap sizes for both neo4j instances and also the memory-mapped settings?
Otherwise a neo4j-instance will assume it is alone on a physical machine and tries to use the available physical RAM for itself. So if you run more than one instance on a machine you MUST configure the memory mapped setting manually (http://wiki.neo4j.org/content/Configuration_Settings). But 100 - 120 nodes per minute processed is much to slow. What OS are you running on ? (if linux, what scheduler is used -> try to use deadline or as). What does top, iostat or vmstat say about io-waits ? What does jconsole or the GC logs say about memory usage and full-gcs ? >From your email I read that you have a system with 2 cores and 7 gigs of RAM? What JVM are you using? Thanks Michael Am 08.08.2011 um 14:37 schrieb Igor Dovgiy: > Hi all, > > I wonder is that a common case, or rather a very specific one; anyway, would > deeply appreciate any help. :) > > For a long time we've been thinking about how to speed up our operations. > Those are traversals: not very deep (level of depth is 2 at most), but > rather complex ones (with custom-made evaluator and selector policy > classes), and our DB for now has more than 4M nodes (but should have at > least 40M when deployed). > > In this quest I've attempted to run multiple server instances at once - and, > well, was greatly surprised by outcome of this decision. :) > > In single-instance mode I got about 100-120 nodes per minute processed, no > matter how many client processes were thrown at DB (results are fetched via > REST API). Results were more-o-less stable, though. > > But in double-instance mode speed bumped up to 200-250 nodes ppm right after > the start, held there for 10 minutes - and then crashed down to meager 50-60 > nodes. Then again, after spending a considerably long time at this level, > for twenty minutes it went to 200-250 nodes ppm again! > > Watching system running with htop, I saw a clear pattern: when both cpu > cores were loaded at 90%, processing speed was great, but, sadly, most of > the time cpu was quite undertasked. ) > I wonder what might cause such behavior, and is there a way for me to > improve the performance, perhaps with additional settings for JVM? > > I've already updated Neo4j for 1.4.1 GA, but the problem still there. We've > got about 7 GB RAM free, two Neo4j instances eat up about 4 GB total. > > P.S. Tried to set java processes cpu core affinity (with taskset), got zero > effect. > > -- > -- iD > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
