Hi David/Michael, After playing around with the settings, I believe our settings for memory mapped files was incorrectly setup, it was set up to take too much memory, due to this the GC would always try to clean up more memory than required.
After bringing down the mapped memory settings to a recommended settings as per neo4j documentation the GC is now under control. Thank you very much for helping us resolve this issue. We will continue to play with the G1 settings recommended by David and see if that helps further optimize the GC. Thanks, Virat On Tue, Mar 4, 2014 at 9:37 PM, David Fox <[email protected]> wrote: > What GC method are you using? We use about a 30gb heap and had a decent > amount of issues with ConcMarkSweep combined with a high amount of > requests. A few people at Neo4j recommended G1 and it helped us a lot. Not > sure how your app works, but with ours it usually takes G1 a little time to > adapt to our traffic and then GC spikes go down significantly and we see > very few. > > > On Tuesday, March 4, 2014 3:40:58 PM UTC-5, Virat Gohil wrote: >> >> Hi, >> >> Before I start, I would like to commend the efforts of the fantastic team >> for making a beautiful and intuitive Graph database. I am new to using >> Neo4j and have been using it for about 2 months now and so far I love it! >> >> I have a few queries with respect to tuning Neo4j for performance, as we >> are having issues where the garbage collection holds the application >> threads for a long time, the following is a sample from the logs: >> >> 2014-03-04 20:05:24.817+0000 INFO [o.n.k.EmbeddedGraphDatabase]: GC >> Monitor: Application threads blocked for an additional 3539ms [total block >> time: 442.367s] >> 2014-03-04 20:05:28.024+0000 INFO [o.n.k.EmbeddedGraphDatabase]: GC >> Monitor: Application threads blocked for an additional 3107ms [total block >> time: 445.474s] >> 2014-03-04 20:05:37.474+0000 INFO [o.n.k.EmbeddedGraphDatabase]: GC >> Monitor: Application threads blocked for an additional 9350ms [total block >> time: 454.824s] >> 2014-03-04 20:05:45.909+0000 INFO [o.n.k.EmbeddedGraphDatabase]: GC >> Monitor: Application threads blocked for an additional 8333ms [total block >> time: 463.157s] >> 2014-03-04 20:06:00.581+0000 INFO [o.n.k.EmbeddedGraphDatabase]: GC >> Monitor: Application threads blocked for an additional 14570ms [total block >> time: 477.727s] >> 2014-03-04 20:06:10.810+0000 INFO [o.n.k.EmbeddedGraphDatabase]: GC >> Monitor: Application threads blocked for an additional 10129ms [total block >> time: 487.856s] >> 2014-03-04 20:06:24.881+0000 INFO [o.n.k.EmbeddedGraphDatabase]: GC >> Monitor: Application threads blocked for an additional 13971ms [total block >> time: 501.827s] >> >> We have a fairly powerful box with about 120G of RAM available at our >> disposal, this box is only going to run only Neo4j DB instance. The >> following is the statistics for our database: >> >> Number of nodes: ~37Million >> Number of relationships: ~100Million >> Number of properties: ~300Million >> >> File sizes from disk: >> neostore.nodestore.db = 319MB >> neostore.propertystore.db = 15GB >> neostore.relationshipstore.db = 3.2GB >> neostore.propertystore.db.arrays= 11MB >> neostore.propertystore.db.strings = 7GB >> >> The typical graph depth is 3 levels for most of our queries. >> >> I have the following queries: >> 1. I read a few articles about enabling Object caching along with mapped >> memory, most of the articles suggested a way to pass the caching attributes >> as properties to the embedded database. However, in our case, we are >> communicating to the DB from outside the box using rest (spring-data-neo4j >> to be exact). I could not find any article that shows how to configure the >> object caching using the properties file. Any pointers to setting up the >> object caches using properties file will be very helpful. >> 2. Keeping the above stats in mind, what would be a recommended heap >> size? and node/relationship cache sizes? >> >> Please let me know if any more data can help clarify things further. >> >> Thanks! >> >> Virat >> >> -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
