On Wed, Sep 18, 2013 at 3:02 AM, spollack <[email protected]> wrote: > I'm trying to track down a significant RSS increase, and performance > decrease, moving from node 0.8.x to 0.10.x. > > WIth the backend worker portion of our application doing its standard > workload, under node 0.8.23, our RSS stabilizes under 400MB. The workload > completes in roughly 12-13 hours. > > With the same application code and workload, under node 0.10.17 (no other > changes whatsoever), our RSS rapidly grows to ~1GB (within the first 15 > minutes or so). This is approximately the amount memory available on this > machine, and so we thrash and performance degrades. The workload takes > roughly 20-27 hours to finish. > > Now my first suspect is the change in garbage collection behavior between > the major versions of node. > > Our application is run with --expose_gc and calls system.gc() 5 times in a > row, every 60 seconds. This strategy worked well under node 0.8.x. > > After seeing the poor results under node 0.10.x, i've tried a handful of > gc-related experiments, such as increasing the frequency of system.gc() > calls to 5 times in a row every 5 seconds, which slowed the RSS increase but > didn't help overall performance. I also tried the --always_compact flag, > with similar results (somewhat less RSS memory usage, but overall > performance still not close to before). > > Under both 0.8.x and 0.10.x, heap memory usage is low, in the 100MB range. > So it's non-heap memory usage that is getting us. > > Potentially some other change in node 0.10.x is using way more non-heap > memory? > > Ideas on what's going on here? > > Thanks, > Seth
You're correct in that there's been a number of changes to V8's garbage collection heuristics. The short version is that it tries to be as lazy as possible now and that shows itself in the way it will usually prefer to grow the heap instead of performing a major garbage collection cycle. If you're running in a memory-constrained environment, try starting node with --max_old_space_size=<n> (where <n> is in megabytes.) The default is 1500 or 1900. -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
