> To get a very rough idea of how much time your program spends in gc, you can > expose the gc by running node with the --nouse_idle_notification --expose_gc > flags and then call gc manually with "global.gc()" and time how long that > takes.
This will give you something that is _far_ from a realistic estimate. First of all contrary to want you might think --nouse-idle-notification does not disable automatic GC in V8. What it does is tells V8 not to perform GC actions (be it advance the sweeper, incremental marker or do a full GC) in response to IdleNotifications that embedder (node.js in this case) sends to V8. If V8 sees fit (e.g. on allocation failure) it _will_ perform it and you can't disable that. Calling gc() force a full non-incremental collection which is much more expensive than (and is quite different from) from incremental collections that V8 tries to use during the normal run of your program. Now about large object with a few million entries. The biggest problem here is that you stash all your entries into a single object, the biggest problem will be spending time scanning that object during GC (V8 does split marking into step, but it does not split scanning of a single object into steps). If you allocate many objects it should not be a problem for V8's incremental GC: here is an excerpt from a --trace-gc output for a small stupid-stupid test I've wrote (https://gist.github.com/3104414): 100000 keys created (total keys: 20800000) 111188 ms: Mark-sweep 4126.8 (4174.9) -> 3978.6 (4042.0) MB, 89 ms (+ 5605 ms in 2232 steps since start of marking, biggest step 22.033936 ms) [StackGuard GC request] [GC in old space requested]. As you can see there is no one huge pause. The work is done in increments. Of course if you can avoid pauses altogether by allocating your huge chunks of memory outside of GC managed heap it's even better. My personal position here is that anybody who allocates 4GBs of GC managed objects is doing something wrong. [of course I am not claiming that there is nothing to improve here on V8 side: marking, sweeping and evacuation steps can be optimized further and even parallelized internally; some portions can be even trivially made concurrent]. [@Ben: it is not as sophisticated as a concurrent soft-real time collector would be, hard to dispute that. It is however much more sophisticated than a straightforward mark-sweep would be :-) I still think that the best fit for node.js might be a combination of GC with a region based memory management] -- Vyacheslav Egorov On Fri, Jul 13, 2012 at 9:02 AM, Joran Greef <[email protected]> wrote: > To get a very rough idea of how much time your program spends in gc, you can > expose the gc by running node with the --nouse_idle_notification --expose_gc > flags and then call gc manually with "global.gc()" and time how long that > takes. > > I have a program where a single JS object was being used as a hash table > with a few million entries. When the gc would run, as far as I am aware, it > would iterate over every one of those entries and it would do this every few > seconds, each time pausing for about 500ms. I switched to a Buffer backed > open addressed hash table which is also more efficient in other respects. > > > On Friday, July 13, 2012 3:10:50 AM UTC+2, Alexey Petrushin wrote: >> >> There are rumors that current Node.js (or, more exactly V8 GC) performs >> badly when there are lots of JS objects and memory used. >> >> Can You please explain what exatly is the problem - lots of objects or >> lots of properties on one object (or array)? >> >> Maybe there are some benchmarks, would be interesting to see actual code >> and numbers. >> >> As far as I know the main problem - lots of properties on one object, not >> lots of objects itself (although I'm not sure). If so - would be the >> in-memory graph database (about couple of hundreds of properties on each >> node at max) a good case? >> >> Also I heard that latest versions of V8 has improved GC and that it solved >> some parts of this problems - is this true, and when it will be available in >> Node.js? > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
