Vyacheslav, to clarify, the suggestion of timing gc() was made in the context of a program with one large object containing a few million entries, in which case that would give a "very rough idea". In that case, there would be little difference between full collection and incremental collection, since the majority of work would be indivisible.
On Friday, July 13, 2012 1:37:17 PM UTC+2, Vyacheslav Egorov wrote: > > > To get a very rough idea of how much time your program spends in gc, you > can expose the gc by running node with the --nouse_idle_notification > --expose_gc flags and then call gc manually with "global.gc()" and time how > long that takes. > > This will give you something that is _far_ from a realistic estimate. > First of all contrary to want you might think > --nouse-idle-notification does not disable automatic GC in V8. What it > does is tells V8 not to perform GC actions (be it advance the sweeper, > incremental marker or do a full > GC) in response to IdleNotifications that embedder (node.js in this > case) sends to V8. If V8 sees fit (e.g. on allocation failure) it > _will_ perform it and you can't disable that. Calling gc() force a > full non-incremental collection which is much more expensive than (and > is quite different from) from incremental collections that V8 tries to > use during the normal run of your program. > > Now about large object with a few million entries. The biggest problem > here is that you stash all your entries into a single object, the > biggest problem will be spending time scanning that object during GC > (V8 does split marking into step, but it does not split scanning of a > single object into steps). If you allocate many objects it should not > be a problem for V8's incremental GC: > > here is an excerpt from a --trace-gc output for a small stupid-stupid > test I've wrote (https://gist.github.com/3104414): > > 100000 keys created (total keys: 20800000) > 111188 ms: Mark-sweep 4126.8 (4174.9) -> 3978.6 (4042.0) MB, 89 ms > (+ 5605 ms in 2232 steps since start of marking, biggest step > 22.033936 ms) [StackGuard GC request] [GC in old space requested]. > > As you can see there is no one huge pause. The work is done in increments. > > Of course if you can avoid pauses altogether by allocating your huge > chunks of memory outside of GC managed heap it's even better. My > personal position here is that anybody who allocates 4GBs of GC > managed objects is doing something wrong. > > [of course I am not claiming that there is nothing to improve here on > V8 side: marking, sweeping and evacuation steps can be optimized > further and even parallelized internally; some portions can be even > trivially made concurrent]. > > [@Ben: it is not as sophisticated as a concurrent soft-real time > collector would be, hard to dispute that. It is however much more > sophisticated than a straightforward mark-sweep would be :-) I still > think that the best fit for node.js might be a combination of GC with > a region based memory management] > > -- > Vyacheslav Egorov > > > On Fri, Jul 13, 2012 at 9:02 AM, Joran Greef <[email protected]> wrote: > > To get a very rough idea of how much time your program spends in gc, you > can > > expose the gc by running node with the --nouse_idle_notification > --expose_gc > > flags and then call gc manually with "global.gc()" and time how long > that > > takes. > > > > I have a program where a single JS object was being used as a hash table > > with a few million entries. When the gc would run, as far as I am aware, > it > > would iterate over every one of those entries and it would do this every > few > > seconds, each time pausing for about 500ms. I switched to a Buffer > backed > > open addressed hash table which is also more efficient in other > respects. > > > > > > On Friday, July 13, 2012 3:10:50 AM UTC+2, Alexey Petrushin wrote: > >> > >> There are rumors that current Node.js (or, more exactly V8 GC) performs > >> badly when there are lots of JS objects and memory used. > >> > >> Can You please explain what exatly is the problem - lots of objects or > >> lots of properties on one object (or array)? > >> > >> Maybe there are some benchmarks, would be interesting to see actual > code > >> and numbers. > >> > >> As far as I know the main problem - lots of properties on one object, > not > >> lots of objects itself (although I'm not sure). If so - would be the > >> in-memory graph database (about couple of hundreds of properties on > each > >> node at max) a good case? > >> > >> Also I heard that latest versions of V8 has improved GC and that it > solved > >> some parts of this problems - is this true, and when it will be > available in > >> Node.js? > > > > -- > > Job Board: http://jobs.nodejs.org/ > > Posting guidelines: > > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > > You received this message because you are subscribed to the Google > > Groups "nodejs" group. > > To post to this group, send email to [email protected] > > To unsubscribe from this group, send email to > > [email protected] > > For more options, visit this group at > > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
