On 23/11/13 01:02, Ben Noordhuis wrote:
> Right. I see that your test case creates 1M file watchers in a loop.
> Those won't be released right away and calling gc() won't change that.
> Disposing a file watcher takes two event loop ticks or more but in
> your test case, everything is done in a single tick. If you smear out
> the loop iterations with setImmediate(), you should see more
> consistent memory usage.
Ok - I tried that. I also added a gc() call after the loop, followed by
a setTimeout() of 60 seconds with a further call to gc() and then set an
process.on('exit') handler to output process.memoryUsage() so I would
only get the output once all of the watchers were closed.
I'm still seeing memory leaking:
using fs.watch:
[root@mta41 ~]# node --expose_gc --max_old_space_size=8 watchfile_leak.js
{ rss: 8491008, heapTotal: 6163968, heapUsed: 1919544 }
{ rss: 13017088, heapTotal: 13487360, heapUsed: 1966256 }
using fs.watchFile:
[root@mta41 ~]# node --expose_gc --max_old_space_size=8 watchfile_leak.js
{ rss: 8495104, heapTotal: 6163968, heapUsed: 1919632 }
{ rss: 13275136, heapTotal: 21875968, heapUsed: 1974984 }
This is with 10,000 watchers being added and immediately removed, so the
process would not exit until the watchers have been terminated.
Based on what we saw in Haraka - there is absolutely no doubt in my mind
that there is something amiss here. We were incorrectly calling
fs.unwatchFile() followed by fs.watchFile() thousands of times per
connection and the memory usage grew massively over a short space of
time. The moment I changed the code to avoid the calls to
fs.watchFile/unwatchFile we went from having an RSS of > 1Gb in about an
hour to around 100Mb.
Cheers,
Steve.
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
