On Sat, Nov 23, 2013 at 2:31 AM, Steve Freegard <[email protected]> wrote:
> On 23/11/13 01:02, Ben Noordhuis wrote:
>> Right. I see that your test case creates 1M file watchers in a loop.
>> Those won't be released right away and calling gc() won't change that.
>> Disposing a file watcher takes two event loop ticks or more but in
>> your test case, everything is done in a single tick. If you smear out
>> the loop iterations with setImmediate(), you should see more
>> consistent memory usage.
>
> Ok - I tried that. I also added a gc() call after the loop, followed by
> a setTimeout() of 60 seconds with a further call to gc() and then set an
> process.on('exit') handler to output process.memoryUsage() so I would
> only get the output once all of the watchers were closed.
>
> I'm still seeing memory leaking:
>
> using fs.watch:
>
> [root@mta41 ~]# node --expose_gc --max_old_space_size=8 watchfile_leak.js
> { rss: 8491008, heapTotal: 6163968, heapUsed: 1919544 }
> { rss: 13017088, heapTotal: 13487360, heapUsed: 1966256 }
>
> using fs.watchFile:
>
> [root@mta41 ~]# node --expose_gc --max_old_space_size=8 watchfile_leak.js
> { rss: 8495104, heapTotal: 6163968, heapUsed: 1919632 }
> { rss: 13275136, heapTotal: 21875968, heapUsed: 1974984 }
>
> This is with 10,000 watchers being added and immediately removed, so the
> process would not exit until the watchers have been terminated.
>
> Based on what we saw in Haraka - there is absolutely no doubt in my mind
> that there is something amiss here. We were incorrectly calling
> fs.unwatchFile() followed by fs.watchFile() thousands of times per
> connection and the memory usage grew massively over a short space of
> time. The moment I changed the code to avoid the calls to
> fs.watchFile/unwatchFile we went from having an RSS of > 1Gb in about an
> hour to around 100Mb.
>
> Cheers,
> Steve.
I can't really reproduce that. Below is what IMO the test case
reasonably should look like and that has RSS hovering around 60 MB
with --max_old_space_size=64. Tested with HEAD of v0.10 and master on
a x86_64 FC20 system.
var fs = require('fs');
var n = 0;
f();
function f() {
if (++n % 1e4 === 0) stats();
fs.unwatchFile('test');
fs.watchFile('test', function() {});
fs.unwatchFile('test');
if (n < 1e6) setImmediate(f);
}
function stats() {
var m = process.memoryUsage();
m.rss >>>= 10, m.heapUsed >>>= 10, m.heapTotal >>>= 10;
console.log(m);
}
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
