Mark - you've had me contemplating this all night, and making me
wonder whether the leak we're seeing is due to how I'm doing things.
Let me elaborate. So I've extended the basic Node web server pattern:
http.createServer(function (req, res) {
// create content....
res.writeHead(200, {'Content-Type': 'text/html'});
res.write(content);
res.end();
}).listen(port, ipAddress);
but since the content is dynamically generated in a child process, I
add the request and response objects to a queue, in effect:
http.createServer(function (req, res) {
// create content....
addToQueue(req,res);
}).listen(port, ipAddress);
Adding to the queue triggers an event to process the queue, and the
request is dispatched to the first available child process. Back
comes the generated response content, triggering "data" events in my
handler for the childProcess.stdout. When a terminating sequence is
detected, the response is sent back to the browser that requested it.
This is done via an index value that is returned by the child process
along with the content payload, the index points to the queuedRequest
object which contains the response object. I then dispatch the
response, using:
queuedRequest.response.writeHead(httpStatus, headers);
queuedRequest.response.write(bodyStr);
queuedRequest.response.end();
This is invoked from within my child process "data" event handler.
>From a functional point of view this works just fine and node-
inspector confirms that everything within my logic clears itself
down. But I'm now wondering whether this pattern is leaving behind
memory structures in the background, since the createServer() callback
is terminating after simply placing the request and response onto my
queue.
Perhaps the correct pattern should be that when my child process
"data" event handler has assembled the response and identified the
correct response object, it should trigger a "received" event that is
then handled within the createServer() callback, eg something like:
http.createServer(function (req, res) {
// create content....
addToQueue(req,res);
res.on("received",function(content,httpStatus,headers) {
res.writeHead(httpStatus, headers);
res.write(bodyStr);
res.end();
};
}).listen(port, ipAddress);
So could my current logic be the source of the leak, and will this
latter pattern address it, or does anyone have any other suggestions?
Many thanks in advance
Rob
On Feb 23, 6:52 pm, Mark Hahn <[email protected]> wrote:
> > BTW, I assume the response.end() should clear down any internal memory
>
> structures that hold the response payload?
>
> I would assume the structures disappear only when request and response go
> out of scope.
>
>
>
>
>
>
>
> On Thu, Feb 23, 2012 at 5:03 AM, rtweed <[email protected]> wrote:
> > One thing to add to Chris's description of the problem (I'm working
> > with him on it):
>
> > - the child processes stay alive (or are supposed to stay alive)
> > indefinitely, with the Node process triggering "data" events coming
> > from them on STDOUT. No "end" events therefore occur under normal
> > circumstances. My theory is that the internal memory buffer used for
> > the child process STDOUT communication stream isn't getting cleared
> > down. Could this be the case? Ideally you'd expect this to happen at
> > each "data" event and not just on an "end" event.
>
> > I don't know whether this is a Windows-specific issue.
>
> > To amplify Chris's last posting, the page content is created
> > dynamically within the child processes. The Node process simply picks
> > up the incoming content on STDOUT and forwards it to the appropriate
> > browser using the usual pattern of:
>
> > response.writeHead(httpStatus, headers);
> > response.write(bodyStr);
> > response.end();
>
> > So there's no retention or cacheing of the response payload in the
> > Node process, and inspection of the objects we're using in our logic
> > (using node-inspector) appears to show that they are clearing down and
> > are being managed correctly.
>
> > BTW, I assume the response.end() should clear down any internal memory
> > structures that hold the response payload?
>
> > So I'd agree with Chris - it sounds more like it's perhaps the child
> > process STDOUT memory structures that are the source of the leakage.
>
> > Rob
>
> > On Feb 23, 11:52 am, Chris Casey <[email protected]> wrote:
> > > It is a custom webserver and we have built no caching as all pages served
> > > are dynamic.
> > > Pretty much convinced though that the problem is in the pages fetched
> > from
> > > the child processes rather than the push to the browser as closing the
> > > child processes releases the memory.
>
> > > Chris
>
> > --
> > Job Board:http://jobs.nodejs.org/
> > Posting guidelines:
> >https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> > You received this message because you are subscribed to the Google
> > Groups "nodejs" group.
> > To post to this group, send email to [email protected]
> > To unsubscribe from this group, send email to
> > [email protected]
> > For more options, visit this group at
> >http://groups.google.com/group/nodejs?hl=en?hl=en
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
