[nodejs] Re: How to Improve Node .js performance

[Zlatko]

On Sunday, May 7, 2017 at 12:32:08 AM UTC+2, lawrence...@gmail.com wrote:
>
> I occasionally see things like this in the log, but this simply kills the 
> app. This does not return nicely formatted JSON, which just happens to lack 
> the right data. 
>
> It does, however, stall, for a very long time, sometimes 20 or 30 minutes. 
>
> [...]
>

This part looks like a simple case of a memory leak. Your code (or some 
library you use) is creating objects, but not releasing them, and at a 
certain point it time, your process is out of memory. 

That is relatively simple to debug. Install the heapdump module 
(https://www.npmjs.com/package/heapdump or something similar), and then 
dump the memory maybe once each hour. Something like this in your main 
script.

const heapdump = require('heapdump');
setTimeout(function() {
  heapdump.writeSnapshot(`/tmp/${process.env.pid}${Date.now()}.heapsnapshot`);
}, 60 * 60 * 1000);
// ... the rest of your code.


Then load up the dumps into Chrome's Javascript console (Dev Console -> 
Profiles) and see what kind of objects are you accumulating or not 
releasing).

(Make sure you have enough disk space for all the heap dumps. Also, 
depending how often your app gets stuck, you can dump more or less 
frequently, this is just a basic example.)

On the other hand, it might not be the memory leak, maybe your app is 
specific and simply needs more memory to function properly under load. But 
I'd bet on the mem leak.


-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/91bf6f7c-8c65-4a4f-a4d9-c58e992761f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nodejs] Re: How to Improve Node .js performance

[lawrence . krubner]

I asked a related question but got no response:

https://groups.google.com/forum/#!topic/nodejs/03jxmdnctPw

In my case I was focused on the error case. 

Here is one thing that helps: 

proxy_read_timeout  

proxy_connect_timeout 

See here:

https://serverfault.com/questions/287102/nginx-proxy-read-timeout-vs-proxy-connect-timeout

I'm trying to figure out what can postpone the moment when NodeJS fails 
completely. 

This doesn't increase the performance of the NodeJS app if you measure 
performance internally to the app. But it does increase the practical 
performance of the app, if you measure from the outside. That is, we can 
forestall the moment when NodeJS dies completely.

One thing I've noticed, which is very worrying, is that once a NodeJS app 
runs out of memory it can take many minutes before it actually dies. 

I have NodeJS running under the control of the daemon Supervisord, with 
this setup for restarts:

[program:dupe_res_v7]
command=/usr/local/bin/node   --max_old_space_size=6000   
/home/ec2-user/daemons/deduplication_api/v7/dupe-res/server.js
autostart=true
autorestart=true
startretries=100
numprocs=1
stderr_logfile=/var/log/dupe_res_stderr_v7.log
stdout_logfile=/var/log/dupe_res_stdout_v7.log
stdout_logfile_maxbytes=50MB
stderr_logfile_maxbytes=50MB


At some users (my fellow co-workers) wrote to me and said "Hey, the API is 
non-responsive." I looked into. The app would run out of money and freeze, 
sometimes for as long as 30 minutes, before it would die and be restarted 
by Supervisord. 

I've learned that aggressive health checks are necessary to kill the app 
sooner, so that Supervisord can restart it sooner. 

 lawrence






On Tuesday, April 18, 2017 at 8:42:15 PM UTC-4, Patrick Hastings wrote:
>
> What exactly is the media you are sending to the client?
> For example, is it:
>
>- HTML pages
>- Video content
>- picture content
>- music
>- other (explain)
>
> It also might help to know *where* the data is coming from; are you 
> querying a MySQL or RDBMS database? Or again are these just flat web pages 
> you are serving up?
>
>
> On Sunday, April 9, 2017 at 3:02:00 PM UTC-4, Amit Roman wrote:
>>
>> I am using Node JS for communication through socket and http to many 
>> client.
>> Many number of client is connected to my node js. 
>> My Node js sending data coming from hardware to connected client.
>> I want to improve the performance of Node js.
>> Please guide me.
>>
>

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/880ee29f-a119-4883-a993-769a7d7ee376%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nodejs] Re: How to Improve Node .js performance

[Patrick Hastings]

What exactly is the media you are sending to the client?
For example, is it:

   - HTML pages
   - Video content
   - picture content
   - music
   - other (explain)
   
It also might help to know *where* the data is coming from; are you 
querying a MySQL or RDBMS database? Or again are these just flat web pages 
you are serving up?


On Sunday, April 9, 2017 at 3:02:00 PM UTC-4, Amit Roman wrote:
>
> I am using Node JS for communication through socket and http to many 
> client.
> Many number of client is connected to my node js. 
> My Node js sending data coming from hardware to connected client.
> I want to improve the performance of Node js.
> Please guide me.
>

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/0f333ce0-bc1e-484d-a4a5-3684b071a84e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.