[nodejs] Re: How common is it to restart a NodeJS app?

[Zlatko]

On Sunday, May 7, 2017 at 12:32:15 AM UTC+2, lawrence...@gmail.com wrote:
>
> Do you keep track of how often your app dies and is restarted? 
>
>
Nope, not directly. But I do scan the logs from time to time to check if 
there's something unusual (including restarts).

 

> Of the die/restart cycle, how much is deliberate on your part (an Erlang 
> style fail fast philosophy) versus how much is s a total surprise? 
>

I try to make any restart explicitly because of a new deploy. If my app is 
restarting on its own occasionally, it's a bug and I try to find it and fix 
it f I see it.
 


> For those of you who have apps that die often, what are the circumstances 
> that cause it to die often? 
>
>
Usually bugs. An uncaught error in init, somewhere where I rely on a 
third-party site and it's module, but those break and I didn't try/catch 
this (because I didn't read the docs and saw that they might, or because 
they didn't document it at all), things like that.

HTH.

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/20bcccde-86c8-4e52-9646-5a4202dfeef7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nodejs] Re: How to Improve Node .js performance

[Zlatko]

On Sunday, May 7, 2017 at 12:32:08 AM UTC+2, lawrence...@gmail.com wrote:
>
> I occasionally see things like this in the log, but this simply kills the 
> app. This does not return nicely formatted JSON, which just happens to lack 
> the right data. 
>
> It does, however, stall, for a very long time, sometimes 20 or 30 minutes. 
>
> [...]
>

This part looks like a simple case of a memory leak. Your code (or some 
library you use) is creating objects, but not releasing them, and at a 
certain point it time, your process is out of memory. 

That is relatively simple to debug. Install the heapdump module 
(https://www.npmjs.com/package/heapdump or something similar), and then 
dump the memory maybe once each hour. Something like this in your main 
script.

const heapdump = require('heapdump');
setTimeout(function() {
  heapdump.writeSnapshot(`/tmp/${process.env.pid}${Date.now()}.heapsnapshot`);
}, 60 * 60 * 1000);
// ... the rest of your code.


Then load up the dumps into Chrome's Javascript console (Dev Console -> 
Profiles) and see what kind of objects are you accumulating or not 
releasing).

(Make sure you have enough disk space for all the heap dumps. Also, 
depending how often your app gets stuck, you can dump more or less 
frequently, this is just a basic example.)

On the other hand, it might not be the memory leak, maybe your app is 
specific and simply needs more memory to function properly under load. But 
I'd bet on the mem leak.


-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/91bf6f7c-8c65-4a4f-a4d9-c58e992761f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [nodejs] Export data to PDF / DOCX

[Witold Szczerba]

P.S. When I said creating the templates in DOCX, I meant using the
LibreOffice. Since it is going to be used to render, you will see exactly
how it is going to look like.  As a bonus, non technical stuff can create
and maintain these templates.

07.05.2017 11:47 "Witold Szczerba"  napisał(a):

> I would try to create these templates in. DOCX instead of HTML. That would
> give you much better control. The DOCX format is just a ZIP file with XML
> inside, so manipulating the content should not be a huge problem.
>
> 05.05.2017 03:53 "Deep SiderZ"  napisał(a):
>
>> Hi everyone,
>>
>> I'm trying to export data to PDF / DOCX files. At present I use a
>> HTML/CSS template that I fill with my data with ejs package. Then when this
>> template is ready I use Libre Office (with soffice command in headless
>> mode) to export it to PDF or DOCX format. It works but the result is ugly
>> ...
>> I would like to be able to have a beautiful result. I want to put
>> pictures, tables, colorized texts in the output format.
>>
>> Do you have an idea on how to achieve this ? It can be a library (one for
>> PDF and one for DOCX or better, juste one for both) or a program that I can
>> exec from my nodejs app. Or it can also be a solution in another language
>> than javascript.
>>
>> Thank you for your help !
>>
>> --
>> Job board: http://jobs.nodejs.org/
>> New group rules: https://gist.github.com/othiym
>> 23/9886289#file-moderation-policy-md
>> Old group rules: https://github.com/joyent/node
>> /wiki/Mailing-List-Posting-Guidelines
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to nodejs+unsubscr...@googlegroups.com.
>> To post to this group, send email to nodejs@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/nodejs/feedc87b-427a-4807-bb31-fe78809f10db%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/CAAWTnth1VVc2nNsyyW2atUOojv1m%3DOz1SVueBk1KQfUuqnbY5w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [nodejs] Why would a NodeJS app give bad data when it is under stress?

[Juraj Kirchheim]

You should really try to isolate the error.

1. Run the function from within a simple node script and see if it returns
the same results consistently. If not, investigate that.
2. Replace the function with something trivial with a busy loop to get
roughly the same runtime duration. If you do get the same non-determinism,
then the error is somewhere up the stack. If not, it's gonna be tough,
because after step one you can be fairly certain that your function is
deterministic, so all you can tell is that it probably causes some
side-effect that is relevant to the stack above. Like you might be trashing
a lot of memory or something and that causes hapi to behave strangely. In
this case I would advice to add back more and more code of the old function
until the non-determinism shows again and act on what you have learned from
that.
3. Assuming that a busy waiting function shows the same non-determinism,
the problem would have to be either in node or in hapi. To test that, your
best bet is to serve the functionality without any kind of framework.
Because you have only one call, it shouldn't be hard to do. You will find
out that either node is the problem or that it isn't. The next step would
be to report that to your client and also raise an issue with whoever is
responsible for the failure by giving them a minimal example which you
should have by this point. Then you should make sure to configure nginx in
a way that the concurrency never reaches a level that the node app can't
handle and start giving back 503 or something. That way you don't serve
incorrect responses.

Best,
Juraj


On Fri, May 5, 2017 at 6:57 PM,  wrote:

> Danizavtz, that is an interesting read. Thank you for sharing it. But
> again, because I deliberately bombard the app with more than it can handle,
> I am expecting it to fail often. And when it fails, I expect to get a
> SocketTimeout exception, or some other kind of timeout error. So it seems
> reasonable that the app becomes overwhelmed and then fails -- that is
> exactly what I am trying to do. But again, I expect only one of two
> responses:
>
> 1.) the app sends the correct data formatted as JSON
>
> 2.) the app throws an exception
>
> What doesn't make sense to me is:
>
> 3.) the app sends correctly formatted JSON, but the actual data from the
> memory cache is missing
>
>
>
>
> On Thursday, May 4, 2017 at 9:53:24 PM UTC-4, Danizavtz wrote:
>>
>> Here is a better link: Sorry
>> https://medium.com/netflix-techblog/node-js-in-flames-ddd073803aa4
>>
>> On 4 May 2017 at 09:28, DaneiL  wrote:
>>
>>> Hello, i read your entire case, it seems to me, that its not related
>>> with node itself, but with hapijs.
>>> Your statement made me reckon a study case made by Netflix.
>>> That you can see here:
>>> https://www.infoq.com/news/2014/12/expressjs-burned-netflix
>>>
>>> I think it should give you some insights of what is happening.
>>> That's all
>>>
>>>
>>> On 3 May 2017 at 18:27,  wrote:
>>>
 No matter how much stress a NodeJS app is subjected to, it should only
 offer 2 responses:

 1.) the correct data

 2.) a SocketTimeout exception

 A NodeJS app which sends back bad data is useless.

 I'm trying to find an answer for the problem described here:

 http://www.smashcompany.com/technology/a-surprising-nodejs-
 failure-mode-deterministic-code-becomes-probabilistic-under-load


 Any thoughts on why this might happen when the load is high enough?





 --
 Job board: http://jobs.nodejs.org/
 New group rules: https://gist.github.com/othiym
 23/9886289#file-moderation-policy-md
 Old group rules: https://github.com/joyent/node
 /wiki/Mailing-List-Posting-Guidelines
 ---
 You received this message because you are subscribed to the Google
 Groups "nodejs" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to nodejs+un...@googlegroups.com.
 To post to this group, send email to nod...@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/ms
 gid/nodejs/a02a9f1f-ef00-4049-8669-b70a00bf9457%40googlegroups.com
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>>
>>> --
>>> []'s
>>>
>>
>>
>>
>> --
>> []'s
>>
> --
> Job board: http://jobs.nodejs.org/
> New group rules: https://gist.github.com/othiym23/9886289#file-
> moderation-policy-md
> Old group rules: https://github.com/joyent/node/wiki/Mailing-List-
> Posting-Guidelines
> ---
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nodejs+unsubscr...@googlegroups.com.
> To post to 

Re: [nodejs] Export data to PDF / DOCX

[Witold Szczerba]

I would try to create these templates in. DOCX instead of HTML. That would
give you much better control. The DOCX format is just a ZIP file with XML
inside, so manipulating the content should not be a huge problem.

05.05.2017 03:53 "Deep SiderZ"  napisał(a):

> Hi everyone,
>
> I'm trying to export data to PDF / DOCX files. At present I use a HTML/CSS
> template that I fill with my data with ejs package. Then when this template
> is ready I use Libre Office (with soffice command in headless mode) to
> export it to PDF or DOCX format. It works but the result is ugly ...
> I would like to be able to have a beautiful result. I want to put
> pictures, tables, colorized texts in the output format.
>
> Do you have an idea on how to achieve this ? It can be a library (one for
> PDF and one for DOCX or better, juste one for both) or a program that I can
> exec from my nodejs app. Or it can also be a solution in another language
> than javascript.
>
> Thank you for your help !
>
> --
> Job board: http://jobs.nodejs.org/
> New group rules: https://gist.github.com/othiym23/9886289#file-
> moderation-policy-md
> Old group rules: https://github.com/joyent/node/wiki/Mailing-List-
> Posting-Guidelines
> ---
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nodejs+unsubscr...@googlegroups.com.
> To post to this group, send email to nodejs@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/nodejs/feedc87b-427a-4807-bb31-fe78809f10db%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/CAAWTntgyWPc-nsm1MP-VNP4wNdDBQsL66RLg3xVUrfsWiFrTWw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.