You should really try to isolate the error. 1. Run the function from within a simple node script and see if it returns the same results consistently. If not, investigate that. 2. Replace the function with something trivial with a busy loop to get roughly the same runtime duration. If you do get the same non-determinism, then the error is somewhere up the stack. If not, it's gonna be tough, because after step one you can be fairly certain that your function is deterministic, so all you can tell is that it probably causes some side-effect that is relevant to the stack above. Like you might be trashing a lot of memory or something and that causes hapi to behave strangely. In this case I would advice to add back more and more code of the old function until the non-determinism shows again and act on what you have learned from that. 3. Assuming that a busy waiting function shows the same non-determinism, the problem would have to be either in node or in hapi. To test that, your best bet is to serve the functionality without any kind of framework. Because you have only one call, it shouldn't be hard to do. You will find out that either node is the problem or that it isn't. The next step would be to report that to your client and also raise an issue with whoever is responsible for the failure by giving them a minimal example which you should have by this point. Then you should make sure to configure nginx in a way that the concurrency never reaches a level that the node app can't handle and start giving back 503 or something. That way you don't serve incorrect responses.
Best, Juraj On Fri, May 5, 2017 at 6:57 PM, <[email protected]> wrote: > Danizavtz, that is an interesting read. Thank you for sharing it. But > again, because I deliberately bombard the app with more than it can handle, > I am expecting it to fail often. And when it fails, I expect to get a > SocketTimeout exception, or some other kind of timeout error. So it seems > reasonable that the app becomes overwhelmed and then fails -- that is > exactly what I am trying to do. But again, I expect only one of two > responses: > > 1.) the app sends the correct data formatted as JSON > > 2.) the app throws an exception > > What doesn't make sense to me is: > > 3.) the app sends correctly formatted JSON, but the actual data from the > memory cache is missing > > > > > On Thursday, May 4, 2017 at 9:53:24 PM UTC-4, Danizavtz wrote: >> >> Here is a better link: Sorry >> https://medium.com/netflix-techblog/node-js-in-flames-ddd073803aa4 >> >> On 4 May 2017 at 09:28, DaneiL <[email protected]> wrote: >> >>> Hello, i read your entire case, it seems to me, that its not related >>> with node itself, but with hapijs. >>> Your statement made me reckon a study case made by Netflix. >>> That you can see here: >>> https://www.infoq.com/news/2014/12/expressjs-burned-netflix >>> >>> I think it should give you some insights of what is happening. >>> That's all >>> >>> >>> On 3 May 2017 at 18:27, <[email protected]> wrote: >>> >>>> No matter how much stress a NodeJS app is subjected to, it should only >>>> offer 2 responses: >>>> >>>> 1.) the correct data >>>> >>>> 2.) a SocketTimeout exception >>>> >>>> A NodeJS app which sends back bad data is useless. >>>> >>>> I'm trying to find an answer for the problem described here: >>>> >>>> http://www.smashcompany.com/technology/a-surprising-nodejs- >>>> failure-mode-deterministic-code-becomes-probabilistic-under-load >>>> >>>> >>>> Any thoughts on why this might happen when the load is high enough? >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Job board: http://jobs.nodejs.org/ >>>> New group rules: https://gist.github.com/othiym >>>> 23/9886289#file-moderation-policy-md >>>> Old group rules: https://github.com/joyent/node >>>> /wiki/Mailing-List-Posting-Guidelines >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "nodejs" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> To view this discussion on the web visit https://groups.google.com/d/ms >>>> gid/nodejs/a02a9f1f-ef00-4049-8669-b70a00bf9457%40googlegroups.com >>>> <https://groups.google.com/d/msgid/nodejs/a02a9f1f-ef00-4049-8669-b70a00bf9457%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> []'s >>> >> >> >> >> -- >> []'s >> > -- > Job board: http://jobs.nodejs.org/ > New group rules: https://gist.github.com/othiym23/9886289#file- > moderation-policy-md > Old group rules: https://github.com/joyent/node/wiki/Mailing-List- > Posting-Guidelines > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/nodejs/c7353e3f-d936-4808-a133-5eebaf1a9723%40googlegroups.com > <https://groups.google.com/d/msgid/nodejs/c7353e3f-d936-4808-a133-5eebaf1a9723%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Job board: http://jobs.nodejs.org/ New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAJHzicZxQLBC6Nm2Bwq0%2BcnGOnGJcrsih47BGqxH4oocciaJyg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
