[nodejs] Re: Why would a NodeJS app give bad data when it is under stress?

[Zlatko]

>
>
> Any thoughts on why this might happen when the load is high enough? 
>
>
Interesting read! I would love to see that code, this way I can only 
speculate that even though you think your code is all synchronous, it is, 
in fact, not. Perhaps one of the functions or libraries has an implicit 
timeout where it then fails silently?

Or maybe you've exhausted the heap but hit some yet undiscovered V8 heap 
edge case where the it can get mixed up if your contexts get shuffled 
around a lot. I'm thinking - you allocate a certain amount of memory for 
those 4 GB of data, but if you're bombing the API, Node might be holding 
thousands extra sockets open which is taking another chunk of memory that 
you didn't account for - and than one of your scoring functions can't 
allocate enough to work?

Did you test this with always hitting the API with the same example? Does 
it get confused in that case too?

Anyway, I don't know how viable it is, but maybe hooking up Chrome dev 
tools and setting a breakpoint at those filters and searches could shine 
some light.

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/a21852f0-5176-4369-9bba-5d536be6c0f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [nodejs] Why would a NodeJS app give bad data when it is under stress?

[DaneiL]

Here is a better link: Sorry
https://medium.com/netflix-techblog/node-js-in-flames-ddd073803aa4

On 4 May 2017 at 09:28, DaneiL  wrote:

> Hello, i read your entire case, it seems to me, that its not related with
> node itself, but with hapijs.
> Your statement made me reckon a study case made by Netflix.
> That you can see here:
> https://www.infoq.com/news/2014/12/expressjs-burned-netflix
>
> I think it should give you some insights of what is happening.
> That's all
>
>
> On 3 May 2017 at 18:27,  wrote:
>
>> No matter how much stress a NodeJS app is subjected to, it should only
>> offer 2 responses:
>>
>> 1.) the correct data
>>
>> 2.) a SocketTimeout exception
>>
>> A NodeJS app which sends back bad data is useless.
>>
>> I'm trying to find an answer for the problem described here:
>>
>> http://www.smashcompany.com/technology/a-surprising-nodejs-
>> failure-mode-deterministic-code-becomes-probabilistic-under-load
>>
>>
>> Any thoughts on why this might happen when the load is high enough?
>>
>>
>>
>>
>>
>> --
>> Job board: http://jobs.nodejs.org/
>> New group rules: https://gist.github.com/othiym
>> 23/9886289#file-moderation-policy-md
>> Old group rules: https://github.com/joyent/node
>> /wiki/Mailing-List-Posting-Guidelines
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to nodejs+unsubscr...@googlegroups.com.
>> To post to this group, send email to nodejs@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/nodejs/a02a9f1f-ef00-4049-8669-b70a00bf9457%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> []'s
>



-- 
[]'s

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/CAET7tnYh6vCkoPd_nMecQuu5JSaWfBqcJLmJERNCu93ihP-YkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [nodejs] Why would a NodeJS app give bad data when it is under stress?

[DaneiL]

Hello, i read your entire case, it seems to me, that its not related with
node itself, but with hapijs.
Your statement made me reckon a study case made by Netflix.
That you can see here:
https://www.infoq.com/news/2014/12/expressjs-burned-netflix

I think it should give you some insights of what is happening.
That's all


On 3 May 2017 at 18:27,  wrote:

> No matter how much stress a NodeJS app is subjected to, it should only
> offer 2 responses:
>
> 1.) the correct data
>
> 2.) a SocketTimeout exception
>
> A NodeJS app which sends back bad data is useless.
>
> I'm trying to find an answer for the problem described here:
>
> http://www.smashcompany.com/technology/a-surprising-nodejs-failure-mode-
> deterministic-code-becomes-probabilistic-under-load
>
>
> Any thoughts on why this might happen when the load is high enough?
>
>
>
>
>
> --
> Job board: http://jobs.nodejs.org/
> New group rules: https://gist.github.com/othiym23/9886289#file-
> moderation-policy-md
> Old group rules: https://github.com/joyent/node/wiki/Mailing-List-
> Posting-Guidelines
> ---
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nodejs+unsubscr...@googlegroups.com.
> To post to this group, send email to nodejs@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/nodejs/a02a9f1f-ef00-4049-8669-b70a00bf9457%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
[]'s

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/CAET7tnbkcE5rsS7-NB_JHOYchsedfPxxbhB%3D4BNXPPvgOZJB4Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [nodejs] Why would a NodeJS app give bad data when it is under stress?

[Witold Szczerba]

So you have a bug in your application. That is sad, but the "My app
misbehaves, NodeJS is guilty!" grief, 3 pages long won't help here. There
is no source code attached.

Did you actually try to figure out what is wrong? Do you have any lead,
anything?

On Wed, May 3, 2017 at 11:27 PM,  wrote:

> No matter how much stress a NodeJS app is subjected to, it should only
> offer 2 responses:
>
> 1.) the correct data
>
> 2.) a SocketTimeout exception
>
> A NodeJS app which sends back bad data is useless.
>
> I'm trying to find an answer for the problem described here:
>
> http://www.smashcompany.com/technology/a-surprising-nodejs-failure-mode-
> deterministic-code-becomes-probabilistic-under-load
>
>
> Any thoughts on why this might happen when the load is high enough?
>
>
>
>
>
> --
> Job board: http://jobs.nodejs.org/
> New group rules: https://gist.github.com/othiym23/9886289#file-
> moderation-policy-md
> Old group rules: https://github.com/joyent/node/wiki/Mailing-List-
> Posting-Guidelines
> ---
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nodejs+unsubscr...@googlegroups.com.
> To post to this group, send email to nodejs@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/nodejs/a02a9f1f-ef00-4049-8669-b70a00bf9457%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/CAAWTntjW-VR3AmzyqjuStRcm9Pc7hkhj_h8JP_Q53aDHfM51TA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[nodejs] Re: Export variable - ExpressJS

[Ethan Garofolo]

There are a couple of ways around this.  I don't know your familiarity with 
Node.js, so if something doesn't make sense, please ask follow up questions.

The skinny of it is, whatever needs to use that macAddress variables needs 
to wait for it to come back.  There's really no way around that.  So rather 
than trying to get the macAddress value available *outside* of your 
callback there, whatever needs to use the macAddress would need to use it 
inside of that callback (as you observed in your "In the function" console 
statement).

Another option would be to use a promise.  Are you familiar with those? 
 You'd end up with something like:

macAddressPromise = new Promise(function (resolve, reject) {
  macGetting.getMac(function (err, macAddress) {
if (err) return reject(err)

return resolve(macAddress)
  })
})

macAddressPromise is now a variable that you can pass around to wherever. 
 To then actually get the address itself instead of a Promise for the 
address, you'd just call:

macAddressPromise.then(function (macAddress) {
  // inside of here you have access to the actual address
})

Am I explaining this well?


On Wednesday, May 3, 2017 at 3:55:59 PM UTC-6, Peter B. wrote:
>
> Hi everyone,
>
> I need your help. I'm working on a project and I have a problem with the 
> fact that nodejs is asynchronous.
> I want to export my variable called 'macAd'. But even with a global 
> variable, I can't do it. 
>
> Can you help me please ?
>
> Here is my code (I use ExpressJS Framework) :
>
>
> 
>
>
>
>
>
>
>
>
>
>
>
>
> I have this output : 
>
>
> 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Thank you.
>

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/14f5a1e6-d477-4345-b9d3-4051c75daa27%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nodejs] Export data to PDF / DOCX

[Deep SiderZ]

Hi everyone,

I'm trying to export data to PDF / DOCX files. At present I use a HTML/CSS 
template that I fill with my data with ejs package. Then when this template 
is ready I use Libre Office (with soffice command in headless mode) to 
export it to PDF or DOCX format. It works but the result is ugly ...
I would like to be able to have a beautiful result. I want to put pictures, 
tables, colorized texts in the output format.

Do you have an idea on how to achieve this ? It can be a library (one for 
PDF and one for DOCX or better, juste one for both) or a program that I can 
exec from my nodejs app. Or it can also be a solution in another language 
than javascript.

Thank you for your help !

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/feedc87b-427a-4807-bb31-fe78809f10db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nodejs] Re: Export variable - ExpressJS

[Zlatko]

On Wednesday, May 3, 2017 at 11:55:59 PM UTC+2, Peter B. wrote:
>
>
> I need your help. I'm working on a project and I have a problem with the 
> fact that nodejs is asynchronous.
> I want to export my variable called 'macAd'. But even with a global 
> variable, I can't do it. 
>


If I understood correctly, you want to export macAd? But later you don't 
see this variable? Well, it might take a few words to explain, but it's a 
simple thing really.
The problem is if you do something like:

let macAd; // *undefined*
module.exports  = macAd; // again, *undefined*
macAd = 3; // doesn't matter now, because the exports is bound to 
*undefined* already

if you require it later, you'll get *undefined*, not a reference to your 
variable. It's passed by value.

But if you export an object instead, you can modify that object later (say, 
by setting a *macAd* property), without changing the reference to the 
object - which means whoever required the module will get to resolve the 
value after you set it.

Have your module set up like this:

const macAd = require('macad')
const obj = {
  macAd: undefined,
};
module.exports = obj;
// Now the export has a firm grip on our object. EVen if the value is 
undefined itself, the object is there and it'll be
// useful later.
console.log(obj); // -> { macAd: undefined }; 

macAd.getMac(function (err, mac) {
  // obviously handle error first.
  obj.macAd = mac;
  console.log(obj); // -> { macAd: '::1' };
});




Then, when you require the module later, at first it's going to be 
undefined, but later it should resolve to your value. E.g.

const myModule = require('./whatever-you-named-it');
const TIMEOUT = 1000; // or however long the 'macad' module takes.
console.log(myModule.macAd); // -> *undefined*
setTimeout(function () {
  console.log(myModule.macAd); // -> *should give you ::1 or something.*
}, TIMEOUT);

Now you have a problem though, your module is async, but you're not sure 
when is it complete and when it's safe to use it. That TIMEOUT might be 
either too much for you or maybe too short if the computer is busy.

You can get around that by wrapping the *getMac* call in an initializer 
function, like this:

const macAd = require('macad')
const obj = {
  macAd: undefined,
  initialize: function () {
// I use a promise but yuou can go with a callback too
return new Promise(function(resolve, reject) { 
  macAd.getMac(function (err, mac) {
if (err) {
  return reject(err);
obj.macAd = mac;
resolve();
  });
});
  }
};
module.exports = obj;
console.log(obj); // { macAd: undefined, initialize: [Function] }

Now you can require a module, and call *initialize* on it, and you'll get a 
promise resolved (or rejected) when the *macAd* is available, like this:

const myModule = require('myModule');
console.log(myModule.macAd); // most likely *undefined*
myModule.initialize()
.then(function() {
  // now myModule.macAd is available.
  console.log(myModule.macAd); // ::1
})
.catch(console.error);

But here we have other issues. For one, you might simply call getMac 
directly elsewhere like this, there's no direct benefit in wrapping it in 
its own module. And another thing - every time you call your module, the 
getMac is going to be resolving. In this particular case it might not be a 
big issue, but what if it wasn't a *getMac*? What if it was something 
calling an expensive 3rd party service? You call it every time you call 
this.

That's why you basically store a reference to that promise, and only call 
the full init once - every other time you return what you already have.

const macAd = require('macad')
let initPromise; // by keeping it out of *obj*, you're keeping it private 
to the module itself.
const obj = {
  macAd: undefined,
  initialize: function () {
// first check if another module already called init - then just 
resolve what we have, they don't have to wait for *getMac*
if (initPromise) {
  return initPromise;
}
// if it *wasn't* already called, call macAd now and store the 
resulting promise in initPromise.
initPromise = new Promise(function (resolve, reject) {
  macAd.getMac(function (err, mac) {
if (err) {
  return reject(err);
}
obj.macAd = mac;
  });
});
return initPromise;
  }
};
module.exports = obj;

Hope this helps you out a bit.

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 

[nodejs] Searching for users using node.js from ActiveDirectory

[Nataraj Gandhi arunachalam]

Dear Team,

I am trying to use activedirectory nodemodule to search for users from the 
Active Directory using the method, findUsers.

If i search for users using the method, findUsers, the following two 
queries work,

//var query = 'givenName=Nataraj Gandhi'; 
var query = 'sn=Arunachalam'; 

However if I try to write a AND condition for both givenName and sn, it 
doesnt work. The following is the query I am trying to execute,

var query = 'givenName="Nataraj Gandhi", sn="Arunachalam"'; 

Kindly advise how I can write an AND query for searching with both 
givenName and sn.


Thank you very much!

Thanks and Regards,
Nataraj



-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/27e895da-ffc3-4953-8491-bbf3f6c2e581%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nodejs] Re: How to Improve Node .js performance

[lawrence . krubner]

I asked a related question but got no response:

https://groups.google.com/forum/#!topic/nodejs/03jxmdnctPw

In my case I was focused on the error case. 

Here is one thing that helps: 

proxy_read_timeout  

proxy_connect_timeout 

See here:

https://serverfault.com/questions/287102/nginx-proxy-read-timeout-vs-proxy-connect-timeout

I'm trying to figure out what can postpone the moment when NodeJS fails 
completely. 

This doesn't increase the performance of the NodeJS app if you measure 
performance internally to the app. But it does increase the practical 
performance of the app, if you measure from the outside. That is, we can 
forestall the moment when NodeJS dies completely.

One thing I've noticed, which is very worrying, is that once a NodeJS app 
runs out of memory it can take many minutes before it actually dies. 

I have NodeJS running under the control of the daemon Supervisord, with 
this setup for restarts:

[program:dupe_res_v7]
command=/usr/local/bin/node   --max_old_space_size=6000   
/home/ec2-user/daemons/deduplication_api/v7/dupe-res/server.js
autostart=true
autorestart=true
startretries=100
numprocs=1
stderr_logfile=/var/log/dupe_res_stderr_v7.log
stdout_logfile=/var/log/dupe_res_stdout_v7.log
stdout_logfile_maxbytes=50MB
stderr_logfile_maxbytes=50MB


At some users (my fellow co-workers) wrote to me and said "Hey, the API is 
non-responsive." I looked into. The app would run out of money and freeze, 
sometimes for as long as 30 minutes, before it would die and be restarted 
by Supervisord. 

I've learned that aggressive health checks are necessary to kill the app 
sooner, so that Supervisord can restart it sooner. 

 lawrence






On Tuesday, April 18, 2017 at 8:42:15 PM UTC-4, Patrick Hastings wrote:
>
> What exactly is the media you are sending to the client?
> For example, is it:
>
>- HTML pages
>- Video content
>- picture content
>- music
>- other (explain)
>
> It also might help to know *where* the data is coming from; are you 
> querying a MySQL or RDBMS database? Or again are these just flat web pages 
> you are serving up?
>
>
> On Sunday, April 9, 2017 at 3:02:00 PM UTC-4, Amit Roman wrote:
>>
>> I am using Node JS for communication through socket and http to many 
>> client.
>> Many number of client is connected to my node js. 
>> My Node js sending data coming from hardware to connected client.
>> I want to improve the performance of Node js.
>> Please guide me.
>>
>

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/880ee29f-a119-4883-a993-769a7d7ee376%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.