Hello Karim, I'd suggest profiling the application during such spikes. It seems that the app is spinning in JS and doing lots of allocations (because of mmap() calls in strace output).
Cheers, Fedor. On Wed, Dec 17, 2014 at 5:03 PM, Karim Tarek < [email protected]> wrote: > > We are facing a problem with our Node js (v0.10.33) application with > CLUSTER module on ubuntu amazon EC2 (c3.large) instance that nearly > everyday we get a 100% cpu spike (on the workers) suddenly and it never > goes down until we restart our application the problem is that these spikes > happens randomly and at weird times (at 3AM) when the traffic is minimal, > when i looked at the logs i found that the logs and any APMs stopped > recording any requests probably because the node js app stopped working, > what i did based on what i read so far is that i did a strace on the > process which was at 100% and here is a part of the output of my stack > trace (full strace attached): > > gettimeofday({1418314214, 491214}, NULL) = 0 > > gettimeofday({1418314214, 491327}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > gettimeofday({1418314214, 619975}, NULL) = 0 > gettimeofday({1418314214, 620055}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3b000000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b000000 > mprotect(0xfcc3b005000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b006000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b006000 > mprotect(0xfcc3b0ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b100000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b100000 > mprotect(0xfcc3b105000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b106000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b106000 > mprotect(0xfcc3b1ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314214, 731602}, NULL) = 0 > gettimeofday({1418314214, 731682}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3b200000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b200000 > mprotect(0xfcc3b205000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b206000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b206000 > mprotect(0xfcc3b2ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314214, 835071}, NULL) = 0 > gettimeofday({1418314214, 835154}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3b300000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b300000 > mprotect(0xfcc3b305000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b306000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b306000 > mprotect(0xfcc3b3ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b400000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b400000 > mprotect(0xfcc3b405000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b406000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b406000 > mprotect(0xfcc3b4ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314214, 938991}, NULL) = 0 > gettimeofday({1418314214, 939072}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3b500000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b500000 > mprotect(0xfcc3b505000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b506000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b506000 > mprotect(0xfcc3b5ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314215, 41662}, NULL) = 0 > gettimeofday({1418314215, 41744}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3b600000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b600000 > mprotect(0xfcc3b605000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b606000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b606000 > mprotect(0xfcc3b6ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314215, 161306}, NULL) = 0 > gettimeofday({1418314215, 161408}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3b700000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b700000 > mprotect(0xfcc3b705000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b706000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b706000 > mprotect(0xfcc3b7ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b800000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b800000 > mprotect(0xfcc3b805000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b806000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b806000 > mprotect(0xfcc3b8ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314215, 287996}, NULL) = 0 > gettimeofday({1418314215, 288120}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3b900000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b900000 > mprotect(0xfcc3b905000, 4096, PROT_NONE) = 0 > mmap(0xfcc3b906000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3b906000 > mprotect(0xfcc3b9ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314215, 472947}, NULL) = 0 > gettimeofday({1418314215, 473040}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3ba00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3ba00000 > mprotect(0xfcc3ba05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3ba06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3ba06000 > mprotect(0xfcc3baff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3bb00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3bb00000 > mprotect(0xfcc3bb05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3bb06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3bb06000 > mprotect(0xfcc3bbff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314215, 593480}, NULL) = 0 > gettimeofday({1418314215, 593565}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3bc00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3bc00000 > mprotect(0xfcc3bc05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3bc06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3bc06000 > mprotect(0xfcc3bcff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314215, 696233}, NULL) = 0 > gettimeofday({1418314215, 696315}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3bd00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3bd00000 > mprotect(0xfcc3bd05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3bd06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3bd06000 > mprotect(0xfcc3bdff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3be00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3be00000 > mprotect(0xfcc3be05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3be06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3be06000 > mprotect(0xfcc3beff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314215, 875801}, NULL) = 0 > gettimeofday({1418314215, 875898}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3bf00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3bf00000 > mprotect(0xfcc3bf05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3bf06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3bf06000 > mprotect(0xfcc3bfff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314216, 38282}, NULL) = 0 > gettimeofday({1418314216, 38365}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3c000000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c000000 > mprotect(0xfcc3c005000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c006000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c006000 > mprotect(0xfcc3c0ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314216, 255472}, NULL) = 0 > gettimeofday({1418314216, 255552}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3c100000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c100000 > mprotect(0xfcc3c105000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c106000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c106000 > mprotect(0xfcc3c1ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c200000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c200000 > mprotect(0xfcc3c205000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c206000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c206000 > mprotect(0xfcc3c2ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314216, 390157}, NULL) = 0 > gettimeofday({1418314216, 390240}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3c300000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c300000 > mprotect(0xfcc3c305000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c306000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c306000 > mprotect(0xfcc3c3ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314216, 532144}, NULL) = 0 > gettimeofday({1418314216, 532225}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3c400000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c400000 > mprotect(0xfcc3c405000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c406000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c406000 > mprotect(0xfcc3c4ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c500000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c500000 > mprotect(0xfcc3c505000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c506000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c506000 > mprotect(0xfcc3c5ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314216, 676703}, NULL) = 0 > gettimeofday({1418314216, 676786}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3c600000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c600000 > mprotect(0xfcc3c605000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c606000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c606000 > mprotect(0xfcc3c6ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314216, 832680}, NULL) = 0 > gettimeofday({1418314216, 845863}, NULL) = 0 > gettimeofday({1418314216, 855742}, NULL) = 0 > mmap(0xfcc3c600000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3c600000 > mmap(0xfcc3c500000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3c500000 > mmap(0xfcc3c400000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3c400000 > mmap(0xfcc3c300000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3c300000 > mmap(0xfcc3c200000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3c200000 > mmap(0xfcc3c100000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3c100000 > mmap(0xfcc3c000000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3c000000 > mmap(0xfcc3bf00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3bf00000 > mmap(0xfcc3be00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3be00000 > mmap(0xfcc3bd00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3bd00000 > mmap(0xfcc3bc00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3bc00000 > mmap(0xfcc3bb00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3bb00000 > mmap(0xfcc3ba00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3ba00000 > mmap(0xfcc3b900000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b900000 > mmap(0xfcc3b800000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b800000 > mmap(0xfcc3b700000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b700000 > mmap(0xfcc3b600000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b600000 > mmap(0xfcc3b500000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b500000 > mmap(0xfcc3b400000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b400000 > mmap(0xfcc3b300000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b300000 > mmap(0xfcc3b200000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b200000 > mmap(0xfcc3b100000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b100000 > mmap(0xfcc3b000000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3b000000 > gettimeofday({1418314216, 872832}, NULL) = 0 > gettimeofday({1418314216, 878758}, NULL) = 0 > gettimeofday({1418314216, 878857}, NULL) = 0 > gettimeofday({1418314216, 878928}, NULL) = 0 > gettimeofday({1418314216, 878996}, NULL) = 0 > gettimeofday({1418314216, 879065}, NULL) = 0 > gettimeofday({1418314216, 879154}, NULL) = 0 > gettimeofday({1418314216, 879648}, NULL) = 0 > gettimeofday({1418314216, 879718}, NULL) = 0 > gettimeofday({1418314216, 880313}, NULL) = 0 > gettimeofday({1418314216, 880383}, NULL) = 0 > gettimeofday({1418314216, 880452}, NULL) = 0 > gettimeofday({1418314216, 880521}, NULL) = 0 > gettimeofday({1418314216, 880590}, NULL) = 0 > gettimeofday({1418314216, 880658}, NULL) = 0 > gettimeofday({1418314216, 885737}, NULL) = 0 > gettimeofday({1418314216, 887874}, NULL) = 0 > gettimeofday({1418314216, 887965}, NULL) = 0 > gettimeofday({1418314216, 888056}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > gettimeofday({1418314217, 8997}, NULL) = 0 > gettimeofday({1418314217, 9096}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > gettimeofday({1418314217, 115505}, NULL) = 0 > gettimeofday({1418314217, 115588}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > gettimeofday({1418314217, 217206}, NULL) = 0 > gettimeofday({1418314217, 217287}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > gettimeofday({1418314217, 315058}, NULL) = 0 > gettimeofday({1418314217, 315137}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > gettimeofday({1418314217, 412213}, NULL) = 0 > gettimeofday({1418314217, 412294}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3c700000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c700000 > mprotect(0xfcc3c705000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c706000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c706000 > mprotect(0xfcc3c7ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c800000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c800000 > mprotect(0xfcc3c805000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c806000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c806000 > mprotect(0xfcc3c8ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314217, 519206}, NULL) = 0 > gettimeofday({1418314217, 519289}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3c900000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c900000 > mprotect(0xfcc3c905000, 4096, PROT_NONE) = 0 > mmap(0xfcc3c906000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3c906000 > mprotect(0xfcc3c9ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314217, 653545}, NULL) = 0 > gettimeofday({1418314217, 653624}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3ca00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3ca00000 > mprotect(0xfcc3ca05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3ca06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3ca06000 > mprotect(0xfcc3caff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3cb00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3cb00000 > mprotect(0xfcc3cb05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3cb06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3cb06000 > mprotect(0xfcc3cbff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314217, 757446}, NULL) = 0 > gettimeofday({1418314217, 757530}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3cc00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3cc00000 > mprotect(0xfcc3cc05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3cc06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3cc06000 > mprotect(0xfcc3ccff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314217, 880698}, NULL) = 0 > gettimeofday({1418314217, 880777}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3cd00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3cd00000 > mprotect(0xfcc3cd05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3cd06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3cd06000 > mprotect(0xfcc3cdff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314217, 996985}, NULL) = 0 > gettimeofday({1418314217, 997063}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3ce00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3ce00000 > mprotect(0xfcc3ce05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3ce06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3ce06000 > mprotect(0xfcc3ceff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3cf00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3cf00000 > mprotect(0xfcc3cf05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3cf06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3cf06000 > mprotect(0xfcc3cfff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314218, 156422}, NULL) = 0 > gettimeofday({1418314218, 156512}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3d000000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d000000 > mprotect(0xfcc3d005000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d006000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d006000 > mprotect(0xfcc3d0ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314218, 331075}, NULL) = 0 > gettimeofday({1418314218, 331155}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3d100000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d100000 > mprotect(0xfcc3d105000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d106000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d106000 > mprotect(0xfcc3d1ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d200000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d200000 > mprotect(0xfcc3d205000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d206000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d206000 > mprotect(0xfcc3d2ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314218, 469695}, NULL) = 0 > gettimeofday({1418314218, 469779}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3d300000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d300000 > mprotect(0xfcc3d305000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d306000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d306000 > mprotect(0xfcc3d3ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314218, 572096}, NULL) = 0 > gettimeofday({1418314218, 572177}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3d400000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d400000 > mprotect(0xfcc3d405000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d406000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d406000 > mprotect(0xfcc3d4ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d500000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d500000 > mprotect(0xfcc3d505000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d506000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d506000 > mprotect(0xfcc3d5ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314218, 710743}, NULL) = 0 > gettimeofday({1418314218, 710827}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3d600000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d600000 > mprotect(0xfcc3d605000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d606000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d606000 > mprotect(0xfcc3d6ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314218, 844711}, NULL) = 0 > gettimeofday({1418314218, 844795}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3d700000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d700000 > mprotect(0xfcc3d705000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d706000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d706000 > mprotect(0xfcc3d7ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314218, 979663}, NULL) = 0 > gettimeofday({1418314218, 979743}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3d800000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d800000 > mprotect(0xfcc3d805000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d806000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d806000 > mprotect(0xfcc3d8ff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d900000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d900000 > mprotect(0xfcc3d905000, 4096, PROT_NONE) = 0 > mmap(0xfcc3d906000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3d906000 > mprotect(0xfcc3d9ff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314219, 128634}, NULL) = 0 > gettimeofday({1418314219, 128719}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3da00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3da00000 > mprotect(0xfcc3da05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3da06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3da06000 > mprotect(0xfcc3daff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314219, 264543}, NULL) = 0 > gettimeofday({1418314219, 264626}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3db00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3db00000 > mprotect(0xfcc3db05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3db06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3db06000 > mprotect(0xfcc3dbff000, 4096, PROT_NONE) = 0 > mmap(0xfcc3dc00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3dc00000 > mprotect(0xfcc3dc05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3dc06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3dc06000 > mprotect(0xfcc3dcff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314219, 400804}, NULL) = 0 > gettimeofday({1418314219, 400886}, NULL) = 0 > futex(0x7faad80008c8, FUTEX_WAKE_PRIVATE, 1) = 1 > mmap(0xfcc3dd00000, 20480, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3dd00000 > mprotect(0xfcc3dd05000, 4096, PROT_NONE) = 0 > mmap(0xfcc3dd06000, 1019904, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfcc3dd06000 > mprotect(0xfcc3ddff000, 4096, PROT_NONE) = 0 > gettimeofday({1418314219, 584835}, NULL) = 0 > gettimeofday({1418314219, 612381}, NULL) = 0 > gettimeofday({1418314219, 625079}, NULL) = 0 > mmap(0xfcc3dd00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3dd00000 > mmap(0xfcc3dc00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3dc00000 > mmap(0xfcc3db00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3db00000 > mmap(0xfcc3da00000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3da00000 > mmap(0xfcc3d900000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3d900000 > mmap(0xfcc3d800000, 1048576, PROT_NONE, > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xfcc3d800000 > > > > <https://lh6.googleusercontent.com/-cW1iGZ63758/VJFSUll2UkI/AAAAAAAAAbw/LAWpHwXTJk0/s1600/Screen%2BShot%2B2014-12-11%2Bat%2B5.40.17%2BPM.png> > > > <https://lh4.googleusercontent.com/-QppJLXgg2uA/VJFSacR1icI/AAAAAAAAAb4/QvN3e9oaQxQ/s1600/Screen%2BShot%2B2014-12-11%2Bat%2B5.49.47%2BPM.png> > > > <https://lh3.googleusercontent.com/-koRqVyIiSiQ/VJFSdTAAedI/AAAAAAAAAcA/JsM37NfgCoQ/s1600/Screen%2BShot%2B2014-12-11%2Bat%2B5.43.03%2BPM.png> > > > Also i performed lsof command on the process and that was the output: > > > <https://lh5.googleusercontent.com/-mGICVWRn_hk/VJFSzk79eqI/AAAAAAAAAcI/9D3hJL_EojA/s1600/Screen%2BShot%2B2014-12-11%2Bat%2B6.12.21%2BPM.png> > > > as you can see there are some (CLOST_WAIT) in the output while the other > process(worker) which was working fine at the time had no (CLOSE_WAIT). > > I am new to node so i don't really understand what is going wrong with my > application so if anyone has any idea of what is going on, or can explain > for me the output of strace and lsof please help! > > -- > Job board: http://jobs.nodejs.org/ > New group rules: > https://gist.github.com/othiym23/9886289#file-moderation-policy-md > Old group rules: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/nodejs/02001bf2-c2d2-4c8f-a09a-932e2b158030%40googlegroups.com > <https://groups.google.com/d/msgid/nodejs/02001bf2-c2d2-4c8f-a09a-932e2b158030%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Job board: http://jobs.nodejs.org/ New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAEv2Vf%2ByRmEm35rQstE7vGxXBNMr9SbYHdyx0pbm2nWDOJnnrg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
