Is there such a thing as a maximum number of functions you can put in the event loop?
I have a highly asynchronous app, which results in a ton of async callbacks getting pushed into the event loop. One "operation" could result in several tens of async functions. Everything runs fine at low concurrency (like 5000 concurrent operations or so), but when I crank it up (20,000), it runs fine for a little while (less than a second), and then gets stuck. It queues up a bunch of callbacks, and they never get resolved. The app it just at a standstill. Is there any technical reason this would be happening? The app in question is basically a task runner: https://github.com/catdad/grandma/ At first, I suspected that IPC was the issue (because multiple threads), so I wrote a whole custom IPC implementation using named sockets. Still the same issue. The IPC bits work just fine, more process.nextTick instances get called, but the function never executes once the concurrency is high enough. I have tried setting --max_old_space_size to a high number (thinking it might be a memory issue) and that did not help. I have also set UV_THREADPOOL_SIZE to 128, 256, and 512, thinking that maybe the libuv thread pool is getting saturated, and that doesn't seem to be helping either. I have found that when using longer async tasks (process.nextTick vs. setTimeout(cb, 10) vs. an http request) that longer tasks get stuck at a much lower concurrency. Assuming there is no bug in the code (I know, everyone hates doing that), is there any technical reason for such an issue? -- Job board: http://jobs.nodejs.org/ New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/267a1ccf-27cc-4a82-8009-d3794c56fcf8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
