Wow, thanks! My hypothesis turns out to be correct! I'm still seeing a delay in disk I/O at higher load, so I will investigate a little bit further. Is there something else that shares this thread pool? Or maybe when I pass IP addresses to http.get they are still passed through the thread pool, even though no actual DNS resolution is necessary. Looks like this is happening after I've saturated the network though, so in practice it doesn't matter that much to me anymore.
Nikola On Mar 5, 3:17 pm, Bert Belder <[email protected]> wrote: > On Mar 5, 10:43 am, Nikola Toshev <[email protected]> wrote: > > > > > > > > > > > I have some code that makes lots of http client requests (several per > > second) and does occasional async fs operation (one every few > > seconds). Normally the fs operation takes milliseconds to complete, > > but as the number of http requests increase, it starts being delayed > > for seconds to minutes, even though CPU load is <10% and disk i/o load > > is minimal. Is there any explanation why is this happening? > > > Http requests also begin to take a while to complete even though the > > network pipe is not saturated. I'm using local DNS caching and it does > > help somewhat, but I'm still not able to go beyond 100 or so http > > requests per second, with CPU usage <10% and network not being > > saturated. I use agent:false for the http requests. Again, I expect to > > be able to saturate the network connection before such slowdowns start > > happening. > > > I've tried several 0.6.x versions of Node. > > > Thanks in advance, > > Nikola > > DNS lookups that are automatically performed use getaddrinfo on the > thread pool. File system operations can get stuck behind behind dns > lookups that take a while to complete. Originally we planned to create > a separate thread pool for DNS queries, but we never got around to it. > In the meantime you could use dns.resolve() to manually look up the ip > address for the host. And please file an issue for this :-) -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
