The problem I was still seeing was my bug: I handled redirects with the default resolver instead of dns.resolve.
The issue is here: https://github.com/joyent/node/issues/2868 On Mar 5, 3:39 pm, Nikola Toshev <[email protected]> wrote: > Wow, thanks! My hypothesis turns out to be correct! > > I'm still seeing a delay in disk I/O at higher load, so I will > investigate a little bit further. Is there something else that shares > this thread pool? Or maybe when I pass IP addresses to http.get they > are still passed through the thread pool, even though no actual DNS > resolution is necessary. Looks like this is happening after I've > saturated the network though, so in practice it doesn't matter that > much to me anymore. > > Nikola > > On Mar 5, 3:17 pm, Bert Belder <[email protected]> wrote: > > > > > > > > > On Mar 5, 10:43 am, Nikola Toshev <[email protected]> wrote: > > > > I have some code that makes lots of http client requests (several per > > > second) and does occasional async fs operation (one every few > > > seconds). Normally the fs operation takes milliseconds to complete, > > > but as the number of http requests increase, it starts being delayed > > > for seconds to minutes, even though CPU load is <10% and disk i/o load > > > is minimal. Is there any explanation why is this happening? > > > > Http requests also begin to take a while to complete even though the > > > network pipe is not saturated. I'm using local DNS caching and it does > > > help somewhat, but I'm still not able to go beyond 100 or so http > > > requests per second, with CPU usage <10% and network not being > > > saturated. I use agent:false for the http requests. Again, I expect to > > > be able to saturate the network connection before such slowdowns start > > > happening. > > > > I've tried several 0.6.x versions of Node. > > > > Thanks in advance, > > > Nikola > > > DNS lookups that are automatically performed use getaddrinfo on the > > thread pool. File system operations can get stuck behind behind dns > > lookups that take a while to complete. Originally we planned to create > > a separate thread pool for DNS queries, but we never got around to it. > > In the meantime you could use dns.resolve() to manually look up the ip > > address for the host. And please file an issue for this :-) -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
