Hi, I have an application that is making many http requests through curl. Each request is handled by a thread. On my OS (OSX) there is a default file descriptor limit of 256. When I make a curl request with curl_easy_perform, it creates two pipes in addition to creating the socket on which to make the request. It appears that these pipes are made for communication internal to curl about async DNS resolution. This means that I'm using three file descriptors for each request, which means get "out of file descriptor" errors when my application has ~80 threads making curl requests.
It looks like I can disable async resolution at compile-time, which I would think would eliminate the pipes and associated file descriptors, but I can't control how the library was built for users. I don't see any way I can disable this async resolution at runtime, but perhaps I'm missing something. Does anyone have any ideas/recommendations to eliminate the FDs being used for DNS resolution or some other ideas on how I can reduce the likelihood of running out of FDs when using lots of simultaneous requests? Thanks, -- Andrew Bell [email protected]
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
