If what you are saying is what it's really doing, and no one else suggests anything, you might also try going multithreaded.
Libcurl is thread safe with a few caveats (check out this page for more info http://curl.haxx.se/libcurl/features.html) On Mon, Jun 6, 2011 at 11:30 AM, Jimish Shah <[email protected]> wrote: > Hi, > > I am trying to use libcurl to scale to several thousand https connections > to an external web server. I have downloaded and investigated the source > code, so i have a fair understanding of the function calls. > > If i use multi-handle and add say 1000 easy handles to it. If each of the > thousand easy handles need to connect simultaneously , i noticed that there > is a blocking "select ()" during the connect phase as explained below. > > Curl_multi_perform() --> multi_runsingle() --> Curl_connect() ....... --> > Singleipconnect --> waitconnect() --> Curl_socket_ready()* --> select ()* > every > one second until error or success or timeout > > In other words, although the socket itself is non-blocking, this function > call waits (multi_runsingle() loops until error or success) until the single > connect on one easy handle either succeeds or fails completely or times-out, > which makes it somewhat pipelined if the responding web-server is slow (in > which case we will repeatedly get EINPROGRESS on the socket). Also the other > 999 easy handle connect are waiting till this one is done. Hence its > serialized in some manner. > > Is there a way wherein libcurl allows to open all the 1000 connections > using some type of a event driven mechanism or some alternate to change the > connection state as and when they are successfull / fail. > > Pls correct me if any of my understanding is incorrect. Also i went through > the mailer, but couldnt find something that answered my questions. > > Pls let me know if i can clarify further. Thanks in advance for helping me > out. > > ~ Jimish > > “Perseverance is not a long race; it is many short races one after the > other” - Walter Elliot > > ------------------------------------------------------------------- > List admin: http://cool.haxx.se/list/listinfo/curl-library > Etiquette: http://curl.haxx.se/mail/etiquette.html >
------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
