On Thu, 22 Aug 2013, yi huang wrote:
Currently, my loop is a standard one, which is like this:while (1) { curl_multi_fdset(multi, &fdread, &rdwrite, &fdexcept, &maxfd); select(&maxfd, &fdread, &fdwrite, &fdexcept, timeout);
This is very naive and is not properly taking into account what curl_multi_fdset() can return in the variables.
Then i set timeout of select to 5 seconds(i don't want to wast too much cpu cycles),
That's not really optimal. Try using curl_multi_timeout() to get the best timeout.
i found that the process_response_body always delayed by 5 seconds.
I suspect you get this due to a minor flaw in libcurl in combination to how your application uses the API. See below.
I've investigated this by reading the source code of libcurl, i found out that after the last read of a http response, the easy_handle enter the CURLM_STATE_DONE state, and in next curl_multi_perform call, it enter CURLM_STATE_COMPLETED state, then the CURLMSG_DONE message got append to multi->msglist, then use code can detect the completion of a request. So i have to wait for another cycle to process my response body, and when io is not busy, this cycle can be very long.
Yes, I think you've found a legitimate problem!I think the code in CURLM_STATE_DONE should set "result = CURLM_CALL_MULTI_PERFORM" pretty much unconditionally so that it'll move on and execute the next state immediately.
I've attached my suggested fix for this. Can you try it out and see if that alone perhaps fixes your problem?
-- / daniel.haxx.se
From b12db0fe5cef56154211b5ddf3802e4016a3eebd Mon Sep 17 00:00:00 2001 From: Daniel Stenberg <[email protected]> Date: Wed, 21 Aug 2013 23:08:38 +0200 Subject: [PATCH] multi: move on from STATE_DONE faster Make sure we always return CURLM_CALL_MULTI_PERFORM when we reach CURLM_STATE_DONE since the state is transient and it can very well continue executing as there is nothing to wait for. Bug: http://curl.haxx.se/mail/lib-2013-08/0211.html Reported-by: Yi Huang --- lib/multi.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/lib/multi.c b/lib/multi.c index 93be770..6b42361 100644 --- a/lib/multi.c +++ b/lib/multi.c @@ -1556,21 +1556,15 @@ static CURLMcode multi_runsingle(struct Curl_multi *multi, } case CURLM_STATE_DONE: + /* this state is highly transient, so run another loop after this */ + result = CURLM_CALL_MULTI_PERFORM; if(data->easy_conn) { /* Remove ourselves from the receive pipeline, if we are there. */ - Curl_removeHandleFromPipeline(data, - data->easy_conn->recv_pipe); + Curl_removeHandleFromPipeline(data, data->easy_conn->recv_pipe); /* Check if we can move pending requests to send pipe */ Curl_multi_process_pending_handles(multi); - if(data->easy_conn->bits.stream_was_rewound) { - /* This request read past its response boundary so we quickly let - the other requests consume those bytes since there is no - guarantee that the socket will become active again */ - result = CURLM_CALL_MULTI_PERFORM; - } - /* post-transfer command */ data->result = Curl_done(&data->easy_conn, CURLE_OK, FALSE); /* @@ -1587,7 +1581,6 @@ static CURLMcode multi_runsingle(struct Curl_multi *multi, if(data->wildcard.state != CURLWC_DONE) { /* if a wildcard is set and we are not ending -> lets start again with CURLM_STATE_INIT */ - result = CURLM_CALL_MULTI_PERFORM; multistate(data, CURLM_STATE_INIT); break; } @@ -1596,7 +1589,6 @@ static CURLMcode multi_runsingle(struct Curl_multi *multi, /* after we have DONE what we're supposed to do, go COMPLETED, and it doesn't matter what the Curl_done() returned! */ multistate(data, CURLM_STATE_COMPLETED); - break; case CURLM_STATE_COMPLETED: -- 1.8.4.rc3
------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
