I just realized this feature request is already tracked under the https://github.com/curl/curl/wiki/curl_multi_unblock wiki page.
It says on the page that some kind of locking (mutexes) are necessary to implement this, I don't think that's true. As long as this unblock function is only accessing the write half of the socketpair and only does an swrite() call, it should be thread-safe (assuming the application had some kind of synchronization between the threads after setting up the multi and before this unblock function is called so the fd itself is seen correctly by the other thread; the send() syscall itself is thread-safe). It's also the app's responsibility not to call unblock after destructing the multi, but this should be like this either way. ________________________________ From: curl-library <[email protected]> on behalf of Gergely Nagy via curl-library <[email protected]> Sent: Tuesday, November 12, 2019 9:50 AM To: [email protected] <[email protected]> Cc: Gergely Nagy <[email protected]> Subject: Feature idea: wakeup option for curl_multi_wait/poll Hi! We have a use-case where we use a single curl multi with lots of easy connections which are added/removed in a relatively adhoc manner, anytime. We previously used the multi_socket API with libuv for this, but we'd like to migrate to the curl_multi_poll API. The problem is that when we for example want to add a new easy connection, we have to wait until curl_multi_poll wakes up. We could play with setting the timeout low enough so this waiting time is not too long, but not too low to keep CPU usage low. We found a solution to this, where we create a socketpair (or emulate one on Windows just like Curl_socketpair does), and add one of the ends as an extra fd. This way we can wake the thread up by writing to the other end. This works pretty well, but I think this could be a feature of the multi API inside curl, probably useful for others as well. As the Curl_socketpair function is not public, doing this manually includes copying code as well. Idea about how this could be added to the API: 1. Two fds should be added to the Curl_multi struct left in an invalid state at first (so there won't be a perf penalty for those not wanting to use this feature) 2. There should be a new CURLMOPT_ENABLE_WAKEUP option that initializes a socketpair and stores the fds when set to 1L. (if some platform doesn't support socketpair at all, it could return an errorcode (for example CURLM_BAD_SOCKET, CURLM_UNKNOWN_OPTION or some new error code just added for this), which could be handled by applications for example by lowering the timeout to make the unfortunately necessary waits shorter) 3. There should be a new function curl_multi_wakeup which writes one byte to the writing end of the socketpair. 4. If the option was set successfully, Curl_multi_wait should add the reading end of the socketpair to its fd list, and after the actual Curl_poll call, if this fd is marked POLLIN, then it should try to read out all data from it and discard it. (This is not so trivial, because it should avoid reading in a long loop if there are lots of wakeup calls meanwhile). Would you be interested in supporting this feature? Thanks, Gergely Nagy
------------------------------------------------------------------- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
