You don't really need to set CURLMOPT_MAX_TOTAL_CONNECTIONS = 1 to use HTTP2 multiplexing.
You basically need to do the following steps: 1. Enable HTTP multiplexing by setting CURLMOPT_PIPELINING option on a multi-handle with CURLPIPE_MULTIPLEX flag. 2. Set CURLOPT_PIPEWAIT option to 1 on easy handle for each transfer you add to that multi-handle. The last setting will force the transfer to wait until the HTTP2 pipeline is ready and not try to open a new connection. Thanks, Dmitry Karpov -----Original Message----- From: curl-library <curl-library-boun...@lists.haxx.se> On Behalf Of Oliver Schonrock via curl-library Sent: Friday, October 25, 2024 8:15 AM To: curl-library@lists.haxx.se Cc: Oliver Schonrock <oli...@schonrocks.com> Subject: [EXTERNAL] Re: How can libcurl/curl_multi perform like curl --parallel ? On 25/10/2024 08:58, Oliver Schonrock via curl-library wrote: > > What is `curl --parallel --parallel-max 150` doing internally and how > can I reproduce this performance with libcurl? Answering my own question... Some investigating with `perf` showed that the 100% CPU process was spending almost all its time negotiating encrypted connections. Enabling CURLOPT_VERBOSE showed that it was opening a separate encrypted connected for each download. Comparing that to the `curl --parallel --verbose` case, showed that `curl` only negotiated a single connection and then did HTTP2 multiplexed streaming just as I had suspected. Some experimentation showed that setting CURLMOPT_MAX_TOTAL_CONNECTIONS = 1 thereby limiting curl_multi to a single encrypted connection, and forcing libcurl to multiplex streams over HTTP2. So now libcurl delivers very similar performance to `curl --parallel`. Some additional tuning of CURLMOPT_MAX_CONCURRENT_STREAMS will probably be required, perhaps to match the --parallel-max=150`, to optimise the full sized download. All good. -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
