On 25/10/2024 08:58, Oliver Schonrock via curl-library wrote:
What is `curl --parallel --parallel-max 150` doing internally and how
can I reproduce this performance with libcurl?
Answering my own question...
Some investigating with `perf` showed that the 100% CPU process was
spending almost all its time negotiating encrypted connections.
Enabling CURLOPT_VERBOSE showed that it was opening a separate encrypted
connected for each download.
Comparing that to the `curl --parallel --verbose` case, showed that
`curl` only negotiated a single connection and then did HTTP2
multiplexed streaming just as I had suspected.
Some experimentation showed that setting
CURLMOPT_MAX_TOTAL_CONNECTIONS = 1
thereby limiting curl_multi to a single encrypted connection, and
forcing libcurl to multiplex streams over HTTP2.
So now libcurl delivers very similar performance to `curl --parallel`.
Some additional tuning of
CURLMOPT_MAX_CONCURRENT_STREAMS
will probably be required, perhaps to match the --parallel-max=150`, to
optimise the full sized download.
All good.
--
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
