On Feb 15, 2014, at 20:05, Jeff King wrote:
I've noticed that git does not reuse http connections when fetching,
and
I'm trying to figure out why. It seems related to our use of two curl
features:
1. If we use the curl_multi interface (even though we are doing the
requests sequentially), we do not get reuse.
Take a look at the curl_multi_setopt page [1]. I think these explain
the curl_multi issue:
CURLMOPT_PIPELINING
Pass a long set to 1 to enable or 0 to disable. Enabling pipelining
on a multi handle will make it attempt to perform HTTP Pipelining as
far as possible for transfers using this handle. This means that if
you add a second request that can use an already existing
connection, the second request will be "piped" on the same
connection rather than being executed in parallel. (Added in 7.16.0)
CURLMOPT_MAX_TOTAL_CONNECTIONS
Pass a long. The set number will be used as the maximum amount of
simultaneously open connections in total. For each new session,
libcurl will open a new connection up to the limit set by
CURLMOPT_MAX_TOTAL_CONNECTIONS. When the limit is reached, the
sessions will be pending until there are available connections. If
CURLMOPT_PIPELINING is 1, libcurl will try to pipeline if the host
is capable of it.
The default value is 0, which means that there is no limit. However,
for backwards compatibility, setting it to 0 when
CURLMOPT_PIPELINING is 1 will not be treated as unlimited. Instead
it will open only 1 connection and try to pipeline on it.
(Added in 7.30.0)
If pipelining is off (the default) and total connections is not 1 it
sounds to me from the description above that the requests will be
executed on separate connections until the maximum number of
connections is in use and then there might be some reuse. That might
not be what's actually going on with multi, but that's my guess and I
think setting CURLMOPT_PIPELINING to to 1 and then also setting
CURLMOPT_MAX_TOTAL_CONNECTIONS to a non-zero value might be what you
want.
2. If we set CURLOPT_HTTPAUTH to CURLAUTH_ANY, we do not get reuse.
[snip]
My curl version is 7.35.0, if that makes a difference.
And that is explained by this from the CHANGES file:
Daniel Stenberg (7 Jan 2014)
- ConnectionExists: fix NTLM check for new connection
When the requested authentication bitmask includes NTLM, we cannot
re-use a connection for another username/password as we then risk
re-using NTLM (connection-based auth).
This has the unfortunate downside that if you include NTLM as a
possible
auth, you cannot re-use connections for other usernames/passwords
even
if NTLM doesn't end up the auth type used.
Reported-by: Paras S
Patched-by: Paras S
Bug: http://curl.haxx.se/mail/lib-2014-01/0046.html
Looks like you're just lucky as that above change first appears in
7.35.0. But it seems there are some patches for older versions so
they might be affected as well [2].
Adjusting your test program to leave the first connection set to
CURLAUTH_ANY, but then setting the second one to CURLAUTH_ANY with the
two NTLM bits turned off (CURLAUTH_NTLM and CURLAUTH_NTLM_WB) results
in the connection being reused for me. :)
With the older cURL versions I already had installed, the second
connection is always reused. I didn't see the non-reuse with your
sample code until I fetched 7.35.0.
--Kyle
[1] http://curl.haxx.se/libcurl/c/curl_multi_setopt.html
[2] http://curl.haxx.se/docs/adv_20140129.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
