Re: git https transport and wrong password
Maybe my git installation was incomplete before when running from ~/bin since I was not able to set break points to http_request() and some debug code was not there until I ran git through bin-wrappers in the source tree. I added some debug prints to http.c functions http_request() and handle_curl_result(), and now I see this chain of events: http_request_reauth() http_request() GET ...info/refs?service=git-upload-pack HTTP/1.1 401 Authorization Required * Ignoring the response-body * Issue another request to this URL: '...' GET ...info/refs?service=git-upload-pack HTTP/1.1 401 Authorization Required handle_curl_result: res = 22, http_code = 401, user = ..., pass = (null) Password for '...': (enter valid password) GET ...info/refs?service=git-upload-pack HTTP/1.1 200 OK So, for some reason the first GET request is issued twice and first 401 is ignored. I'll try to debug run_active_slot() next... -Mikko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git https transport and wrong password
[+cc Daniel for curl questions below] On Wed, Apr 03, 2013 at 12:43:02PM +0300, Mikko Rapeli wrote: Maybe my git installation was incomplete before when running from ~/bin since I was not able to set break points to http_request() and some debug code was not there until I ran git through bin-wrappers in the source tree. Debugging git-over-http is somewhat difficult because the interesting bits happen in sub-processes. You can get much closer to the http calls by running the transport helper directly, like: gdb --args git-remote-https https://yourhost/ which will start by reading commands from stdin (try list to get it to fetch the remote refs). I added some debug prints to http.c functions http_request() and handle_curl_result(), and now I see this chain of events: http_request_reauth() http_request() GET ...info/refs?service=git-upload-pack HTTP/1.1 401 Authorization Required * Ignoring the response-body * Issue another request to this URL: '...' GET ...info/refs?service=git-upload-pack HTTP/1.1 401 Authorization Required handle_curl_result: res = 22, http_code = 401, user = ..., pass = (null) Password for '...': (enter valid password) GET ...info/refs?service=git-upload-pack HTTP/1.1 200 OK So, for some reason the first GET request is issued twice and first 401 is ignored. I'll try to debug run_active_slot() next... Right, I think that's curl trying to make use of the username in the URL. Try this (I'm using github here as a convenient http servers, but you should be able to replicate with your internal server): $ GIT_CURL_VERBOSE=1 git ls-remote https://f...@github.com/requires/auth \ 21 /dev/null | egrep '^|^ HTTP|^Authorization|requested URL' GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 HTTP/1.1 401 Authorization Required GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 Authorization: Basic Zm9vOg== HTTP/1.1 401 Authorization Required * The requested URL returned error: 401 Password for 'https://f...@github.com': GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 Authorization: Basic Zm9vOmJhcg== HTTP/1.1 401 Authorization Required * The requested URL returned error: 401 So you can see that curl makes _two_ requests internally before it returns the 401. One unadorned, and one with just the username (Zm9vOg==, which decodes to foo:) for the auth. Then git prompts for the password, and we retry (and of course I am feeding it a bogus username/password combo, so we get another 401). I would expect without the username in the URL for it to make only two requests: one to get the first 401, then git collects the credentials, then a follow-up with the credentials. But instead we get: $ GIT_CURL_VERBOSE=1 git ls-remote https://github.com/requires/auth \ 21 /dev/null | egrep '^|^ HTTP|^Authorization|requested URL' GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 * The requested URL returned error: 401 Authorization Required Username for 'https://github.com': foo Password for 'https://f...@github.com': GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 HTTP/1.1 401 Authorization Required GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 Authorization: Basic Zm9vOmJhcg== HTTP/1.1 401 Authorization Required * The requested URL returned error: 401 So we get a 401, as expected, git prompts for the credentials and feeds them directly to curl, but then we still get _two_ requests: we trigger another 401, and only then does curl provide the authorization header to the server. I'm not sure if that extra auth is intended or not. It's also possible that git is screwing up in providing the credentials to curl, but I don't think so. We feed them to the curl handle as soon as we get them, and there should be only one handle in use here. -Peff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git https transport and wrong password
On Wed, Apr 03, 2013 at 10:12:12AM -0400, Jeff King wrote: I would expect without the username in the URL for it to make only two requests: one to get the first 401, then git collects the credentials, then a follow-up with the credentials. But instead we get: $ GIT_CURL_VERBOSE=1 git ls-remote https://github.com/requires/auth \ 21 /dev/null | egrep '^|^ HTTP|^Authorization|requested URL' GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 * The requested URL returned error: 401 Authorization Required Username for 'https://github.com': foo Password for 'https://f...@github.com': GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 HTTP/1.1 401 Authorization Required GET /requires/auth/info/refs?service=git-upload-pack HTTP/1.1 Authorization: Basic Zm9vOmJhcg== HTTP/1.1 401 Authorization Required * The requested URL returned error: 401 So we get a 401, as expected, git prompts for the credentials and feeds them directly to curl, but then we still get _two_ requests: we trigger another 401, and only then does curl provide the authorization header to the server. I'm not sure if that extra auth is intended or not. git uses CURLAUTH_ANY which means: first try without authentication (CURLAUTH_NONE), if that fails it will try (I guess) CURLAUTH_BASIC|DIGEST| GSS|NTML and so on, and only then it will fail with the 401. It seems that skipping CURLAUTH_NONE try is not possible even if it's not a good idea when a username and possibly password is available. Changing CURLAUTH_ANY to skip CURLAUTH_NONE could also break other users. Since netrc support really needs this one try from git to curl before password prompt I guess in our case using HTTPS with git is simply not feasible. Changing the corporate single sign-on policies is also hard so I will now try to get SSH transport running on the server. Account locking will still be quite easy but hopefully only after multiple false passwords to the SSH promp. -Mikko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
git https transport and wrong password
Hi, I have a problem with git (1.7.9 and 1.8.2.357.gcc3e4eb) and https transport to gerrit server (2.5.1-3-g719dfc7). I'm producing the problem on Cygwin but my colleagues have same issue on Linux as well. Gerrit server is matching corporate policies with single sign on, so after three failed login attempts the account gets locked until a password reset. Git amplifies this problem by asking for users password only once, and if user made a typo git is still re-using the wrong password enough times to get an account immediately locked. I have client side logs with GIT_CURL_VERBOSE=1 but from intranet so can't publish them directly. Here's roughly what the log shows: --- $ GIT_CURL_VERBOSE=1 git fetch ... GET /gerrit/.../info/refs?service=git-upload-pack HTTP/1.1 ... HTTP/1.1 401 Authorization Required ... -- I guess git prompts for password here. -- * Issue another request to this URL: 'https://..info/refs?service=git-upload-pack' ... * Re-using existing connection! ... ... * Server auth using Basic with user '...' GET /gerrit/.../info/refs?service=git-upload-pack HTTP/1.1 Authorization: Basic ... ... HTTP/1.1 401 Authorization Required Date: ... * Authentication problem. Ignoring this. ... * The requested URL returned error: 401 * Closing connection 0 ... * About to connect() to ... ... * Connected to ... ... * STATE: PROTOCONNECT = DO handle... * Server auth using Basic with user '...' GET /gerrit/.../info/refs?service=git-upload-pack HTTP/1.1 Authorization: Basic ... ... * STATE: DO = DO_DONE handle... * STATE: DO_DONE = WAITPERFORM handle... * STATE: WAITPERFORM = PERFORM handle... ... HTTP/1.1 302 Found ... Location: ...funnylongurl ... * Ignoring the response-body * Connection #1 to host ... left intact * Issue another request to this URL: '...funnylongurl' ... * Server auth using Basic with user '...' GET ...funnylongurl Authorization: Basic ... ... * The requested URL returned error: 500 Internal Server Error * Closing connection 1 ... * About to connect()... ... * Server auth using Basic with user '...' GET /gerrit/.../info/refs HTTP/1.1 Authorization: Basic ... ... HTTP/1.1 302 Found Date... Set-Cookie... Cache-Control: no-store Location: ...funnylongurl ... * Re-using existing connection! (#2)... GET ...funnylongurl ... * The requested URL returned error: 500 Internal Server Error * Closing connection 2 ... error: The requested URL returned error: 500 Internal Server Error while accessing ... fatal: HTTP request failed --- Any idea what could be wrong here? Is git client really retrying with the bad password? Regards, -Mikko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git https transport and wrong password
On Tue, Apr 02, 2013 at 06:54:40PM +0300, Mikko Rapeli wrote: I have client side logs with GIT_CURL_VERBOSE=1 but from intranet so can't publish them directly. Here's roughly what the log shows: Maybe this is simpler summary: $ grep HTTP\/1.1 log.txt GET ...info/refs?service=git-upload-pack HTTP/1.1 401 Authorization required password prompt here, and ctrl-c does not work in Cygwin, sigh. GET ...info/refs?service=git-upload-pack HTTP/1.1 401 Authorization required GET ...info/refs?service=git-upload-pack HTTP/1.1 302 Found account locked I presume GET longredirecturl GET ...info/refs HTTP/1.1 302 Found GET longredirecturl I was not able reproduce this issue using curl directly to get the info/refs page. -Mikko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git https transport and wrong password
On Tue, Apr 02, 2013 at 06:54:40PM +0300, Mikko Rapeli wrote: I have a problem with git (1.7.9 and 1.8.2.357.gcc3e4eb) and https transport to gerrit server (2.5.1-3-g719dfc7). I'm producing the problem on Cygwin but my colleagues have same issue on Linux as well. Gerrit server is matching corporate policies with single sign on, so after three failed login attempts the account gets locked until a password reset. Git amplifies this problem by asking for users password only once, and if user made a typo git is still re-using the wrong password enough times to get an account immediately locked. Hmm. The sequence should be: - request, get 401 - prompt user for password - retry request with password - if 401, die IOW, we should make only a single request with the credential, and immediately die afterwards. We do hit once to get the initial 401, but we do not even provide a username, so unless the corporate policy is locking out based on IP, it should not matter (and if it is, that shows a fundamental misunderstanding about how a 401 is supposed to work). But from your log, I see: --- $ GIT_CURL_VERBOSE=1 git fetch ... GET /gerrit/.../info/refs?service=git-upload-pack HTTP/1.1 ... HTTP/1.1 401 Authorization Required ... Here's our first 401. OK. -- I guess git prompts for password here. -- Maybe...see below. * Server auth using Basic with user '...' GET /gerrit/.../info/refs?service=git-upload-pack HTTP/1.1 Authorization: Basic ... ... HTTP/1.1 401 Authorization Required Date: ... * Authentication problem. Ignoring this. ... * The requested URL returned error: 401 We get another 401. Now git should die. But it doesn't: * STATE: PROTOCONNECT = DO handle... * Server auth using Basic with user '...' GET /gerrit/.../info/refs?service=git-upload-pack HTTP/1.1 Authorization: Basic ... It makes another request instead. Weirdly, this does not result in a 401: * STATE: DO = DO_DONE handle... * STATE: DO_DONE = WAITPERFORM handle... * STATE: WAITPERFORM = PERFORM handle... ... HTTP/1.1 302 Found ... Location: ...funnylongurl ... * Ignoring the response-body * Connection #1 to host ... left intact * Issue another request to this URL: '...funnylongurl' ... * Server auth using Basic with user '...' GET ...funnylongurl Authorization: Basic ... ... * The requested URL returned error: 500 Internal Server Error * Closing connection 1 We get redirected somewhere where we provide the (presumably wrong) credential again. I do not think that is git's fault; the server asked us to make the extra request. Is that part of the lockout procedure? If it is not, it seems odd that the server would issue a redirect for a bogus auth (shouldn't it just keep giving us 401?). I do not know what is going on with the redirection there, but I have a hunch on the extra auth round-trip. What does your remote URL look like? Does it have your username (e.g., https://user@host/project.git)? I have noticed that if curl sees such a URL, it attempts to do a password-less authentication itself, before even handing control back to git. So my above sequence would become: 1. git feeds URL to curl, who makes request 2. we get a 401 3. curl says Oh, I have a username; let me try that and re-requests 4. we get another 401, because we need a password 5. curl says that didn't work and hands control back to git 6. git requests a password from the user and gives it to curl 7. curl retries with the password, but it's wrong, so that results in a 401, too At the end of it, we've now made _two_ failed requests for user X, rather than one. I don't know if there's a way to tell curl not to try the extra user-only round-trip. But you can strip the username out of your URL to avoid it. -Peff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git https transport and wrong password
On Tue, Apr 02, 2013 at 03:28:45PM -0400, Jeff King wrote: We get redirected somewhere where we provide the (presumably wrong) credential again. I do not think that is git's fault; the server asked us to make the extra request. Is that part of the lockout procedure? If it is not, it seems odd that the server would issue a redirect for a bogus auth (shouldn't it just keep giving us 401?). I think it is supposed to be a catch all failure mode without any authentication but is just wrong/buggy. I'll try to debug these by issuing curl commands step by step. I do not know what is going on with the redirection there, but I have a hunch on the extra auth round-trip. What does your remote URL look like? Does it have your username (e.g., https://user@host/project.git)? Yes, that's the giturl format I have. I have noticed that if curl sees such a URL, it attempts to do a password-less authentication itself, before even handing control back to git. So my above sequence would become: 1. git feeds URL to curl, who makes request 2. we get a 401 3. curl says Oh, I have a username; let me try that and re-requests 4. we get another 401, because we need a password 5. curl says that didn't work and hands control back to git 6. git requests a password from the user and gives it to curl 7. curl retries with the password, but it's wrong, so that results in a 401, too At the end of it, we've now made _two_ failed requests for user X, rather than one. I don't know if there's a way to tell curl not to try the extra user-only round-trip. But you can strip the username out of your URL to avoid it. It did seem like there was just one GET and 401 return before password was promptet. I'll tripple check that. Played around with command line curl a bit and at least it did the right thing with a URL without username -- failed with 401 after single try -- and with URL without username but username provided -u 'username' which succeeded or failed on single try based on password. Don't know anything about curl but maybe git could parse the url for a username and prompt for the password before the first 401 failure roundtrip that's now in place. I guess most of this logic is in http.c. -Mikko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git https transport and wrong password
On Tue, Apr 02, 2013 at 10:47:51PM +0300, Mikko Rapeli wrote: Don't know anything about curl but maybe git could parse the url for a username and prompt for the password before the first 401 failure roundtrip that's now in place. I guess most of this logic is in http.c. We used to do that but stopped, as curl might also be able to retrieve the password from .netrc; the extra prompt was an annoyance to users in this situation. Now that we have the credential subsystem, I would recommend dropping usernames from all git-over-http URLs, and either: 1. Using a credential helper that supports secure long-term storage (osxkeychain, wincred, etc). 2. Specifying the username to the credential subsystem explicitly, by putting something like: [credential https://yourhost/;] username = yourusername in your git config. Obviously (1) is nicer, but you may have corporate policies against storing credentials. Or you may have a complicated single sign-on procedure, where the password changes. In that case, I would still say it is worth writing a custom helper script that can feed the temporary credential to git. -Peff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git https transport and wrong password
On Tue, Apr 02, 2013 at 04:05:51PM -0400, Jeff King wrote: On Tue, Apr 02, 2013 at 10:47:51PM +0300, Mikko Rapeli wrote: Don't know anything about curl but maybe git could parse the url for a username and prompt for the password before the first 401 failure roundtrip that's now in place. I guess most of this logic is in http.c. We used to do that but stopped, as curl might also be able to retrieve the password from .netrc; the extra prompt was an annoyance to users in this situation. Ok, I think I've seen this before and ended up storing passwords in .netrc. Now that we have the credential subsystem, I would recommend dropping usernames from all git-over-http URLs, and either: 1. Using a credential helper that supports secure long-term storage (osxkeychain, wincred, etc). 2. Specifying the username to the credential subsystem explicitly, by putting something like: [credential https://yourhost/;] username = yourusername in your git config. Obviously (1) is nicer, but you may have corporate policies against storing credentials. Or you may have a complicated single sign-on procedure, where the password changes. In that case, I would still say it is worth writing a custom helper script that can feed the temporary credential to git. Thanks, I'll have a look at these helpers. Policies we may have but in practice I think many just store plaintext passwords in giturls, which is obviously the worst case, but it works against accidental typos in the password prompt (though blows up when the mandatory password change comes along). -Mikko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
