Re: [PATCH 5/6] http: treat http-alternates like redirects

2016-12-01 Thread Jeff King
On Thu, Dec 01, 2016 at 03:02:23PM -0800, Brandon Williams wrote:

> > diff --git a/http.c b/http.c
> > index 825118481..051fe6e5a 100644
> > --- a/http.c
> > +++ b/http.c
> > @@ -745,6 +745,7 @@ static CURL *get_curl_handle(void)
> > if (is_transport_allowed("ftps"))
> > allowed_protocols |= CURLPROTO_FTPS;
> > curl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);
> > +   curl_easy_setopt(result, CURLOPT_PROTOCOLS, allowed_protocols);
> >  #else
> > if (transport_restrict_protocols())
> > warning("protocol restrictions not applied to curl redirects 
> > because\n"
> 
> Because I don't know much about how curl worksOnly
> http/https/ftp/ftps protocols are allowed to be passed to curl?  Is that
> because curl only understands those particular protocols?

No, curl understands more protocols, and that is exactly the problem. We
don't want to accidentally have curl access file://, smtp://, or
similar, based on what some server puts in their http-alternates file.

You should only be able to get to this code-path by calling one of
git-remote-{http,https,ftp,ftps}. So there is no problem with
restricting the protocol beyond those options. And there should be no
problem with restricting within that set; if the protocol we intend to
feed to curl had been disallowed by policy, git would have blocked it
before hitting git-remote in the first place.

-Peff


Re: [PATCH 5/6] http: treat http-alternates like redirects

2016-12-01 Thread Brandon Williams
On 12/01, Jeff King wrote:
>   - set CURLOPT_PROTOCOLS alongside CURLOPT_REDIR_PROTOCOLS
> restrict ourselves to a known-safe set and respect any
> user-provided whitelist.



> diff --git a/http.c b/http.c
> index 825118481..051fe6e5a 100644
> --- a/http.c
> +++ b/http.c
> @@ -745,6 +745,7 @@ static CURL *get_curl_handle(void)
>   if (is_transport_allowed("ftps"))
>   allowed_protocols |= CURLPROTO_FTPS;
>   curl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);
> + curl_easy_setopt(result, CURLOPT_PROTOCOLS, allowed_protocols);
>  #else
>   if (transport_restrict_protocols())
>   warning("protocol restrictions not applied to curl redirects 
> because\n"

Because I don't know much about how curl worksOnly
http/https/ftp/ftps protocols are allowed to be passed to curl?  Is that
because curl only understands those particular protocols?

-- 
Brandon Williams


[PATCH 5/6] http: treat http-alternates like redirects

2016-12-01 Thread Jeff King
The previous commit made HTTP redirects more obvious and
tightened up the default behavior. However, there's another
way for a server to ask a git client to fetch arbitrary
content: by having an http-alternates file (or a regular
alternates file, which is used as a backup).

Similar to the HTTP redirect case, a malicious server can
claim to have refs pointing at object X, return a 404 when
the client asks for X, but point to some other URL via
http-alternates, which the client will transparently fetch.
The end result is that it looks from the user's perspective
like the objects came from the malicious server, as the
other URL is not mentioned at all.

Worse, because we feed the new URL to curl ourselves, the
usual protocol restrictions do not kick in (neither curl's
default of disallowing file://, nor the protocol
whitelisting in f4113cac0 (http: limit redirection to
protocol-whitelist, 2015-09-22).

Let's apply the same rules here as we do for HTTP redirects.
Namely:

  - unless http.followRedirects is set to "always", we will
not follow remote redirects from http-alternates (or
alternates) at all

  - set CURLOPT_PROTOCOLS alongside CURLOPT_REDIR_PROTOCOLS
restrict ourselves to a known-safe set and respect any
user-provided whitelist.

  - mention alternate object stores on stderr so that the
user is aware another source of objects may be involved

The first item may prove to be too restrictive. The most
common use of alternates is to point to another path on the
same server. While it's possible for a single-server
redirect to be an attack, it takes a fairly obscure setup
(victim and evil repository on the same host, host speaks
dumb http, and evil repository has access to edit its own
http-alternates file).

So we could make the checks more specific, and only cover
cross-server redirects. But that means parsing the URLs
ourselves, rather than letting curl handle them. This patch
goes for the simpler approach. Given that they are only used
with dumb http, http-alternates are probably pretty rare.
And there's an escape hatch: the user can allow redirects on
a specific server by setting http..followRedirects to
"always".

Reported-by: Jann Horn 
Signed-off-by: Jeff King 
---
 http-walker.c  |  8 +---
 http.c |  1 +
 t/t5550-http-fetch-dumb.sh | 38 ++
 3 files changed, 44 insertions(+), 3 deletions(-)

diff --git a/http-walker.c b/http-walker.c
index 0b2425531..25a8b1ad4 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -274,9 +274,8 @@ static void process_alternates_response(void *callback_data)
struct strbuf target = STRBUF_INIT;
strbuf_add(&target, base, serverlen);
strbuf_add(&target, data + i, posn - i - 7);
-   if (walker->get_verbosely)
-   fprintf(stderr, "Also look at %s\n",
-   target.buf);
+   warning("adding alternate object store: %s",
+   target.buf);
newalt = xmalloc(sizeof(*newalt));
newalt->next = NULL;
newalt->base = strbuf_detach(&target, NULL);
@@ -302,6 +301,9 @@ static void fetch_alternates(struct walker *walker, const 
char *base)
struct alternates_request alt_req;
struct walker_data *cdata = walker->data;
 
+   if (http_follow_config != HTTP_FOLLOW_ALWAYS)
+   return;
+
/*
 * If another request has already started fetching alternates,
 * wait for them to arrive and return to processing this request's
diff --git a/http.c b/http.c
index 825118481..051fe6e5a 100644
--- a/http.c
+++ b/http.c
@@ -745,6 +745,7 @@ static CURL *get_curl_handle(void)
if (is_transport_allowed("ftps"))
allowed_protocols |= CURLPROTO_FTPS;
curl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS, allowed_protocols);
+   curl_easy_setopt(result, CURLOPT_PROTOCOLS, allowed_protocols);
 #else
if (transport_restrict_protocols())
warning("protocol restrictions not applied to curl redirects 
because\n"
diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh
index 532507b7c..264a1ab8b 100755
--- a/t/t5550-http-fetch-dumb.sh
+++ b/t/t5550-http-fetch-dumb.sh
@@ -330,5 +330,43 @@ test_expect_success 'http.followRedirects defaults to 
"initial"' '
test_must_fail git clone $HTTPD_URL/redir-objects/repo.git default
 '
 
+# The goal is for a clone of the "evil" repository, which has no objects
+# itself, to cause the client to fetch objects from the "victim" repository.
+test_expect_success 'set up evil alternates scheme' '
+   victim=$HTTPD_DOCUMENT_ROOT_PATH/victim.git &&
+   git init --bare "$victim" &&
+   git -C "$victim" --work-tree=.