Re: Host header checking too strict?

[William A Rowe Jr]

We simply accept them, and let dns reject them when not resolvable.



On Tue, Jun 26, 2018, 14:22 Daniel Ferradal  wrote:

> This was implemented last year in 2.4.24. Much has happened and just a
> few strugglers including I, had to deal with it, it seems.
>
> I remember I mentioned the CVE at work and that seemed enough for
> everyone to accept the change and nobody proposes _ in new names since
> then. IIRC "httpprocotoloptions unsafe" could override this already.
> Hey Eric, we even had a discussion about this at IRC, remember? :)
>
> So what is the option we would offer instead?
> El mar., 26 jun. 2018 a las 0:19, Roy T. Fielding
> () escribió:
> >
> > On Jun 25, 2018, at 8:57 AM, William A Rowe Jr 
> wrote:
> >
> > On Mon, Jun 25, 2018 at 5:31 AM, Joe Orton  wrote:
> >>
> >> On Fri, Jun 22, 2018 at 05:21:08PM -0400, Eric Covener wrote:
> >> > After CVE-2016-8743 we only accept hostnames that are valid in DNS,
> >> > which notably excludes underscores.  But it seems like 7230 does not
> >> > require HTTP Host: to use a DNS registry, and excluding  '_' should
> >> > have broken IDN (punycode) international domain names.
> >> >
> >> > Meanwhile I have seen several reports of e.g. departmental servers or
> >> > proxypreservehost=off-like failures with hostnames w/ underscores.
> >> >
> >> > Should we be more tolerant here, or offer an option?
> >> >
> >> > [ ] No
> >> > [X] Just underscores, which seems to come up alot?
> >>
> >> Yup, we had Fedora users complain about this as well after 2.6.25, +1
> >> for underscores in hostnames allowed by default.
> >
> >
> > I'll concur, I see no problem "violating" the spec in this single
> respect.
> > Note that the same is not true of, say, http field names. There,
> ambiguity
> > between - and _ due to CGI is an actual problem.
> >
> >
> > The spec is at
> >
> >   https://tools.ietf.org/html/rfc3986#section-3.2.2
> >
> > and
> >
> >
> https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#header.host
> >
> > Whatever we are doing, underscore is allowed by the spec.  DNS is
> irrelevant here
> > because hostnames are not limited to DNS names.
> >
> > It is reasonable for us to limit Host to be the set of allowed virtual
> hosts we are
> > willing to match, so we can certainly exclude the weird delimiters, but
> we
> > don't want to prevent access to hosts we allow to be configured.
> >
> > BTW, note that the second link above is to the current editors' draft of
> HTTP,
> > which is being revised now.  If anyone wants to reduce the grammar here
> or
> > elsewhere, now is the time to make it an issue at
> >
> >   https://github.com/httpwg/http-core
> >
> > Cheers,
> >
> > Roy
> >
>
>
> --
> Daniel Ferradal
> HTTPD Project
> #httpd help at Freenode
>

Re: Host header checking too strict?

[Daniel Ferradal]

This was implemented last year in 2.4.24. Much has happened and just a
few strugglers including I, had to deal with it, it seems.

I remember I mentioned the CVE at work and that seemed enough for
everyone to accept the change and nobody proposes _ in new names since
then. IIRC "httpprocotoloptions unsafe" could override this already.
Hey Eric, we even had a discussion about this at IRC, remember? :)

So what is the option we would offer instead?
El mar., 26 jun. 2018 a las 0:19, Roy T. Fielding
() escribió:
>
> On Jun 25, 2018, at 8:57 AM, William A Rowe Jr  wrote:
>
> On Mon, Jun 25, 2018 at 5:31 AM, Joe Orton  wrote:
>>
>> On Fri, Jun 22, 2018 at 05:21:08PM -0400, Eric Covener wrote:
>> > After CVE-2016-8743 we only accept hostnames that are valid in DNS,
>> > which notably excludes underscores.  But it seems like 7230 does not
>> > require HTTP Host: to use a DNS registry, and excluding  '_' should
>> > have broken IDN (punycode) international domain names.
>> >
>> > Meanwhile I have seen several reports of e.g. departmental servers or
>> > proxypreservehost=off-like failures with hostnames w/ underscores.
>> >
>> > Should we be more tolerant here, or offer an option?
>> >
>> > [ ] No
>> > [X] Just underscores, which seems to come up alot?
>>
>> Yup, we had Fedora users complain about this as well after 2.6.25, +1
>> for underscores in hostnames allowed by default.
>
>
> I'll concur, I see no problem "violating" the spec in this single respect.
> Note that the same is not true of, say, http field names. There, ambiguity
> between - and _ due to CGI is an actual problem.
>
>
> The spec is at
>
>   https://tools.ietf.org/html/rfc3986#section-3.2.2
>
> and
>
>   
> https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#header.host
>
> Whatever we are doing, underscore is allowed by the spec.  DNS is irrelevant 
> here
> because hostnames are not limited to DNS names.
>
> It is reasonable for us to limit Host to be the set of allowed virtual hosts 
> we are
> willing to match, so we can certainly exclude the weird delimiters, but we
> don't want to prevent access to hosts we allow to be configured.
>
> BTW, note that the second link above is to the current editors' draft of HTTP,
> which is being revised now.  If anyone wants to reduce the grammar here or
> elsewhere, now is the time to make it an issue at
>
>   https://github.com/httpwg/http-core
>
> Cheers,
>
> Roy
>


-- 
Daniel Ferradal
HTTPD Project
#httpd help at Freenode

Re: Host header checking too strict?

[Roy T. Fielding]

> On Jun 25, 2018, at 8:57 AM, William A Rowe Jr  wrote:
> 
> On Mon, Jun 25, 2018 at 5:31 AM, Joe Orton  > wrote:
> On Fri, Jun 22, 2018 at 05:21:08PM -0400, Eric Covener wrote:
> > After CVE-2016-8743 we only accept hostnames that are valid in DNS,
> > which notably excludes underscores.  But it seems like 7230 does not
> > require HTTP Host: to use a DNS registry, and excluding  '_' should
> > have broken IDN (punycode) international domain names.
> > 
> > Meanwhile I have seen several reports of e.g. departmental servers or
> > proxypreservehost=off-like failures with hostnames w/ underscores.
> > 
> > Should we be more tolerant here, or offer an option?
> > 
> > [ ] No
> > [X] Just underscores, which seems to come up alot?
> 
> Yup, we had Fedora users complain about this as well after 2.6.25, +1 
> for underscores in hostnames allowed by default.
> 
> I'll concur, I see no problem "violating" the spec in this single respect.
> Note that the same is not true of, say, http field names. There, ambiguity
> between - and _ due to CGI is an actual problem.

The spec is at

  https://tools.ietf.org/html/rfc3986#section-3.2.2 


and 

  
https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#header.host
 


Whatever we are doing, underscore is allowed by the spec.  DNS is irrelevant 
here
because hostnames are not limited to DNS names.

It is reasonable for us to limit Host to be the set of allowed virtual hosts we 
are
willing to match, so we can certainly exclude the weird delimiters, but we
don't want to prevent access to hosts we allow to be configured.

BTW, note that the second link above is to the current editors' draft of HTTP,
which is being revised now.  If anyone wants to reduce the grammar here or
elsewhere, now is the time to make it an issue at

  https://github.com/httpwg/http-core 

Cheers,

Roy

Re: Host header checking too strict?

[William A Rowe Jr]

On Mon, Jun 25, 2018 at 5:31 AM, Joe Orton  wrote:

> On Fri, Jun 22, 2018 at 05:21:08PM -0400, Eric Covener wrote:
> > After CVE-2016-8743 we only accept hostnames that are valid in DNS,
> > which notably excludes underscores.  But it seems like 7230 does not
> > require HTTP Host: to use a DNS registry, and excluding  '_' should
> > have broken IDN (punycode) international domain names.
> >
> > Meanwhile I have seen several reports of e.g. departmental servers or
> > proxypreservehost=off-like failures with hostnames w/ underscores.
> >
> > Should we be more tolerant here, or offer an option?
> >
> > [ ] No
> > [X] Just underscores, which seems to come up alot?
>
> Yup, we had Fedora users complain about this as well after 2.6.25, +1
> for underscores in hostnames allowed by default.
>

I'll concur, I see no problem "violating" the spec in this single respect.
Note that the same is not true of, say, http field names. There, ambiguity
between - and _ due to CGI is an actual problem.

Re: Host header checking too strict?

[Joe Orton]

On Fri, Jun 22, 2018 at 05:21:08PM -0400, Eric Covener wrote:
> After CVE-2016-8743 we only accept hostnames that are valid in DNS,
> which notably excludes underscores.  But it seems like 7230 does not
> require HTTP Host: to use a DNS registry, and excluding  '_' should
> have broken IDN (punycode) international domain names.
> 
> Meanwhile I have seen several reports of e.g. departmental servers or
> proxypreservehost=off-like failures with hostnames w/ underscores.
> 
> Should we be more tolerant here, or offer an option?
> 
> [ ] No
> [X] Just underscores, which seems to come up alot?

Yup, we had Fedora users complain about this as well after 2.6.25, +1 
for underscores in hostnames allowed by default.

Regards, Joe

AW: Host header checking too strict?

[Plüm , Rüdiger , Vodafone Group]

> -Ursprüngliche Nachricht-
> Von: Eric Covener 
> Gesendet: Freitag, 22. Juni 2018 23:21
> An: Apache HTTP Server Development List 
> Betreff: Host header checking too strict?
> 
> After CVE-2016-8743 we only accept hostnames that are valid in DNS,
> which notably excludes underscores.  But it seems like 7230 does not
> require HTTP Host: to use a DNS registry, and excluding  '_' should
> have broken IDN (punycode) international domain names.
> 
> Meanwhile I have seen several reports of e.g. departmental servers or
> proxypreservehost=off-like failures with hostnames w/ underscores.
> 
> Should we be more tolerant here, or offer an option?
> 
> [ ] No
> [X] Just underscores, which seems to come up alot?

Regards

Rüdiger

Re: Host header checking too strict?

[Yann Ylavic]

On Sat, Jun 23, 2018 at 12:16 AM, William A Rowe Jr  wrote:
>
> (Sub-delims have all sorts of problematic designations, we really want
> to accept a "wildcard" '*' hostname? I'd suggest keep to the known
> "unwise" exceptions, and leave it part of the "unsafe" protocol behavior.)

Marking underscores "unsafe", with the current all or nothing
granularity, would be worse than not allowing them IMHO.
It should be either a dedicated setting (opt out if you ask me), or I
think even hardcoded-ly tolerated.

Regarding "unsafe" sub-delims, is there any need?

Re: Host header checking too strict?

[Yann Ylavic]

On Fri, Jun 22, 2018 at 11:21 PM, Eric Covener  wrote:
>
> [X] Just underscores, which seems to come up alot?

Until other complains, I've never heard of any other so far.

Re: Host header checking too strict?

[William A Rowe Jr]

On Fri, Jun 22, 2018 at 5:13 PM, William A Rowe Jr 
wrote:

> On Fri, Jun 22, 2018 at 4:42 PM, Eric Covener  wrote:
>
>> > should have broken IDN (punycode) international domain names.
>>
>> those are obviously dashes, not underscores, so not affected at all.
>>
>
> That assertion was a bit extreme :) But on principal, underbars are not
> valid (internet) DNS, but seem widely deployed in the intranet, notably
> a certain M$ who has actively endorsed and promoted them to sysadmins.
>
> I'd say let them pass, in that it shouldn't matter if a_b isn't resolvable
> any more or less than axb is resolvable. There is no ambiguity in the
> designation that I'm aware of.
>

(Sub-delims have all sorts of problematic designations, we really want
to accept a "wildcard" '*' hostname? I'd suggest keep to the known
"unwise" exceptions, and leave it part of the "unsafe" protocol behavior.)

Re: Host header checking too strict?

[William A Rowe Jr]

On Fri, Jun 22, 2018 at 4:42 PM, Eric Covener  wrote:

> > should have broken IDN (punycode) international domain names.
>
> those are obviously dashes, not underscores, so not affected at all.
>

That assertion was a bit extreme :) But on principal, underbars are not
valid (internet) DNS, but seem widely deployed in the intranet, notably
a certain M$ who has actively endorsed and promoted them to sysadmins.

I'd say let them pass, in that it shouldn't matter if a_b isn't resolvable
any more or less than axb is resolvable. There is no ambiguity in the
designation that I'm aware of.

Re: Host header checking too strict?

[Eric Covener]

> should have broken IDN (punycode) international domain names.

those are obviously dashes, not underscores, so not affected at all.

Host header checking too strict?

[Eric Covener]

After CVE-2016-8743 we only accept hostnames that are valid in DNS,
which notably excludes underscores.  But it seems like 7230 does not
require HTTP Host: to use a DNS registry, and excluding  '_' should
have broken IDN (punycode) international domain names.

Meanwhile I have seen several reports of e.g. departmental servers or
proxypreservehost=off-like failures with hostnames w/ underscores.

Should we be more tolerant here, or offer an option?

[ ] No
[ ] Just underscores, which seems to come up alot?
[ ] all of reg-name? https://tools.ietf.org/html/rfc3986#section-3.2.2
   reg-name= *( unreserved / pct-encoded / sub-delims )
   unreserved= ALPHA / DIGIT / "-" / "." / "_" / "~"
   sub-delims= "!" / "$" / "&" / "'" / "(" / ")"
 / "*" / "+" / "," / ";" / "="

I am tempted on underscores by default, but all of reg-name looks a
like a lot that I have never seen someone report any of the other
chars.  I certainly would not want & coming back in by default.

--
Eric Covener
cove...@gmail.com