On 5/1/25 1:40 AM, Willy Tarreau wrote: > Hi, Thanks for the quick response!
> On Thu, May 01, 2025 at 01:08:22AM -0400, Demi Marie Obenour wrote: >> I noticed that if HAProxy receives a message containing \x01 in a >> field value, it will happily forward that message. This appears >> to be permitted by RFC9113 and RFC9114. However, it might cause >> a problem if the message is a response, the response is sent to >> the client over HTTP/3, and the client uses nghttp3. nghttp3 will >> silently discard the field, causing HAProxy and the client to >> disagree on whether the field exists. >> >> Is this a serious concern, or is it considered too theoretical to >> matter? From my reading of >> https://github.com/ngtcp2/nghttp3/discussions/346, >> it seems that the nghttp3 maintainer considers this to be working >> as intended (a valid decision). Is there anything that should be >> be done on the HAProxy side, or is HAProxy thinking that a header >> exists when a client thinks it does not expected behavior? > > When H2 begun, some people saw a great opportunity in its ability > to be fully binary-transparent, while others (like us) dealing with > intermediaries saw a big danger in it. From what I remember (but I > could be wrong, I'd need to recheck), the spec only requires that > chars are filtered when converting from H2 to H1. And IIRC in the > end only CR, LF and NUL are forbidden in header values in H2, as > a protective measure for H2 to H1 translation, while HPACK takes > care of permitting absolutely everything. Nowadays, HTTP/1.1, HTTP/2, and HTTP/3 all use the same header field syntax, specified in RFC9110: - Allowed byte are 0x09, 0x20 ... 0x7E, and 0x80 ... 0xFF (all ranges inclusive). - 0x09 and 0x20 must not be the first or last byte. > The thing is that full H2 applications are explicitly permitted > to rely on this. For example you could imagine sending payload's > SHA256 digests directly in a header, escaping NUL with \0, LF > with \n, CR with \r and '\' with '\\' an keep it compact like > this. So while we have to perform the cleanup for H2,H3 -> H1, > we cannot afford to be excessive and go beyond the spec where > applications expect it to work as permitted by the spec. RFC9113 (HTTP/2) and RFC9114 (HTTP/3) disagree: if a field contains any bytes in the range 0x00 ... 0x08, 0x0A ... 0x1F, or 0x7F, the whole message is considered malformed. However, intermediaries (like HAProxy) are only required to detect 0x00, 0x0A, and 0x0D. RFC9113 requires that an HTTP/2 message be considered malformed if a field starts or ends with 0x09 or 0x20. I am not sure if RFC9114 requires this for HTTP/3, but I presume that is the intent. > However I find it particularly dangerous to silently discard some > headers. It's offering an invitation to a remote attacker to declare > which headers will be locally discarded. It's exactly the same as > using a Connection header that has long been abused for this. The > spec does not offer such provisions for selectively eliminating > headers. If a message is not acceptable, it must be rejected, > otherwise it must be accepted. Header fields form semantics together > and selectively accepting/dropping some of them can only result in > changing the semantics of the message. For example, in H1, > transfer-encoding has precedence over content-length. Imagine if an > agent would selectively accept transfer-encoding. This could make > one in the path rely on it and another one ignore it and rely on a > different content-length. I agree. nghttp3 is an HTTP/3 library, so desynchronization attacks are not possible, but this still seems very risky. HAProxy does refuse to append to a 'forwarded' header containing invalid characters, and HAProxy doesn't support HTTP/3 to the backend anyway. That said, I would not be surprised if there is a vulnerability somewhere. > Maybe we could start to think about having options to be stricter > than the spec (e.g. enforce HTTP/1 syntax on all versions) in order > to protect known vulnerable implementations, but I'm afraid that if > we go down that route, this becomes and endless chase... My reading (of your message, RFC9110, and RFC9112 through RFC9114) is is that the spec itself has gotten stricter over time, and nowadays it recommends (HTTP/1.x) or requires (HTTP/2 or HTTP/3) that HAProxy be as strict about field values as it already is about field names. This is the only way I know of to ensure that HAProxy always agrees with downstream and upstream interpretations of field values. Stripping of leading and/or trailing whitespace should be much less of a concern because it cannot be used to cause HAProxy to itself create a malformed message, but as per the standards invalid characters in field values must cause the whole message to be rejected. -- Sincerely, Demi Marie Obenour (she/her/hers)
OpenPGP_0xB288B55FFF9C22C1.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
