Bug#1063342: libcurl now rejects HTTP/1.0 responses to HEAD containing body

2024-02-06 Thread Daniel Stenberg

On Tue, 6 Feb 2024, Ian Jackson wrote:

See https://github.com/curl/curl/pull/12842

--

 / daniel.haxx.se



Bug#1063342: libcurl now rejects HTTP/1.0 responses to HEAD containing body

2024-02-06 Thread Ian Jackson
Package: libcurl3-gnutls
Version: 8.6.0-1

tl;dr: I found a regression in bug-compatibility but I have no idea if
   it should be considered a problem.

Hi.

I investigated the failing dgit autopkgtest, which is (at leasat one
of the reasons) preventing src:curl from migrating.

I found that the root cause was that dgit's test suite has a stunt
http server which mishandles HTTP HEAD requests: it doesn't look at
the request method at all, so it responds to HEAD the same as GET,
with a body.  So that is wrong.

The new libcurl rejects this, with a "Weird server reply" error.

I have filed the bug in the test case's stunt httpd as #1063341 (with
severity serious) and we will fix it in src:dgit soon.

However, I wonder whether this behavioural change in curl is
intentional or desirable.  It seems to me that it might pose a
compatibility hazard.  I know that compatibility, even with broken
peers, is often important in the web space.

I haven't tested the behaviour with HTTP/1.1.  HTTP/1.1 has different
framing arrangements: depending on the framing, a similar bug in a
server would result in a framing error so such a buggy server wouldn't
survive.  But with HTTP/1.0, a response which erroneously includes the
body is unambiguous and parseable.

I don't know if HTTP/1.0 is common enough, and compatibility with such
buggy HTTP servers important enough, to be concerned.  I thought I
would file this bug to inform you about the situation and let you
decide.  I hope you find that helpful.

Please downgrade, close, or forward to upstream, or upgrade, this bug,
as seems appropriate.

Thanks for your attention and your maintenance of this critical
package.

Regards,
Ian.

30178 read(7, "H", 1)   = 1
 | 0  48H|
30178 read(7, "E", 1)   = 1
 | 0  45E|
30178 read(7, "A", 1)   = 1
 | 0  41A|
30178 read(7, "D", 1)   = 1
 | 0  44D|
30178 read(7, " ", 1)   = 1
 | 0  20 |
30178 read(7, "/", 1)   = 1
 | 0  2f/|
30178 read(7, "p", 1)   = 1
 | 0  70p|
...
30178 write(7, "HTTP/1.0 404 Not found\r\nContent-Type: text/html; 
charset=ISO-8859-1\r\n\r\nhttp://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\;>\nhttp://www.w3.org/1999/xhtml\; lang=\"en-US\" 
xml:lang=\"en-US\">\n\nNot found\n\n\n\nNot found\n\n", 426) = 426
 | 0  48 54 54 50 2f 31 2e 30  20 34 30 34 20 4e 6f 74  HTTP/1.0 404 Not |
 | 00010  20 66 6f 75 6e 64 0d 0a  43 6f 6e 74 65 6e 74 2d   found..Content- |
 | 00020  54 79 70 65 3a 20 74 65  78 74 2f 68 74 6d 6c 3b  Type: text/html; |
 | 00030  20 63 68 61 72 73 65 74  3d 49 53 4f 2d 38 38 35   charset=ISO-885 |
 | 00040  39 2d 31 0d 0a 0d 0a 3c  21 44 4f 43 54 59 50 45  9-1.http://www.w3.o |
 | 000e0  72 67 2f 31 39 39 39 2f  78 68 74 6d 6c 22 20 6c  rg/1999/xhtml" l |
 | 000f0  61 6e 67 3d 22 65 6e 2d  55 53 22 20 78 6d 6c 3a  ang="en-US" xml: |
 | 00100  6c 61 6e 67 3d 22 65 6e  2d 55 53 22 3e 0a 3c 68  lang="en-US">..Not  |
 | 00120  66 6f 75 6e 64 3c 2f 74  69 74 6c 65 3e 0a 3c 6d  found.. |
 | 00180  0a 3c 62 6f 64 79 3e 0a  3c 68 31 3e 4e 6f 74 20  ..Not  |
 | 00190  66 6f 75 6e 64 3c 2f 68  31 3e 0a 3c 2f 62 6f 64  found..   |
30178 close(7)  = 0

...

dgit: error: fetch of http://127.0.0.1:40339/pari-extra.git/HEAD failed (Weird 
server reply):

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.