Bug#682048: wget corrupts ssl chunked transfer-encoded downloads

2012-07-19 Thread Russell Stuart
Package: wget
Version: 1.12-2.1
Severity: important


After running this command:

  $ wget --no-check-certificate 
https://www.lubemobile.com.au:2078/backup-7.19.2012_00-21-23_lubemobi.tar.gz 
--user=redacted --password='redacted'

The file was downloaded was corrupt (it wasn't a valid gzip file).
Looking at it revealed this:

  $ hd backup-7.19.2012_00-21-23_lubemobi.tar.gz | sed 1q
  00 31 66 66 66 65 0d 0a 1f 8b 08 00 93 99 07 50 00  1fffe.P.

The leading 1fffe\r\n was suspicious.  It looks like the content length
from a chunked http transfer (ie http header Transfer-Encoding: chunked).
Sure enough, the 1fffe\r\n insertion was repeated on every
0x1fffe+7==0x20007 byte byte boundary:

  $ hd xxx | sed -n -e 8193p -e 16385,16386p -e 24578p
  02 1a 5a f7 ae e4 0d 0a 31 66 66 66 65 0d 0a d7 59  .Z.1fffe...Y
  04 6c 57 9d f7 b0 f2 90 42 7c da bd 95 0d 0a 31 66  lW.B|.1f
  040010 66 66 65 0d 0a 03 99 f5 a5 dc 56 64 7c 9f 85 f8  ffe...Vd|...
  060010 19 e6 47 0d 0a 31 66 66 66 65 0d 0a 8f a2 4a f0  ..G..1fffeJ.

MITM'ed the https connection to verify this was in fact the case:
  000 c connect(('www.lubemobile.com.au', 2078))
  000 s 'CONNECT www.lubemobile.com.au:2078 HTTP/1.0\r\n'
  000 s 'User-Agent: Wget/1.12 (linux-gnu)\r\n'
'\r\n'
  000 r 'HTTP/1.0 200 Connection established\r\n'
  000 r '\r\n'
  000 S 'GET /backup-7.19.2012_00-21-23_lubemobi.tar.gz HTTP/1.0\r\n'
  000 S 'User-Agent: Wget/1.12 (linux-gnu)\r\n'
'Accept: */*\r\n'
'Host: www.lubemobile.com.au:2078\r\n'
'\r\n'
  000 R 'HTTP/1.1 401 Unauthorized\r\n'
  000 R 'Date: Thu, 19 Jul 2012 05:56:11 GMT\r\n'
'Server: cPanel\r\n'
'Content-Length: 23\r\n'
'Connection: Keep-Alive\r\n'
'WWW-Authenticate: Basic realm=cPanel WebDisk\r\n'
'Content-Type: text/plain\r\n'
'\r\n'
  000 R 'Authentication Required'
  000 C close()
  000 S recv() EOF
  001 c connect(('www.lubemobile.com.au', 2078))
  001 s 'CONNECT www.lubemobile.com.au:2078 HTTP/1.0\r\n'
  001 s 'User-Agent: Wget/1.12 (linux-gnu)\r\n'
'\r\n'
  001 r 'HTTP/1.0 200 Connection established\r\n'
  001 r '\r\n'
  001 S 'GET /backup-7.19.2012_00-21-23_lubemobi.tar.gz HTTP/1.0\r\n'
  001 S 'User-Agent: Wget/1.12 (linux-gnu)\r\n'
'Accept: */*\r\n'
'Host: www.lubemobile.com.au:2078\r\n'
'Authorization: Basic redacted\r\n'
'\r\n'
  001 R 'HTTP/1.1 200 OK\r\n'
  001 R 'Date: Thu, 19 Jul 2012 05:56:15 GMT\r\n'
'Server: cPanel\r\n'
'Connection: Keep-Alive\r\n'
'Transfer-Encoding: chunked\r\n'
'Accept-Ranges: bytes\r\n'
'Content-Type: application/download\r\n'
'Last-Modified: Thu, 19 Jul 2012 05:22:46 GMT\r\n'
'\r\n'
  001 R 
'1fffe\r\n\x1f\x8b\x08\x00\x93\x99\x07P\x00\x03\xec\xfd]\xb7\xa6\xd7u\x9f\xf9\xe9T\xfc\x14\x15\x8d\x1c\xf4\x1b\x80g\xceu\xbf-6\xcd\x8e\xdaV\x92\x1e\xc3\x96\x1dK\xdd\xa3;\'\x1c
 
Q4\x11\x81\x00\x03\x80\xa2\x95\x83|\xf6\xcc\xfb\x99\x0f6!\xadK\x12.*\r\xabm\x949,\x89\x00/\x02\xab\xd6\xc2\x7fW\xd5\xde\xbf\xfa\xf9\xc7\xbf\xf8\xab\xdf\xfe\xe6\x83\xf3\xc3\x98\x1f\xe6#\xf2g\x8f\xc7\x07\x19\x1f\xe4\xf8\xd9g\xbf\xfd\xf9\xfb_\x7f\xf1\xf3O?\xfa\xa3\x7f\xea\xb7G};\xf7\xc7\xf3\x7f\xd6\xb7\xfb\x7f\xc6\xd8\xcfo\xfe\xef\xe7\xbf\x17Y\xff\xe6\x19\xdb\x91\xe3\x8f\x1e\xb1o[\xfc\xd1\xbb\xfd\x9f\xfc\xdf\xfc\x1d\xbe\xfd\xf6\xab\xaf?\xfe\xf2\xdd\xbb?\xfa\xf2\x8b/\xbe\xfe\x87\xfe\xbco\x8e\xe3\xfb\xf8k\xfa\x1e\xbf\xfd\xfc\x1f\xff\xfe\xff\xea\xab\xcf~\xf1\xfe\xcb\xaf\xbf\xfa\x83/\xc2\xf3\xfb\xff\xf1\xb7\xbe\xff\xeb;;\xf1\xfb\x7f\x7f~\xff\x1f\xfbq\xfc3\xfb\xfe\xff\xdd\xaf\xde\xbf\xff\xec\xfb\xf8\x0b\xfa~\xbf}\x87\xef\xff\xcf\xbf\xf8\xf9\x17\x9f\xfc\xcd/?\xfd\xec\xfdW\x7f\xd8\x7f\xc7\xfd\x1d||\xc7\xe
 
f\xff#\xb6?\xba\xff\x97}\xff\xa3w\x8f\xff\xff\xfe\xad\xf2\xb7\x1f\xbe\xff\xff\xb1\xef\xff_\xbf\xff\xfa\xe3\x7f\xd2\x08\x98\xf7\xdf\xdf\xff\xfb\xb1\x8f\x1f\xde\xff\xf7\xf1\xed\xbb~\xff\x7f\xf2\xf3_\x7f\xfc\x9b\x0f\xff\xe6\xe3_\xff\x01g\xf0|\xff\xdb\xf6\xb7\xbf\xff\x7f\xff?\x1f\xfb\xbe\xfdQ\xc4v\xea;?\x8f\xfb\x9f\xff\xe7\x9e\xdb?\xb3\xf7\xff\x8f\xfd\xf1\xff\x83~\xfb\xe0\x83\x0f\xde\xfd\xe8\xdf\xfco\x7f\xf1\xff\xf8\xd7?~\xf7\xa3w\xef\xf9\xf9W\xcf\xff\xf9\xee\xdd7\xdf\xfd?\xfb\xf9g_\xfc\x87\x1f\xbf\x8bs\xfb\xb0\xde\xe8\x87\xb9\x8d\x0f\xe3\xc8\xbf\xfd\xa7|\xf3\xbf|\xf6\x9e\xfe\xc4O~\xfe\xdb\xaf\xde\x7f\xf9\xf7t\x9f\xff\xde\xb7\xff{\xbf\xe3\x7f\xf7\xbbw\xd5\xfc\xeb\xf7_~\xb7\xbf\xb2\x7f\xec\xbf\xe7\x1f\xfe\x1b\xf8\x87\xff\xdb\xbe\xf8\xdd\xe7\xf7\x1f\xf8\xf1\xa3\xbf\xff\xcf\xfdO\xfd}M\xdf\xbe\xeb\xfb\xff\xf5\xc7\x9f~\xd6\x7f_\xfe\xbf\x03\xdf\xff\xb7\xff\xf9\x1f\x7f\xf7\xe3\xbf\xf3\xdc\xce\x7ff\xef\xff?\xd3\x7f\xfe\x7f\xf2\xc5_\xbf\xff\xc5\x17_\xff\xb3\xbc\x9b?|\xfb\xdf\xff\xdbw}
 

Bug#682048: wget corrupts ssl chunked transfer-encoded downloads

2012-07-19 Thread Micah Cowan
On 07/18/2012 11:23 PM, Russell Stuart wrote:
 Package: wget
 Version: 1.12-2.1

Wget 1.12 doesn't support HTTP/1.1. It's inappropriate for the server to
send chunked content in response to an HTTP/1.0 request.

For HTTP/1.1 and chunked support, try a newer version of Wget, such as
1.13.4 (available from unstable).

-mjc


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org