I've narrowed down what I think is a bug in cookie handling related to http authorization digests in the last three versions of wget. I found that 1.9.1 and earlier does the right thing, but that all later versions do not.
What I am doing, is a request like this: wget -O usb.tivo --http-user=tivo --http-passwd=2807747290 "http://192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960"; What I get at the console that issued this command is (using 1.10.2): /usr/local/src/wget-1.10.2/src/wget -O usb.tivo --http-user=tivo --http-passwd=2807747290 "http://192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960"; --07:41:48-- http://1192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960 => `usb.tivo' Connecting to 192.168.23.33:80... connected. HTTP request sent, awaiting response... 401 Digest Authorization Required Reusing existing connection to 192.168.23.33:80. HTTP request sent, awaiting response... 400 Bad Request 07:41:48 ERROR 400: Bad Request. Here's what 1.9.1 does: /usr/local/src/wget-1.9.1/src/wget -O usb.tivo --http-user=tivo --http-passwd=2807747290 "http://192.168.23.33:80/download/foo.TiVo?Container=%2FNowPlaying&id=15960"; --07:44:29-- http://192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960 => `usb.tivo' Connecting to 192.168.23.33:80... connected. HTTP request sent, awaiting response... 401 Digest Authorization Required Connecting to 192.168.23.33:80... connected. HTTP request sent, awaiting response... 200 File Follows Length: unspecified [video/x-tivo-mpeg] [ <=> ] 1,711,782 799.11K/s So, 1.9.1 works and 1.10.2 does not. That doesn't mean wget has a problem, it could be that my server is taking advantage of a bug in 1.9.1, however, looking at the server logs, I see the problem. The problem is that 1.10 and later are not sending the cookies back up, even if I add --enable-digest at build time and --keep-session-cookies on the cmd line. (The cookie is "sid" that gets ignored in 1.10.2.) Server logs for 1.9.1: Oct 21 14:48:25 10 [TvHttp:80:51][5477]: REQUEST: 212 bytes: GET /download/foo.TiVo?Container=%2FNowPlaying&id=15960 HTTP/1.0^M User-Agent: Wget/1.9.1^M Host: 192.168.23.33^M Accept: */*^M Connection: Keep-Alive^M Authorization: Basic dGl2bzoyODA3NzQ3Mjkw^M ^M Oct 21 14:48:25 10 [TvHttp:80:51][5477]: REPLY: 371 bytes, HTTP/1.1 401 Digest Authorization Required^M Server: tivo-httpd-1:b-7-2/2005.10.10-0007:6F9:alpha^M Set-Cookie: sid=7DB0C3425AAA0253; path=/; expires="Saturday, 16-Feb-2013 00:00:00 GMT";^M WWW-Authenticate: Digest realm="TiVo DVR", peg: A/V Filter is off nonce="3F2C4DAAA0EF594D", qop="auth"^M Content-Length: 38^M Content-Type: text/html^M Connection: keep-alive^M Keep-Alive: max=10, timeout=30^M ^M Oct 21 14:48:25 10 [TvHttp:80:52][5477]: REQUEST: 406 bytes: GET /download/foo.TiVo?Container=%2FNowPlaying&id=15960 HTTP/1.0^M User-Agent: Wget/1.9.1^M Host: 192.168.23.33^M Accept: */*^M Connection: Keep-Alive^M Cookie: sid=7DB0C3425AAA0253^M Authorization: Digest username="tivo", realm="TiVo DVR", nonce="3F2C4DAAA0EF594D", uri="/download/foo.TiVo?Container=%2FNowPlaying&id=15960", response="326cd21ece1639b3f273099803b29382"^M ^M Oct 21 14:48:25 10 TvHttpDownloadModule[5477]: download sid=7DB0C34257FC0253, rid=15960, off=0 Server logs for 1.10.2: Oct 21 14:49:31 10 [TvHttp:80:53][5477]: REQUEST: 213 bytes: GET /download/foo.TiVo?Container=%2FNowPlaying&id=15960 HTTP/1.0^M User-Agent: Wget/1.10.2^M Accept: */*^M Authorization: Basic dGl2bzoyODA3NzQ3Mjkw^M Host: 192.168.23.33^M Connection: Keep-Alive^M ^M Oct 21 14:49:31 10 [TvHttp:80:53][5477]: REPLY: 371 bytes, HTTP/1.1 401 Digest Authorization Required^M Server: tivo-httpd-1:b-7-2/2005.10.10-0007:6F9:alpha^M Set-Cookie: sid=6D3223D6865F2836; path=/; expires="Saturday, 16-Feb-2013 00:00:00 GMT";^M WWW-Authenticate: Digest realm="TiVo DVR", nonce="9749DE695AAAE83A", qop="auth"^M Content-Length: 38^M Content-Type: text/html^M Connection: keep-alive^M Keep-Alive: max=10, timeout=30^M ^M Oct 21 14:49:31 10 [TvHttp:80:53][5477]: REQUEST: 377 bytes: GET /download/foo.TiVo?Container=%2FNowPlaying&id=15960 HTTP/1.0^M User-Agent: Wget/1.10.2^M Accept: */*^M Authorization: Digest username="tivo", realm="TiVo DVR", nonce="9749DE695AAAE83A", uri="/download/foo.TiVo?Container=%2FNowPlaying&id=15960", response="56bdebe5207862c929bc79e11816caa0"^M Host: 192.168.23.33^M Connection: Keep-Alive^M ^M Oct 21 14:49:31 10 [TvHttp:80:53][5477]: REPLY: 244 bytes, HTTP/1.1 400 Bad Request^M Server: tivo-httpd-1:b-7-2/2005.10.10-0007:6F9:alpha^M Set-Cookie: sid=6D3223D6865F2836; path=/; expires="Saturday, 16-Feb-2013 00:00:00 GMT";^M Content-Length: 39^M TiVo-Message: session id missing^M Connection: close^M ^M It looks like the change that caused the problem was making session cookies not used by default, and that the flag to re-enable them doesn't work. Here's the changelog entry that looks like when it was introduced: 2003-11-05 Hrvoje Niksic <[EMAIL PROTECTED]> * cookies.c (save_cookies_mapper): Respect the setting of keep-session-cookies. (cookie_jar_load): Import session cookies. Based on code submitted by Nicolas Schodet. * utils.c (datetime_str): Use information in TM when it's non-NULL. * main.c (main): New option `--keep-session-cookies'. Looking in the source tree, it looks like cookies.c actually does what the changelog entry says. Unless the capture (recognition) of cookies is where the problem is. triagegate:/usr/local/src/wget-1.10.2/src # grep keep-session-cookies * ChangeLog: keep-session-cookies. ChangeLog: * main.c (main): New option `--keep-session-cookies'. cookies.c: user specified `--keep-session-cookies' in the past. cookies.c: the user has specified `keep-session-cookies' again. */ main.c: { "keep-session-cookies", 0, OPT_BOOLEAN, "keepsessioncookies", -1 }, main.c: --keep-session-cookies load and save session (non-permanent) cookies.\n"), Binary file main.o matches Binary file wget matches triagegate:/usr/local/src/wget-1.10.2/src # grep keepsessioncookies * init.c: { "keepsessioncookies", &opt.keep_session_cookies, cmd_boolean }, Binary file init.o matches main.c: { "keep-session-cookies", 0, OPT_BOOLEAN, "keepsessioncookies", -1 }, Binary file main.o matches Binary file wget matches triagegate:/usr/local/src/wget-1.10.2/src # grep keep_session_cookies * cookies.c: if (!cookie->permanent && !opt.keep_session_cookies) init.c: { "keepsessioncookies", &opt.keep_session_cookies, cmd_boolean }, options.h: int keep_session_cookies; /* whether session cookies should be
