Hi,> Wget fails to download pages from Adam Curtis' archived blog, such as this:
This is expected since the server answers with 404 instead of 200 and wget dicards any content in this case (by default).
To do store the content on error, you have to add `--content-on-error` on the command line.
Cheers, Tim On 8/22/26 13:24, [email protected] wrote:
Wget fails to download pages from Adam Curtis' archived blog, such as this: https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 But Curl succeeds. This may be because it is re-encoding the % symbols. output of wget -d:22.122345.635 Local URI encoding = 'UTF-8' 22.122345.635 Input URI encoding = 'UTF-8' 22.122345.635 Fetched HSTS data from '/home/miles/.local/share/wget/.wget-hsts' 22.122345.635 Fetched HPKP data from '/home/miles/.local/share/wget/.wget-hpkp' 22.122345.635 set_exit_status(0) 22.122345.635 path webarchive/https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9 -> 22.122345.635 webarchive/https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9 22.122345.635 local filename = 'https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9' 22.122345.635 host_add_job: job fname https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9 22.122345.635 host_add_job: 0x5622fb64ec70 https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 22.122345.635 host_add_job: qsize 1 host-qsize=1 22.122345.635 queue_size: qsize=1 22.122345.635 queue_size: qsize=1 22.122345.635 queue_size: qsize=1 22.122345.635 [0] action=1 pending=0 host=0x0 goaway=n/a 22.122345.635 dequeue job https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 22.122345.635 resolving www.bbc.co.uk:443... 22.122345.643 has 212.58.237.1:443 22.122345.643 has 212.58.236.129:443 22.122345.643 has 212.58.237.129:443 22.122345.643 has 212.58.235.1:443 22.122345.643 has 212.58.235.129:443 22.122345.643 has 212.58.236.1:443 22.122345.643 has 212.58.237.129:443 22.122345.643 has 212.58.236.1:443 22.122345.643 has 212.58.235.1:443 22.122345.643 has 212.58.237.1:443 22.122345.643 has 212.58.235.129:443 22.122345.643 has 212.58.236.129:443 22.122345.643 trying 212.58.237.1:443... 22.122345.659 GnuTLS init 22.122345.835 GnuTLS system certificate store is empty 22.122345.835 Certificates loaded: 388 22.122345.835 GnuTLS init done 22.122345.836 TLS False Start requested 22.122345.836 SNI www.bbc.co.uk 22.122345.836 OCSP stapling requested for www.bbc.co.uk 22.122345.836 ALPN offering h2 22.122345.836 ALPN offering http/1.1 22.122345.894 OCSP stapling is not supported by 'www.bbc.co.uk' 22.122345.894 host has no pubkey pinnings stored in hpkp db 22.122345.894 host has no pubkey pinnings stored in hpkp db 22.122345.894 host has no pubkey pinnings stored in hpkp db 22.122345.894 host has no pubkey pinnings stored in hpkp db 22.122345.894 host has no pubkey pinnings stored in hpkp db 22.122345.894 host has no pubkey pinnings stored in hpkp db 22.122345.894 host has no pubkey pinnings stored in hpkp db 22.122345.894 host has no pubkey pinnings stored in hpkp db 22.122345.894 TLS False Start: off 22.122345.894 ALPN: Server accepted protocol 'h2' 22.122345.895 Handshake completed 22.122345.895 established connection www.bbc.co.uk 22.122345.895 cookie_create_request_header for host=www.bbc.co.uk path=webarchive/https%3A%2F%2Fwww.bbc.co.uk%2Fblogs%2Fadamcurtis%2Fentries%2Fa2094c9d-9864-348e-a241-7aa9 22.122345.895 HTTP2 stream id 1 22.122345.895 [0] action=1 pending=1 host=0x5622fb64e6b0 goaway=false 22.122345.895 [0] action=2 pending=1 host=0x5622fb64e6b0 goaway=false 22.122345.895 ## pending_requests = 1 22.122345.895 [FRAME 0] > SETTINGS 22.122345.895 [FRAME 0] > WINDOW_UPDATE 22.122345.895 [FRAME 1] > HEADERS 22.122345.895 [FRAME 1] > :method: GET 22.122345.895 [FRAME 1] > :path: /webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 22.122345.895 [FRAME 1] > :scheme: https 22.122345.895 [FRAME 1] > :authority: www.bbc.co.uk 22.122345.895 [FRAME 1] > accept-encoding: gzip, deflate, br, zstd 22.122345.895 [FRAME 1] > accept: */* 22.122345.895 [FRAME 1] > user-agent: Wget/2.2.1 22.122345.909 [FRAME 0] < SETTINGS 22.122345.909 [FRAME 0] < WINDOW_UPDATE 22.122345.909 [FRAME 0] > SETTINGS 22.122345.910 [FRAME 0] < SETTINGS 22.122346.113 :status: 404 22.122346.113 date: Sat, 22 Aug 2026 11:23:46 GMT 22.122346.113 content-type: text/html;charset=utf-8 22.122346.113 content-length: 9687 22.122346.113 belfrage-cache-status: MISS 22.122346.113 bid: bruce 22.122346.113 brequestid: 268b641cf47a4cd78df3312be105da1c 22.122346.113 bsig: b24bbece8fc068ce8b9dfd6803e8e57e 22.122346.113 cache-control: public, stale-if-error=90, stale-while-revalidate=30, max-age=5 22.122346.113 content-encoding: gzip 22.122346.113 req-svc-chain: GTM,BELFRAGE,MOZART 22.122346.113 vary: accept-encoding 22.122346.114 via: 2.0 BBC-GTM, 1.1 Belfrage 22.122346.114 x-content-type-options: nosniff 22.122346.114 x-mrid: n1 22.122346.114 x-xss-protection: 1; mode=block 22.122346.114 origin-agent-cluster: ?0 22.122346.114 x-lb-nocache: true 22.122346.114 nel: {"report_to":"default","max_age":2592000,"include_subdomains":true,"failure_fraction":0.25} 22.122346.114 x-bbc-edge-cache-status: MISS 22.122346.114 x-bbc-origin-response-status: 404 22.122346.114 report-to: {"group":"default","max_age":2592000,"endpoints":[{"url":"https://default.bbc-reporting-api.app/report-endpoint","priority":1}],"include_subdomains":true <https://default.bbc-reporting-api.app/report-endpoint%22,%22priority%22:1%7D],%22include_subdomains%22:true>} 22.122346.114 server: BBC-GTM 22.122346.114 strict-transport-security: max-age=31536000; preload 22.122346.114 permissions-policy: browsing-topics=(), join-ad-interest-group=(), run-ad-auction=() 22.122346.114 x-robots-tag: bingbot: noarchive 22.122346.114 [FRAME 1] < HEADERS 22.122346.114 closing stream 1 22.122346.114 ## response status 404 22.122346.114 set_exit_status(8) 22.122346.114 keep_alive=1 22.122346.114 update HSTS www.bbc.co.uk:443 (maxage=31536000, includeSubDomains=0) 22.122346.114 _host_remove_job: 0x5622fb64ec70 22.122346.114 host_remove_job: qsize=0 host->qsize=0 22.122346.114 [0] action=1 pending=0 host=0x5622fb64e6b0 goaway=false 22.122346.114 closing connection 22.122346.115 [0] action=1 pending=0 host=0x0 goaway=n/a 22.122346.115 main: wake up 22.122346.115 main: done 22.122346.115 Successfully updated '/home/miles/.local/share/wget/.wget-hsts'. 22.122346.115 Saved 1 HSTS entry into '/home/miles/.local/share/wget/.wget-hsts' 22.122346.115 blacklist https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9 ---- Certificate info [0]: Valid since: Mon 13 Jul 2026 14:21:36 BST Expires: Thu 28 Jan 2027 13:21:36 GMT Fingerprint: e3756b8bc1f345cc3018a2b146b88ca5 Serial number: 518996aacb43bd1f78166375 Public key: EC/ECDSA, High (256 bits) Version: #3 DN: C=GB,ST=London,L=London,O=BRITISH BROADCASTING CORPORATION,CN=www.bbc.co.uk Issuer's DN: C=BE,O=GlobalSign nv-sa,CN=GlobalSign ECC OV SSL CA 2018 Issuer's OID: 2.5.4.6 Issuer's UID: 2.5.4.6 Certificate info [1]: Valid since: Wed 21 Nov 2018 00:00:00 GMT Expires: Tue 21 Nov 2028 00:00:00 GMT Fingerprint: 9635de986bc7f9b19f09aec50e3a3909 Public key: EC/ECDSA, Ultra (384 bits) Version: #3 DN: C=BE,O=GlobalSign nv-sa,CN=GlobalSign ECC OV SSL CA 2018 Issuer's DN: OU=GlobalSign ECC Root CA - R5,O=GlobalSign,CN=GlobalSign Issuer's OID: 2.5.4.11 Issuer's UID: 2.5.4.11 Certificate info [2]: Valid since: Wed 21 Nov 2018 00:00:00 GMT Expires: Sun 18 Mar 2029 10:00:00 GMT Fingerprint: 8f665697f9d7f2c6c6b96a99259b7ffc Serial number: 01ee5f223e3d355b992bc82ab4 Public key: EC/ECDSA, Ultra (384 bits) Version: #3 DN: OU=GlobalSign ECC Root CA - R5,O=GlobalSign,CN=GlobalSign Issuer's DN: OU=GlobalSign Root CA - R3,O=GlobalSign,CN=GlobalSign Issuer's OID: 2.5.4.11 Issuer's UID: 2.5.4.11 Certificate info [3]: Valid since: Wed 19 Sep 2018 01:00:00 BST Expires: Fri 28 Jan 2028 12:00:00 GMT Fingerprint: 58bfb0eacf17e2d0a65a4d061042c3bf Serial number: 01ee5f169dff97352b6465d66a Public key: RSA, Medium (2048 bits) Version: #3 DN: OU=GlobalSign Root CA - R3,O=GlobalSign,CN=GlobalSign Issuer's DN: C=BE,O=GlobalSign nv-sa,OU=Root CA,CN=GlobalSign Root CA Issuer's OID: 2.5.4.6 Issuer's UID: 2.5.4.6 ---- Ephemeral ECDH using curve (null) Key Exchange: ECDHE-RSA Protocol: TLS1.3 Certificate Type: X.509 Cipher: AES-256-GCM MAC: AEAD ---- [0] Downloading 'https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9' ... HTTP ERROR response 404 [https://www.bbc.co.uk/webarchive/https%253A%252F%252Fwww.bbc.co.uk%252Fblogs%252Fadamcurtis%252Fentries%252Fa2094c9d-9864-348e-a241-7aa9]
OpenPGP_signature.asc
Description: OpenPGP digital signature
