On 2/14/24 11:21, Chris Smith via Primary discussion list for GNU Wget wrote:
Hi guys,
This is not so much a bug as requests being blocked by the cloudflare server.

Checkout:
     https://www.investing.com/crypto/bitcoin/btc-usd-historical-data

The server just gives a 403 Forbidden.


The URL works in Firefox, but fails to download using cURL or Wget.
I have tried various user-agent strings, so that is not the problem.

Maybe the page requires a Referrer: header? In theory it could be anything, e.g. the page is HTTP/3 only or whatnot :|

You can start with F12 (developer console) in Firefox, load the page and check the request headers. Or better: there is a copy-as-curl when right-clicking on the request line that puts a full curl command line into your clipboard.


The following URL works for Firefox, cURL and Wget:
     https://www.investing.com/equities/lloyds-banking-grp-historical-data

Kind regards,
Chris Smith <chris.sm...@eclipso.eu>

_________________________________________________________________
________________________________________________________
Your E-Mail. Your Cloud. Your Office. eclipso Mail Europe. 
https://www.eclipso.de



Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature

Reply via email to