On Sat, 18 Oct 2025 at 12:03, ToddAndMargo via curl-users <[email protected]> wrote: > > Hi All, > > curl-8.9.1-4.fc41.x86_64 > > How do I get around "You've been blocked" on > this web sire: > > $ curl -L > https://www.softpedia.com/get/System/Back-Up-and-Recovery/Icedrive.shtml#download > -o eraseme.html > > adding > --user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:143.0) > Gecko/20100101 Firefox/143.0' > > Did not help. > > Yours in confusion, > -T > > -- > Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users > Etiquette: https://curl.se/mail/etiquette.html
softpedia.com use the cloudflare WAF thing. Cloudflare WAF detect curl by minor details in its TLS handshake, and block curl, even if the user-agent is whitelisted, WAF detect that it's actually not firefox because the TLS details doesn't match. There is a project called curl-impersonate, https://github.com/lwthiker/curl-impersonate , where curl has been modified to carefully mimic the Chromium and Firefox TLS implementation, for example: ./curl-impersonate-ff --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:140.0) Gecko/20100101 Firefox/140.0' 'https://www.softpedia.com/get/System/Back-Up-and-Recovery/Icedrive.shtml' works. The useragent claims to be firefox, and it's been modified to mimic Firefox TLS. -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.html
