Mac-Fly via curl-library <curl-library@lists.haxx.se> wrote on 2023-09-26 at 23:29:16:
> since Monday, many of my tiny monitoring checks (analyse web pages for > changes or specific content) where I used curl just fine over _years_ > suddenly fail with HTTP 403 access denied. While investigating, I found > out that these pages are all "protected" by Cloudfare. If I use a > browser (Firefox) go to one of these webpages I see the message as in > the image attached. > > It basically says: > "[Homepage name] must check the security of your connection" > And then, at the bottom a Cloudfare reference "Ray ID" is shown as well > as a Cloudfare slogan. > > As an example to reproduce: You can use any project page on sourceforge.net > > To rant a little: I don't now whats wrong with the internet these days > and why such checks are required at all. I am sure they break a lot of > applications like mine! (Rant off.) As a Tor user I'm used to Cloudflare breaking the web and even added an entry to the ElectroBSD FAQ [0]. > Now, I found many reports and "solutions" that are related to that issue > (like manipulating the header sent, setting cookies etc.) but they are > _all_ not working in these cases. Sometimes Clouflare expects the client to execute proprietary JavaScript to solve captures or do some other proof-of-work "for security reasons", so manipulating headers isn't always sufficient. My understanding is that Cloudflare "customers" can configure the type of "protection" they get and can, for example, disable captures etc. for Tor users. Unless I'm mistaken, Cloudflare "customers" frequently don't pay with their money but with their private data and the data of the visitors and I wouldn't be surprised if Cloudflare is using the collected data for nefarious purposes but this is getting off-topic for this list. > I am sure I am not the only one and now I am searching here for answers > because I believe many curl users are affected, too. Please help me! :-) You could contact the owners of the website and request that they instruct Cloudflare to allow requests with curl again, but I suspect that many website owners don't want you to access their website with curl anyway, so they may not do that willingly ... In case of SourceForge you could also argue that SourceForge is (or used to be) a free software site, so they shouldn't require their visitors to execute proprietary JavaScript to access the site. I've used this argument in the past for other free-software-related sites and sometimes it worked and the site owners even thanked me as they were unaware of this issue. Unfortunately SourceForge changed owners a couple of times in the past and they "lost" some staff, so it's possible that you don't get any response from a human ... Fabian [0] <https://www.fabiankeil.de/gehacktes/electrobsd/#cloudflare-garbage> -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
