Roi Martin wrote: > https://git.savannah.gnu.org seems to be up, but since today at around > 8:00 AM UTC, I'm getting a timeout error when I try to connect.
Thank you for the report. I appreciate the feedback. "It's the squeaky wheel that gets the oil." > $ curl -4 -v https://git.savannah.gnu.org/git/elpa/gnu.git It's also good to know exactly what command is running. Is this a simplification analogue to running git clone? Otherwise I can't imagine any reason for using curl like this. We are forced to continue to tighten access. I think it likely that we will need to fingerprint git clients and only allow git clients moving forward. In order to improve service availability we have recently created a mirror pool of servers on a distributed collective of independent volunteer donated systems. Try it! We have multiple systems in the pool. Instead of the one single primary upstream system we have multiple mirrors. This scales-out and is somewhat better at handling the onslaught of the endless AI scraping botnets. git clone --depth=1 https://https.git.savannah.gnu.org/git/elpa/gnu.git We have not implemented it yet but I think it likely we will implement an HTTP Redirect for https://git.savannah.gnu.org/git/ to automatically redirect to https://https.git.savannah.gnu.org/git/ at some point in order to move load off the primary and onto the mirrors. I have done that for the gitweb & cgit human browsing interfaces. I wanted to burn-in the new process on the web browser before doing it for the machine protocols. But I think it is time to do it for that protocol service too. > * Host git.savannah.gnu.org:443 was resolved. Your DNS looked up the name and received an address. > * IPv6: (none) You do not have IPv6 available to you. > * IPv4: 209.51.188.168 > * Trying 209.51.188.168:443... > * connect to 209.51.188.168 port 443 from 192.168.1.17 port 52440 failed: > Connection timed out > * Failed to connect to git.savannah.gnu.org port 443 after 132636 ms: Could > not connect to server > * closing connection #0 > curl: (28) Failed to connect to git.savannah.gnu.org port 443 after 132636 > ms: Could not connect to server I think it likely that the primary system was overwhelmed with the onslaught of the AI scraping botnets. At least one company is proud of their abuse and claims to have 150 million smart TVs in their proxy pool making it hard to avoid scraping. They are proud of this. This is the new normal. This is why we are pushing load off the primary and over to the mirrored secondaries. The https:// protocol uses the git-http-backend FastCGI process. Though this is called fast cgi it is still a quite heavy resource intensive service. Any individual server can only serve a limited number of clients. The primary gets overwhelmed by botnet scrapers that seem confused and are scraping those like they are web pages creating useless wasteful load. Additionally the network stack can handle up to about 500 connections per second. After that the TCP protocol must drop packets. When there are millions of bots hammering away this overwhelms the network connectivity and excess connections get timeouts. > BTW my ssh access works fine. HTTPS is of course port 443 while SSH is port 22. Different ports. In particular the AI scraping botnet army is concentrating on http and https at this time. The bots trying to get through ssh port 22 are a different independent set of abuse. The https abuse set is the biggest abuse set. In summary I think the primary system was simply overloaded. Please use the mirror pool system. Please report back and say how this works for you. Bob
