There is what we could call a "growth market" of grey market organizations and individuals selling residential proxies that route traffic through actual residential cablemodem, DSL, FTTH connections at peoples' houses. Usually this is implemented one of two ways, a router/home gateway device that's been pwned and is part of a botnet, or by tricking people into installing some form of proxy software on one of their persistently connected home computers.
Used for both bot scraping and also more mundane credit card fraud and various scams, providing people access to georestricted online casinos, weird hawala-adjacent online money/cryptocurrency exchange and transfer platforms. Just a vast array of shady stuff. Google "residential proxies for sale" to see the tip of the iceberg. On Wed, Jul 16, 2025 at 4:38 PM Andrew Latham via NANOG < [email protected]> wrote: > Chris > > Spot on, and I am getting the feeling this is where the value to a > geo-ip service comes to play that offers defined "eyeball networks" to > allow. > > On Wed, Jul 16, 2025 at 12:57 PM Chris Adams via NANOG > <[email protected]> wrote: > > > > Once upon a time, Marco Moock <[email protected]> said: > > > Place a link to a file that is hidden to normal people. Exclude the > > > directory via robots.txt. > > > > > > Then use fail2ban to block all IP addresses that poll the file. > > > > The problem with a lot of the "AI" scrapers is that they're apparently > > using botnets and will often only make a single request from a given IP > > address, so reactive blocking doesn't work (and can cause other issues, > > like trying to block 100,000 IPs, which fail2ban for example doesn't > > really handle well). > > -- > > Chris Adams <[email protected]> > > _______________________________________________ > > NANOG mailing list > > > https://lists.nanog.org/archives/list/[email protected]/message/AFJF4UQJZW6ALTY6SA7OHBN2AZC72SZQ/ > > > > -- > - Andrew "lathama" Latham - > _______________________________________________ > NANOG mailing list > > https://lists.nanog.org/archives/list/[email protected]/message/DHUYTBIXFMWE2KWC5NKCR7AJIWPYUL4E/ _______________________________________________ NANOG mailing list https://lists.nanog.org/archives/list/[email protected]/message/YF2DNATKWKI6JF47F5OZZCYGBXOXMODX/
