Re: Web-bot tarpit aka spider trap (was: swamp rat bots Q)

2020-12-06 Thread Doug McGarrett
On 12/6/20 8:26 PM, Gene Heskett wrote: On Sunday 06 December 2020 16:40:37 Charles Curley wrote: On Sun, 06 Dec 2020 13:15:50 -0600 "Martin McCormick" wrote: I found our old dial-up modem in a box of odds and ends 2 years ago and wondered if it could read callerID tones sent

Re: Web-bot tarpit aka spider trap (was: swamp rat bots Q)

2020-12-06 Thread Gene Heskett
On Sunday 06 December 2020 16:40:37 Charles Curley wrote: > On Sun, 06 Dec 2020 13:15:50 -0600 > > "Martin McCormick" wrote: > > I found our old dial-up modem in a box of odds and ends 2 > > years ago and wondered if it could read callerID tones sent after > > the first ring. It can so I

Re: Web-bot tarpit aka spider trap (was: swamp rat bots Q)

2020-12-06 Thread Charles Curley
On Sun, 06 Dec 2020 13:15:50 -0600 "Martin McCormick" wrote: > I found our old dial-up modem in a box of odds and ends 2 > years ago and wondered if it could read callerID tones sent after > the first ring. It can so I started on a perl program that > initializes the modem for callerID

Re: Web-bot tarpit aka spider trap (was: swamp rat bots Q)

2020-12-06 Thread David Christensen
On 12/6/20 11:15 AM, Martin McCormick wrote: It's nice to see that I am not the only sick puppy out there. At 69 years old, I still don't have much trouble with getting in touch with my inner 12-year-old when it comes to intrusive marketing which is so prevalent these days. I

Re: Web-bot tarpit aka spider trap (was: swamp rat bots Q)

2020-12-06 Thread Martin McCormick
It's nice to see that I am not the only sick puppy out there. At 69 years old, I still don't have much trouble with getting in touch with my inner 12-year-old when it comes to intrusive marketing which is so prevalent these days. I found our old dial-up modem in a box of odds and

Re: swamp rat bots Q

2020-12-05 Thread Andrei POPESCU
On Vi, 04 dec 20, 18:55:22, Gene Heskett wrote: > On Friday 04 December 2020 17:37:02 Tixy wrote: > > > > OK, I'll do your proofreading... [...] > Fat fingers syndrome, I've suffered from that for 86 years. Typing errors happen to everyone. Triple-checking the result when something is not

Re: swamp rat bots Q

2020-12-04 Thread grumpy
On Sat, 5 Dec 2020, elvis wrote: Just goes to show if you whinge hard enough and pretend to be useless it will irritate someone enough to do your work. You don't even bother to check your robots.txt and complained about some evil bot all year. Priceless. now now little girl don't get your

Re: swamp rat bots Q

2020-12-04 Thread elvis
Just goes to show if you whinge hard enough and pretend to be useless it will irritate someone enough to do your work. You don't even bother to check your robots.txt and complained about some evil bot all year. Priceless. On 5/12/20 8:37 am, Tixy wrote: On Fri, 2020-12-04 at 17:06 -0500,

Re: swamp rat bots Q

2020-12-04 Thread Gene Heskett
On Friday 04 December 2020 17:37:02 Tixy wrote: > On Fri, 2020-12-04 at 17:06 -0500, Gene Heskett wrote: > > On Friday 04 December 2020 16:14:29 Tixy wrote: > > > On Fri, 2020-12-04 at 14:51 -0500, Gene Heskett wrote: > > > > On Friday 04 December 2020 12:39:24 Reco wrote: > > > > > Hi. > >

Re: swamp rat bots Q

2020-12-04 Thread Tixy
On Fri, 2020-12-04 at 17:06 -0500, Gene Heskett wrote: > On Friday 04 December 2020 16:14:29 Tixy wrote: > > > On Fri, 2020-12-04 at 14:51 -0500, Gene Heskett wrote: > > > On Friday 04 December 2020 12:39:24 Reco wrote: > > > > Hi. > > > > > > > > On Fri, Dec 04, 2020 at 08:39:42AM -0500,

Re: swamp rat bots Q

2020-12-04 Thread Gene Heskett
On Friday 04 December 2020 16:14:29 Tixy wrote: > On Fri, 2020-12-04 at 14:51 -0500, Gene Heskett wrote: > > On Friday 04 December 2020 12:39:24 Reco wrote: > > > Hi. > > > > > > On Fri, Dec 04, 2020 at 08:39:42AM -0500, Gene Heskett wrote: > > > > But I asked specifically how to enable it

Re: swamp rat bots Q

2020-12-04 Thread Tixy
On Fri, 2020-12-04 at 14:51 -0500, Gene Heskett wrote: > On Friday 04 December 2020 12:39:24 Reco wrote: > > > Hi. > > > > On Fri, Dec 04, 2020 at 08:39:42AM -0500, Gene Heskett wrote: > > > But I asked specifically how to enable it for one bot, and I've > > > asked that question several

Re: Web-bot tarpit aka spider trap (was: swamp rat bots Q)

2020-12-04 Thread tomas
On Fri, Dec 04, 2020 at 11:02:48AM -0500, Gene Heskett wrote: [...] > Sounds like a good idea, I'll have to think about it, feed the bots in > 256 byte pieces every 5 seconds to keep them from timing out, with 256 > bytes from rnd mixed in to make a dos packet? :) Just be sure the crc is >

Re: swamp rat bots Q

2020-12-04 Thread Gene Heskett
On Friday 04 December 2020 13:16:23 Reco wrote: > On Fri, Dec 04, 2020 at 12:06:41PM -0600, gru...@mailfence.com wrote: > > On Fri, 4 Dec 2020, Reco wrote: > > > Hi. > > > > > > On Fri, Dec 04, 2020 at 08:39:42AM -0500, Gene Heskett wrote: > > > > But I asked specifically how to enable it for

Re: swamp rat bots Q

2020-12-04 Thread Gene Heskett
On Friday 04 December 2020 12:39:24 Reco wrote: > Hi. > > On Fri, Dec 04, 2020 at 08:39:42AM -0500, Gene Heskett wrote: > > But I asked specifically how to enable it for one bot, and I've > > asked that question several times, getting smoke and mirror answers > > you all assume are

Re: swamp rat bots Q

2020-12-04 Thread Carl Fink
On 12/4/2020 1:16 PM, Reco wrote: What's *your* suggestion for solving this problem? Speaking only for myself: maybe someone could point Gene to a new build of fail2ban? A howto for setting it up? A better tool for the job? Saying "Just read the man page" is literally the cliche unhelpful

Re: swamp rat bots Q

2020-12-04 Thread Reco
On Fri, Dec 04, 2020 at 12:06:41PM -0600, gru...@mailfence.com wrote: > On Fri, 4 Dec 2020, Reco wrote: > > > Hi. > > > > On Fri, Dec 04, 2020 at 08:39:42AM -0500, Gene Heskett wrote: > > > But I asked specifically how to enable it for one bot, and I've asked > > > that question several

Re: swamp rat bots Q

2020-12-04 Thread grumpy
On Fri, 4 Dec 2020, Reco wrote: Hi. On Fri, Dec 04, 2020 at 08:39:42AM -0500, Gene Heskett wrote: But I asked specifically how to enable it for one bot, and I've asked that question several times, getting smoke and mirror answers you all assume are helpfull, but which are useless to a

Re: swamp rat bots Q

2020-12-04 Thread Reco
Hi. On Fri, Dec 04, 2020 at 08:39:42AM -0500, Gene Heskett wrote: > But I asked specifically how to enable it for one bot, and I've asked > that question several times, getting smoke and mirror answers you all > assume are helpfull, but which are useless to a new user installing the >

Re: Web-bot tarpit aka spider trap (was: swamp rat bots Q)

2020-12-04 Thread Gene Heskett
On Friday 04 December 2020 09:00:05 Nicolas George wrote: > hdv@gmail (12020-12-04): > > Let me offer you an alternative option. (Most) bots work by > > analysing the referrals on each page in your website. Right? So, why > > not add a link to a page that normal users will never visit (e.g. > >

Web-bot tarpit aka spider trap (was: swamp rat bots Q)

2020-12-04 Thread Nicolas George
hdv@gmail (12020-12-04): > Let me offer you an alternative option. (Most) bots work by analysing the > referrals on each page in your website. Right? So, why not add a link to a > page that normal users will never visit (e.g. because they do not see the > link and thus will never click on it), but

Re: swamp rat bots Q

2020-12-04 Thread Gene Heskett
On Friday 04 December 2020 05:47:40 Jeremy Nicoll wrote: > On Fri, 4 Dec 2020, at 05:03, Gene Heskett wrote: > > Fail2ban does not come configured to do anything. In this case, not > > even waste cpu cycles. I've now read thru most of the configs, which > > may have been semi applicable in 2013,

Re: swamp rat bots Q

2020-12-04 Thread Gene Heskett
On Friday 04 December 2020 03:49:39 hdv@gmail wrote: > On 2020-12-03 13:35, Gene Heskett wrote: > > I've had it with a certain bot that that ignore my robots.txt and > > proceeds to mirror my site, several times a day, burning up my > > upload bandwidth. They've moved it to 5 different addresses

Re: swamp rat bots Q

2020-12-04 Thread Andrei POPESCU
On Vi, 04 dec 20, 00:03:57, Gene Heskett wrote: > > Fail2ban does not come configured to do anything. In this case, not even > waste cpu cycles. I've now read thru most of the configs, which may have > been semi applicable in 2013, the date of its last update. But this, in > case no one has

Re: swamp rat bots Q

2020-12-04 Thread Jeremy Nicoll
On Fri, 4 Dec 2020, at 05:03, Gene Heskett wrote: > Fail2ban does not come configured to do anything. In this case, not even > waste cpu cycles. I've now read thru most of the configs, which may have > been semi applicable in 2013, the date of its last update. But this, in > case no one has

Re: swamp rat bots Q

2020-12-04 Thread hdv@gmail
On 2020-12-03 13:35, Gene Heskett wrote: I've had it with a certain bot that that ignore my robots.txt and proceeds to mirror my site, several times a day, burning up my upload bandwidth. They've moved it to 5 different addresses since midnight. I want to nail the door shut on the first

Re: swamp rat bots Q

2020-12-03 Thread Andy Smith
On Fri, Dec 04, 2020 at 01:40:53AM -0500, Gene Heskett wrote: > On Friday 04 December 2020 01:03:34 Andy Smith wrote: > > Again we have been down this avenue before, but I will try one last > > time. […] > > So, can you show us a few lines of logs from your > >

Re: swamp rat bots Q

2020-12-03 Thread john doe
On 12/4/2020 7:40 AM, Gene Heskett wrote: On Friday 04 December 2020 01:03:34 Andy Smith wrote: Hello, On Fri, Dec 04, 2020 at 12:03:57AM -0500, Gene Heskett wrote: what file do I edit to add todays /var/log/httpd/other_vhosts_access.log to its list of logs to watch. That's the log file

Re: swamp rat bots Q

2020-12-03 Thread Gene Heskett
On Friday 04 December 2020 01:34:52 john doe wrote: > On 12/4/2020 7:03 AM, Andy Smith wrote: > > Hello, > > > > On Fri, Dec 04, 2020 at 12:03:57AM -0500, Gene Heskett wrote: > >> what file do I edit to add todays > >> /var/log/httpd/other_vhosts_access.log to its list of logs to > >> watch.

Re: swamp rat bots Q

2020-12-03 Thread Gene Heskett
On Friday 04 December 2020 01:03:34 Andy Smith wrote: > Hello, > > On Fri, Dec 04, 2020 at 12:03:57AM -0500, Gene Heskett wrote: > > what file do I edit to add todays > > /var/log/httpd/other_vhosts_access.log to its list of logs to > > watch. That's the log file with the real data in it today.

Re: swamp rat bots Q

2020-12-03 Thread john doe
On 12/4/2020 7:03 AM, Andy Smith wrote: Hello, On Fri, Dec 04, 2020 at 12:03:57AM -0500, Gene Heskett wrote: what file do I edit to add todays /var/log/httpd/other_vhosts_access.log to its list of logs to watch. That's the log file with the real data in it today. And does it need enabled in

Re: swamp rat bots Q

2020-12-03 Thread Andy Smith
Hello, On Fri, Dec 04, 2020 at 12:03:57AM -0500, Gene Heskett wrote: > what file do I edit to add todays > /var/log/httpd/other_vhosts_access.log to its list of logs to > watch. That's the log file with the real data in it today. And > does it need enabled in another, different file. Again we

Re: swamp rat bots Q

2020-12-03 Thread Gene Heskett
On Thursday 03 December 2020 21:13:35 Andy Smith wrote: > Hello, > > On Thu, Dec 03, 2020 at 07:35:27AM -0500, Gene Heskett wrote: > > I've had it with a certain bot that that ignore my robots.txt and > > proceeds to mirror my site, several times a day, burning up my > > upload bandwidth. They've

Re: swamp rat bots Q

2020-12-03 Thread Andy Smith
Hello, On Thu, Dec 03, 2020 at 07:35:27AM -0500, Gene Heskett wrote: > I've had it with a certain bot that that ignore my robots.txt and > proceeds to mirror my site, several times a day, burning up my upload > bandwidth. They've moved it to 5 different addresses since midnight. This must be

Re: swamp rat bots Q

2020-12-03 Thread Keith Christian
Gene, Fail2ban can be difficult to comprehend at first, so here are some ideas: As either the fail2ban user (may need root), run this command to see that fail2ban is active and what "jails" are active, a jail corresponds to one type of message in the log file fail2ban is watching, which are set

Re: swamp rat bots Q

2020-12-03 Thread Håkon Alstadheim
Den 03.12.2020 19:58, skrev Gene Heskett: On Thursday 03 December 2020 09:14:29 Greg Wooledge wrote: On Thu, Dec 03, 2020 at 09:02:47AM -0500, Gene Heskett wrote: Yes John. But explain to me what fail2ban is sopposed to do? It's supposed to "monitor" (tail -F equivalent) your log files,

Re: swamp rat bots Q

2020-12-03 Thread Gene Heskett
On Thursday 03 December 2020 09:14:29 Greg Wooledge wrote: > On Thu, Dec 03, 2020 at 09:02:47AM -0500, Gene Heskett wrote: > > Yes John. But explain to me what fail2ban is sopposed to do? > > It's supposed to "monitor" (tail -F equivalent) your log files, and > look for anomalies. If it finds

Re: swamp rat bots Q

2020-12-03 Thread john doe
On 12/3/2020 3:02 PM, Gene Heskett wrote: On Thursday 03 December 2020 08:07:33 john doe wrote: On 12/3/2020 1:35 PM, Gene Heskett wrote: I've had it with a certain bot that that ignore my robots.txt and proceeds to mirror my site, several times a day, burning up my upload bandwidth. They've

Re: swamp rat bots Q

2020-12-03 Thread Greg Wooledge
On Thu, Dec 03, 2020 at 09:02:47AM -0500, Gene Heskett wrote: > Yes John. But explain to me what fail2ban is sopposed to do? It's supposed to "monitor" (tail -F equivalent) your log files, and look for anomalies. If it finds one, it's supposed to take action, which is typically adding an entry

Re: swamp rat bots Q

2020-12-03 Thread Gene Heskett
On Thursday 03 December 2020 08:07:33 john doe wrote: > On 12/3/2020 1:35 PM, Gene Heskett wrote: > > I've had it with a certain bot that that ignore my robots.txt and > > proceeds to mirror my site, several times a day, burning up my > > upload bandwidth. They've moved it to 5 different

Re: swamp rat bots Q

2020-12-03 Thread john doe
On 12/3/2020 1:35 PM, Gene Heskett wrote: I've had it with a certain bot that that ignore my robots.txt and proceeds to mirror my site, several times a day, burning up my upload bandwidth. They've moved it to 5 different addresses since midnight. I want to nail the door shut on the first

swamp rat bots Q

2020-12-03 Thread Gene Heskett
I've had it with a certain bot that that ignore my robots.txt and proceeds to mirror my site, several times a day, burning up my upload bandwidth. They've moved it to 5 different addresses since midnight. I want to nail the door shut on the first attempted access by these AH's. Does anyone