On 5/29/26 23:41, Bob Proulx wrote:
Jacob Bachmeyer wrote:
[...]
Could abusive crawlers be tying up, not only the httpd, but also the
VM host and therefore the other services running in different VMs?
That was the point I was making. :-)
Is there only one host or could the cluster be physically partitioned
with non-Web services on one subset and concentrate the httpds on the
rest of the cluster?
Abusive crawlers cannot tie up the MX's host if the httpds are on a
different host box. :-)
They *could* still eat the entire uplink bandwidth, if the httpds can
give it to them. :-(
I know abusive crawlers are the obvious culprit these days, but could that
abuse be covering up another problem, such as growing hardware flakiness?
The hardware in this case seems to be working just fine. It's just
overwhelmed.
... and of course the cloudy abuse crawlers will auto-scale to tie up
whatever hardware you throw in front of them...
I am not sure that this has a real solution before the "AI" bubble pops
and the MACACs die. :-/ (I call them MACACs (MAss/ive Cloud/y
Abuse/ive Crawlers, take your pick on the variant) pronounced like
"macaque" to reflect their monkey-like greed.)
-- Jacob
