On 05/27/2014 08:36 PM, Willy Tarreau wrote:
I don't see why GoogleBot would see them since they should only affect
the offending clients.
Is it by any chance possible that my ISP is somehow screwing up
connections? Because I see this kind of aborts/503s even from regular
clients fetching regular stuff?
Could be possible, but that sounds really strange. You could easily check
though, if you own a machine somewhere outside your ISP's network. Simply
send a request from there to your site and sniff at both ends. You'll see
if the trace matches or not. It could be possible that the ISP is running
a misconfigured transparent proxy which systematically closes the request
path after sending the request (as haproxy used to do with option forceclose
in early version 1.1 12 years ago). Or maybe it's part of an IDS or anti-ddos
mechanism that's automatically enabled when they run into trouble.
I've talked to ISP technicians and what they told me is that company has
bandwith cap at XYZ Mbits, and once that limit is reached additional
packets are simply dropped.
So, packets dropping at peaks seems promising as explanation of some of
the behaviour we have observed...
Will investigate more, and report back.
