Hi Bar,
On Thu, May 10, 2012 at 07:02:58PM +0300, Bar Ziony wrote:
> Hey,
>
> We're running haproxy 1.4.20 as our LB, nginx is listening on the same
> machine on port 443 and terminating SSL, proxying the unencrypted requests
> to haproxy on localhost:80.
>
> I see many of these errors on the haproxy log:
> May 10 15:54:06 lb-01 haproxy[6563]:
> 1.1.1.1:50929[10/May/2012:15:54:01.113] public public/<NOSRV>
> -1/-1/-1/-1/5519 400 187 -
> - CR-- 3045/3045/0/0/0 0/0 "<BADREQ>"
>
> * I changed the source IP for the sake of this example.
>
> We get around 5-15 of these per second, and I checked some of the IPs and
> it seems at least some of them are IPs that users registered with (maybe
> it's a very big proxy or something so it's not actually those users).
As you can see, the client took 5.5 seconds to send an incomplete request
then closed the connection ('CR'). It is possible that some users have
developped monitoring scripts which are targetting your site. I sometimes
get a number of these on the haproxy web site too. While the ones sending
valid requests are just a bandwidth annoyance, the ones sending invalid
requests are much more annoying.
If the requests are completely invalid, you can find a capture of them on
the stats socket using "show errors" :
echo "show errors" | socat stdio /var/run/haproxy.stat
(or whereever you put it, check "stats socket" in your global section)
> We're running on a pretty fast Linode VPS (1GB RAM), it handled 5000
> requests per second on testing (which is low, I know, but it is still a
> VPS). We are doing much less than 5000 req/sec ...
> The CPU usage is 10-20% for haproxy alone (10% more for nginx), and 10-20%
> RAM usage for haproxy (~150MB RES, ~180MB VIRT). Does that make sense?
Yes, nothing sounds strange here.
> What are these requests? Is it possible these are regular users trying to
> somehow get to our web app and not succeeding ?
That's really unlikely, because such invalid requests happen at a layer which
is only controlled by the browser. A normal browser cannot emit invalid
requests. Only bots do. It's possible that some of your users are running
crappy site sucking plugins, or home-made search engines which emit invalid
requests. Some of the invalid requests I'm used to observe are those where
the user forget to send the last CR/LF, so the request is incomplete.
Well, if you have 15 of these a second, just run tcpdump for a few seconds
to capture some of them and you'll know what these are.
Regards,
Willy
