Il 16/05/2015 11:35, Willy Tarreau ha scritto:
Hi Marco,
On Mon, May 11, 2015 at 02:32:47PM +0200, Marco Corte wrote:
Hi.
I am running haproxy on 2 ubuntu 12.04 LTS boxes with some IP managed
by keepalived.
One week ago I updated many packages including haproxy that is now
version 1.5.12.
Since then, the "peer" traffic between the nodes increased a lot.
If both nodes are active, each one owning some IPs, the 'peers' traffic
between the nodes seems higher (10x) than before the package update.
Moreover, even if one of the peers does not have any active IP, i.e.
haproxy is doing nothing, the outgoing traffic measured on the NIC
(SNMP) is about 500-600kb/s.
If I disable the 'peers' section, the outgoing traffic drops to 60 kb/s.
That's not expected at all, what was the previous version ?
I upgraded from version 1.5.11.
For these hosts I always use the ubuntu packages coming from Vincent
Bernat: https://launchpad.net/~vbernat/+archive/ubuntu/haproxy-1.5
The behaviour is similar changing the active/passive role between the
nodes: the inactive one still generates a lot of traffic. Is this an
expected behaviour?
The 'peers' configuration is very standard:
peers li01
peer pgli01 10.64.38.1:1024
peer pmli01 10.64.38.2:1024
Then there are 24 'stick-tables' similar to this:
listen XXXX
mode tcp
option tcplog
option ssl-hello-chk
stick-table type ip size 1k expire 30m store conn_cur,conn_rate(10s) peers
li01
tcp-request connection reject if { src_conn_cur ge 100 }
tcp-request connection reject if { src_conn_rate ge 50 }
tcp-request connection track-sc0 src
stick on src
default-server inter 5s fastinter 2s downinter 30s
server XXX1-443 10.64.38.227:443 check
server XXX2-443 10.64.38.228:443 check
Note, the peers protocol doesn't synchronize data other than server_id
with peers, however it updates keys that have got some traffic. Thus I'm
starting to wonder whether you would not have been running an older version
which did not propagate such updates in the past. It would be nice if you
could confirm this.
I really did nothing else than a package update (apt-get dist-upgrade)
that also updated the kernel and a few other packages. The previous
update was about 3 weeks before.
I only saw a higher CPU load at first, then I correlate it to the higher
NIC traffic.
Please note that there is no real problem, since theservers still have
plenty of resources to use.
The only symptom that let me think to an haproxy issue was the traffic
and CPU usage drop in both peers, when turning off the _inactive_
haproxy instance.
Noone is experiencing a similar behaviour, so I am starting thinking to
a problem in another package (the kernel, a library, ...).
Any suggestion in how to investigate this further?
I am already planning a reboot with an older kernel in the next days.
Thank you.
.marcoc
