Willy,
thanks for the prompt answer.
Yeah, I could probably use SIGTERM, but I need to be 100% sure the process is
killed, otherwise I'll run into serious trouble with my system architecture.
We are still trying to track down what happened. With the duplicate requests,
might not be an issue of haproxy, BTW.
I'm still wondering why haproxy allows launching duplicate copies of its main
process without complaining or generating any special error log.
My configuration, anonymized, is the following (we are using a per-request
balancing with a sickness table based on an URL param):
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
chroot /var/run/haproxy
pidfile /var/run/haproxy/haproxy.pid
daemon
stats socket /var/run/haproxy/stats
defaults
log global
timeout queue 1m
timeout connect 15s
timeout client 1m
timeout server 1m
timeout check 5s
maxconn 100000
#---------------------------------------------------------------------
# main cluster configuration (IPs masked for privacy)
#---------------------------------------------------------------------
listen LB_Health_check
mode http
bind x.x.x.x:12344
balance source
server SRV1 localhost:12345
listen BE1
mode http
bind y.y.y.y:8888
option httplog
balance url_param ipaddr
http-check disable-on-404
option httpchk GET /Param?
server TAC_connector_node1 a.a.a.a:8888 check inter 5000 fall 2 rise 5
server TAC_connector_node2 b.b.b.b:8888 check inter 5000 fall 2 rise 5
listen BE2
mode http
bind y.y.y.y:8889
option httplog
balance url_param ipaddr
http-check disable-on-404
option httpchk GET /Param?
server TAC_connector_node1 a.a.a.a:8889 check inter 5000 fall 2 rise 5
server TAC_connector_node2 b.b.b.b:8889 check inter 5000 fall 2 rise 5
-----Original Message-----
From: Willy Tarreau [mailto:[email protected]]
Sent: martedì 16 luglio 2013 09:01
To: Glauco Bigini
Cc: [email protected]
Subject: Re: Haproxy not stopping an sending duplicate requests
Hello Glauco,
On Mon, Jul 15, 2013 at 05:45:53PM +0000, Glauco Bigini wrote:
> Hello,
>
> We have an haproxy setup here (great piece of software!) showing a strange
> issue.
>
> After months of flawless operation, suddenly the service initscript is
> not able to close the process anymore. Need to use a kill -9 manually.
Did you try kill instead ? Haproxy doesn't intercept the SIGTERM, so there is
no reason to use -9. If it doesn't work with a normal kill, there might be a
bigger issue. Old processes remaining alive can happen is some long connections
remain active (eg: if you have some overly long timeouts or no timeouts at all
in a given instance).
> Even more strange, at every restart the process is not closed, but
> another instance is started. We use a configuration with stickiness
> tables and the backend servers receive duplicate requests from haproxy
> when multiple process instances are running.
Huh ? are you sure these are not health checks ? Do you find the requests in
the logs ? It could be possible that a browser sent a request through the first
process over an existing connection, the process is killed, then the browser
resends the request over a new connection via the replacement process.
But otherwise I see no way for duplicating traffic between processes ! It's
already not possible inside a single process...
> It's not a particularly serious issue (at least we know how to work
> around it), but I was wondering how the duplicate request part could work.
>
> I would expect some errors at haproxy startup or in the logs when
> another instance is already running, but from haproxy perspective
> everything is running fine. Is this the expected behavior?
Too many strange things to give a verdict on the behaviour at the moment !
Could you please post your config (without confidential info) ?
Thanks,
Willy
