On Tue, Feb 2, 2016 at 3:56 PM, David Birdsong <[email protected]> wrote:
> Has nobody else run into this w/ consul? Given the plethora of tools > around consul and haproxy and templating, I know others are using reloads > to keep backend current, but the old haproxy PIDs stick around listening w/ > incorrect backends. > I'm not using consul but am using haproxy in a docker container and reloading when backend hosts change registrations. I haven't seen this issue. I run using haproxy-systemd-wrapper and HUP that process to reload. -Bryan > > On Thu, Jan 28, 2016 at 8:52 PM David Birdsong <[email protected]> > wrote: > >> On Thu, Jan 28, 2016 at 6:35 PM, Pavlos Parissis < >> [email protected]> wrote: >> >>> On 28/01/2016 10:35 μμ, David Birdsong wrote: >>> > I've been running into a problem for a few weeks that I was hoping to >>> > see disappear w/ a simple upgrade to 1.6.3. >>> > >>> > I'm using consul and it's templating to dynamically expand a backend >>> > list which then runs an haproxy reload using the init scripts in the >>> > contrib dir. >>> > >>> > I haven't been able to trace how the situation is triggered,but >>> > basically I find haproxy processes that are still listening on their >>> > bound sockets long after having received a reload signal via the '-sf' >>> > parameter. >>> > >>> > My first fix was to ensure that reloads weren't happening too fast and >>> > potentially stomping on the pid file which would explain a process >>> never >>> > getting a signal--so I now have a lock + sleeps for the automatic >>> reload >>> > script. No change. >>> > >>> > What's probably most interesting is that a SIGKILL is necessary to >>> > remove the 'stale' processes that are still listening on their sockets >>> > (as determined by lsof.) >>> > >>> > I can supply the config if needed, there is a single listener w/ a pem >>> > directory specified if that's helpful. >>> > >>> >>> The previous processes don't die that fast because there are connections >>> still alive on them. Which is caused by very long timeout settings. >>> >> >> The haproxy instance has many tcp-mode connections, and so, yes, the >> processes often hang around for many days. I'm familiar with haproxy's >> exiting when all connections have finished. >> >> What I'm looking for help about is the listen socket remaining. lsof >> indicates a listener on the port and so accepts new connections on the >> 'stale' process and configuration file. >> >> >>> >>> The behavior you see is normal. >>> >>> Cheers, >>> Pavlos >>> >>> >>> >>>
