Re: haproxy doesn't get SIGUSR1

2015-12-10 Thread Willy Tarreau
Hi Vincent,

On Wed, Dec 02, 2015 at 01:44:49PM +0100, Vincent Bernat wrote:
>  ???  2 décembre 2015 09:30 +0100, Lukas Tribus  :
> 
> > Also see Lukas Lösche's reports and efforts:
> > https://github.com/haproxy/haproxy/issues/48
> 
> Totally unrelated with the current issue, but on the GitHub page, this
> is said that issues are ignored but some of them are actually
> active. It's possible to disable "Issues" through the administration tab
> of the repository. Shouldn't they be disabled to avoid to scatter bug
> reports?

That's what we discussed with Jeff but apparently if we disable them,
we'll also lose them and some seem valid and worth investigating.

> Otherwise, the mailing list could also be subscribed so that we don't
> miss some issues (I have no experience in this).

I think so as well, especially since the return address from GitHub's
issue tracker makes it possible to respond via e-mail.

Willy




Re: haproxy doesn't get SIGUSR1

2015-12-02 Thread Vincent Bernat
 ❦  2 décembre 2015 09:30 +0100, Lukas Tribus  :

> Also see Lukas Lösche's reports and efforts:
> https://github.com/haproxy/haproxy/issues/48

Totally unrelated with the current issue, but on the GitHub page, this
is said that issues are ignored but some of them are actually
active. It's possible to disable "Issues" through the administration tab
of the repository. Shouldn't they be disabled to avoid to scatter bug
reports?

Otherwise, the mailing list could also be subscribed so that we don't
miss some issues (I have no experience in this).
-- 
Avoid temporary variables.
- The Elements of Programming Style (Kernighan & Plauger)



Re: haproxy doesn't get SIGUSR1

2015-12-02 Thread 任晓磊
Tried that flock way, but it doesn't take effect.

Lukas Tribus 于2015年12月2日周三 下午4:30写道:

> > I'm using service_loadbalancer from kubernetes
> > (https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
> )
> > . This program would re-spawn haproxy when it found a change of
> > upstream endpoints.
> > When service_loadbalancer starts, it runs haproxy -sf $(cat pidfile)
> > several times very quickly, and on that moment, haproxy -sf doesn't
> > take effect, there were many haproxy processes left.
>
> Also see Lukas Lösche's reports and efforts:
> https://github.com/haproxy/haproxy/issues/48
> http://thread.gmane.org/gmane.comp.web.haproxy/25105/
>
> and downstream marathon-lb report:
> https://github.com/mesosphere/marathon-lb/issues/5
>
> Workaround in marathon-lb with manual locking:
>
> https://github.com/mesosphere/marathon-lb/commit/83260fdf687c774064b54d3bb009f5b3a1d75c97
>
>
>
>
> Regards,
>
> Lukas
>
>


RE: haproxy doesn't get SIGUSR1

2015-12-02 Thread Lukas Tribus
> I'm using service_loadbalancer from kubernetes 
> (https://github.com/kubernetes/contrib/tree/master/service-loadbalancer ) 
> . This program would re-spawn haproxy when it found a change of 
> upstream endpoints. 
> When service_loadbalancer starts, it runs haproxy -sf $(cat pidfile) 
> several times very quickly, and on that moment, haproxy -sf doesn't 
> take effect, there were many haproxy processes left.

Also see Lukas Lösche's reports and efforts:
https://github.com/haproxy/haproxy/issues/48
http://thread.gmane.org/gmane.comp.web.haproxy/25105/

and downstream marathon-lb report:
https://github.com/mesosphere/marathon-lb/issues/5

Workaround in marathon-lb with manual locking:
https://github.com/mesosphere/marathon-lb/commit/83260fdf687c774064b54d3bb009f5b3a1d75c97




Regards,

Lukas

  

haproxy doesn't get SIGUSR1

2015-12-01 Thread 任晓磊
I'm using service_loadbalancer from kubernetes (
https://github.com/kubernetes/contrib/tree/master/service-loadbalancer ) .
This program would re-spawn haproxy when it found a change of upstream
endpoints.
When service_loadbalancer starts, it runs haproxy -sf $(cat pidfile)
several times very quickly, and on that moment, haproxy -sf doesn't take
effect, there were many haproxy processes left.

I ran strace on the new haproxy -sf , and ensured it did send a SIGUSR1 to
the old process indicated by pidfile.
I tried to send SIGUSR1, SIGTERM, SIGINT to the left haproxy processes
while strace-ing on them, but they did not get the signals. I checkec
/proc/PID/status file, the SigBlk line is fffc7bfa7a27. SIGKILL can
kill the old process.

I searched SIG_SETMASK in the source code, and found it in signal.c file.
Maybe you should remove SIGUSR1 from blocked_sig set in signal_init()
function?


inspected old process with gdb:
(gdb) where
#0 0x7f42b2916763 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1  0x004d47db in
_do_poll (p=0x73a5e0 , exp=1633798236) at src/ev_epoll.c:125
#2  0x0040a8ef in
run_poll_loop () at src/haproxy.c:1576
#3  0x0040b438 in main
(argc=8, argv=0x7fff4168bd08) at src/haproxy.c:1912
(gdb) print jobs
$1 = 4
(gdb) print stopping
$2 = 0

ls -l /proc/46/fd/
total 0
lrwx-- 1 root root 64 Dec 2 06:00 0 -> anon_inode:[eventpoll]
lrwx-- 1 root root 64 Dec 2 06:00 4 -> socket:[5699]
lrwx-- 1 root root 64 Dec 2 06:00 5 -> socket:[5700]
lrwx-- 1 root root 64 Dec 2 06:00 6 -> socket:[5701]
lrwx-- 1 root root 64 Dec 2 06:00 7 -> socket:[5702]

the process have four sockets, just same as the jobs variable

lsof -p 46 | grep 
haproxy 46 root 4u unix 0x88102014e900 0t0 5699 /tmp/haproxy.45.tmp
haproxy 46 root 5u IPv4 5700 0t0 TCP *:jetcmeserver (LISTEN)
haproxy 46 root 6u IPv4 5701 0t0 TCP *:http (LISTEN)
haproxy 46 root 7u IPv4 5702 0t0 TCP *:mysql (LISTEN)

inspected old process with strace:
Process 46 attached
epoll_wait(0, {}, 200, 1000) = 0
epoll_wait(0, {}, 200, 1000) = 0
epoll_wait(0, {}, 200, 1000) = 0
epoll_wait(0, {}, 200, 1000) = 0
..

I don't know when did it block SIGUSR1 .