RE: High Cpu usage
Cpu back at 100% after a few hours again, the maxaccept -1 is no solution.
Van: Remco Verhoef [remco.verh...@redfive.biz]
Verzonden: vrijdag 27 maart 2009 19:13
Aan: haproxy@formilux.org
Onderwerp: RE: High Cpu usage
I've added tune.maxaccept -1 to haproxy.cfg global, this seems to work with
1.3.16.
default maxaccept has been changed from 15.7 to 15.8 in haproxy.c:
if (global.tune.maxaccept <= 0) {
if (global.nbproc > 1)
global.tune.maxaccept = 8; /* leave some conns to
other processes */
else
global.tune.maxaccept = -1; /* accept all incoming
conns */
}
maxaccept is used in client.c for handling the events, and when max_accept is
-1, it tests the (p->feconn < p->maxconn ).
while (p->feconn < p->maxconn && max_accept--) {
struct sockaddr_storage addr;
socklen_t laddr = sizeof(addr);
Van: Remco Verhoef [remco.verh...@redfive.biz]
Verzonden: vrijdag 27 maart 2009 18:00
Aan: haproxy@formilux.org
Onderwerp: Re: High Cpu usage
I've also reverted to 1.3.15.8 but seems to have the same issue. Not tried the
1.3.15.7 yet, I've compared sources and it seems the maxaccept behaviour has
changed. Tried to add tune.maxaccept 100 to the cfg with version 1.3.16, but
issue still occurs.
The epoll_wait / poll and gettimeofday is called too often:
% time seconds usecs/call callserrors syscall
-- --- --- - -
47.810.005649 0169141 epoll_wait
42.820.005059 0169140 gettimeofday
% time seconds usecs/call callserrors syscall
-- --- --- - -
76.980.144607 0982302 poll
16.420.030855 0982303 gettimeofday
Van: Alex Forrow [a...@fav.or.it]
Verzonden: vrijdag 27 maart 2009 17:34
Aan: haproxy@formilux.org
Onderwerp: Re: High Cpu usage
2009/3/25 Corin Langosch :
/> Hi,/
/>/
/> I see exactly the same problem here, running a 2.6.26 kernel on
amd64./
/>/
/> 1.3.15.7 is working fine without any problems, so it seems to be
a bug in/
/> 1.3.16./
/>/
/> Corin/
/>/
I think so too, going back to version 1.3.15.7, cpu peaks at 2% with
the same load (about 10-20 concurrent sessions) and config.
/Sune
I am getting this also, on Linux 2.6.18 (CentOS 5) x86_64. Reverted to
1.3.15.7 and problem disappears.
Using a basic config doing simple http balancing.
Alex
RE: High Cpu usage
I've added tune.maxaccept -1 to haproxy.cfg global, this seems to work with
1.3.16.
default maxaccept has been changed from 15.7 to 15.8 in haproxy.c:
if (global.tune.maxaccept <= 0) {
if (global.nbproc > 1)
global.tune.maxaccept = 8; /* leave some conns to
other processes */
else
global.tune.maxaccept = -1; /* accept all incoming
conns */
}
maxaccept is used in client.c for handling the events, and when max_accept is
-1, it tests the (p->feconn < p->maxconn ).
while (p->feconn < p->maxconn && max_accept--) {
struct sockaddr_storage addr;
socklen_t laddr = sizeof(addr);
Van: Remco Verhoef [remco.verh...@redfive.biz]
Verzonden: vrijdag 27 maart 2009 18:00
Aan: haproxy@formilux.org
Onderwerp: Re: High Cpu usage
I've also reverted to 1.3.15.8 but seems to have the same issue. Not tried the
1.3.15.7 yet, I've compared sources and it seems the maxaccept behaviour has
changed. Tried to add tune.maxaccept 100 to the cfg with version 1.3.16, but
issue still occurs.
The epoll_wait / poll and gettimeofday is called too often:
% time seconds usecs/call callserrors syscall
-- --- --- - -
47.810.005649 0169141 epoll_wait
42.820.005059 0169140 gettimeofday
% time seconds usecs/call callserrors syscall
-- --- --- - -
76.980.144607 0982302 poll
16.420.030855 0982303 gettimeofday
Van: Alex Forrow [a...@fav.or.it]
Verzonden: vrijdag 27 maart 2009 17:34
Aan: haproxy@formilux.org
Onderwerp: Re: High Cpu usage
2009/3/25 Corin Langosch :
/> Hi,/
/>/
/> I see exactly the same problem here, running a 2.6.26 kernel on
amd64./
/>/
/> 1.3.15.7 is working fine without any problems, so it seems to be
a bug in/
/> 1.3.16./
/>/
/> Corin/
/>/
I think so too, going back to version 1.3.15.7, cpu peaks at 2% with
the same load (about 10-20 concurrent sessions) and config.
/Sune
I am getting this also, on Linux 2.6.18 (CentOS 5) x86_64. Reverted to
1.3.15.7 and problem disappears.
Using a basic config doing simple http balancing.
Alex
Re: High Cpu usage
I've also reverted to 1.3.15.8 but seems to have the same issue. Not tried the 1.3.15.7 yet, I've compared sources and it seems the maxaccept behaviour has changed. Tried to add tune.maxaccept 100 to the cfg with version 1.3.16, but issue still occurs. The epoll_wait / poll and gettimeofday is called too often: % time seconds usecs/call callserrors syscall -- --- --- - - 47.810.005649 0169141 epoll_wait 42.820.005059 0169140 gettimeofday % time seconds usecs/call callserrors syscall -- --- --- - - 76.980.144607 0982302 poll 16.420.030855 0982303 gettimeofday Van: Alex Forrow [a...@fav.or.it] Verzonden: vrijdag 27 maart 2009 17:34 Aan: haproxy@formilux.org Onderwerp: Re: High Cpu usage 2009/3/25 Corin Langosch : /> Hi,/ />/ /> I see exactly the same problem here, running a 2.6.26 kernel on amd64./ />/ /> 1.3.15.7 is working fine without any problems, so it seems to be a bug in/ /> 1.3.16./ />/ /> Corin/ />/ I think so too, going back to version 1.3.15.7, cpu peaks at 2% with the same load (about 10-20 concurrent sessions) and config. /Sune I am getting this also, on Linux 2.6.18 (CentOS 5) x86_64. Reverted to 1.3.15.7 and problem disappears. Using a basic config doing simple http balancing. Alex
Re: High Cpu usage
2009/3/25 Corin Langosch : /> Hi,/ />/ /> I see exactly the same problem here, running a 2.6.26 kernel on amd64./ />/ /> 1.3.15.7 is working fine without any problems, so it seems to be a bug in/ /> 1.3.16./ />/ /> Corin/ />/ I think so too, going back to version 1.3.15.7, cpu peaks at 2% with the same load (about 10-20 concurrent sessions) and config. /Sune I am getting this also, on Linux 2.6.18 (CentOS 5) x86_64. Reverted to 1.3.15.7 and problem disappears. Using a basic config doing simple http balancing. Alex
cpu 100% at strange times, epoll_wait and gettimeofday gets called too often
Hi,
We're experiencing strange behaviour of haproxy-1.3.15.8 and haproxy-1.3.16, at
frequent times it will use 100% cpu. It appears that it is wait_time is not
used. I've used both poll and epoll, same behaviour. The kernel is 2.6.26-1-686
#1 SMP.
The box is used, but not using many connections.
Any ideas?
Thanks,
Remco
config:
-
global
maxconn 1
log 127.0.0.1 local0
uid 200
gid 200
chroot /var/empty
daemon
tune.maxpollevents 200
nbproc 1
nosepoll
noepoll
strace with poll
Process 32211 attached - interrupt to quit
^CProcess 32211 detached
% time seconds usecs/call callserrors syscall
-- --- --- - -
76.980.144607 0982302 poll
16.420.030855 0982303 gettimeofday
3.250.006101 0 2694713 send
2.130.004000 0 58097 24425 recv
0.370.000687 0 1976 1976 connect
0.230.000425 0 2502 close
0.220.000413 1 526 sendto
0.190.000349 0 1976 socket
0.090.000173 0 1057 522 accept
0.080.000146 0 2511 fcntl64
0.030.65 0 2511 setsockopt
0.020.33 0 523 shutdown
0.000.00 0 1 restart_syscall
0.000.00 0 1 brk
0.000.00 0 1448 getsockopt
-- --- --- - -
100.000.187854 2064681 26936 total
==
strace with epoll
Process 31718 attached - interrupt to quit
^CProcess 31718 detached
% time seconds usecs/call callserrors syscall
-- --- --- - -
47.810.005649 0169141 epoll_wait
42.820.005059 0169140 gettimeofday
6.190.000731 0 281197 send
2.820.000333 0 6045 2716 recv
0.190.22 093 socket
0.180.21 0 156 getsockopt
0.000.00 0 107 close
0.000.00 0 109 fcntl64
0.000.00 0 221 1 epoll_ctl
0.000.00 09393 connect
0.000.00 03216 accept
0.000.00 015 sendto
0.000.00 014 shutdown
0.000.00 0 109 setsockopt
-- --- --- - -
100.000.011815348086 2923 total
==
strace attached with poll
---
gettimeofday({1238154432, 213206}, NULL) = 0
poll([{fd=0, events=POLLOUT}, {fd=1, events=POLLOUT}, {fd=2, events=POLLOUT},
{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7,
events=POLLIN}, {fd=8, events=POLLIN}, {fd=9, events=POLLOUT}, {fd=11,
events=POLLOUT}, {fd=13, events=POLLIN}, {fd=15, events=POLLOUT}, {fd=16,
events=POLLIN}, {fd=17, events=POLLIN}], 14, 0) = 0 (Timeout)
gettimeofday({1238154432, 213356}, NULL) = 0
poll([{fd=0, events=POLLOUT}, {fd=1, events=POLLOUT}, {fd=2, events=POLLOUT},
{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7,
events=POLLIN}, {fd=8, events=POLLIN}, {fd=9, events=POLLOUT}, {fd=11,
events=POLLOUT}, {fd=13, events=POLLIN}, {fd=15, events=POLLOUT}, {fd=16,
events=POLLIN}, {fd=17, events=POLLIN}], 14, 0) = 0 (Timeout)
gettimeofday({1238154432, 213506}, NULL) = 0
poll([{fd=0, events=POLLOUT}, {fd=1, events=POLLOUT}, {fd=2, events=POLLOUT},
{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7,
events=POLLIN}, {fd=8, events=POLLIN}, {fd=9, events=POLLOUT}, {fd=11,
events=POLLOUT}, {fd=13, events=POLLIN}, {fd=15, events=POLLOUT}, {fd=16,
events=POLLIN}, {fd=17, events=POLLIN}], 14, 0) = 0 (Timeout)
gettimeofday({1238154432, 213683}, NULL) = 0
poll([{fd=0, events=POLLOUT}, {fd=1, events=POLLOUT}, {fd=2, events=POLLOUT},
{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7,
events=POLLIN}, {fd=8, events=POLLIN}, {fd=9, events=POLLOUT}, {fd=11,
events=POLLOUT}, {fd=13, events=POLLIN}, {fd=15, events=POLLOUT}, {fd=16,
events=POLLIN}, {fd=17, events=POL
Re: some specfile fixes
Hi Jan-Frode, On Thu, Mar 26, 2009 at 03:45:53PM +0100, Jan-Frode Myklebust wrote: > > And here's the patch that does everything I want to do to the > specfile... Sorry about the noise. Thanks for your work on this. I have no way to test that the specfiles work, and I only update a few fields in them at each release. So it's really a good thing that someone like you checks them and proposes fixes. > It might also be nice to rename haproxy-small.spec to > haproxy.spec-small, then haproxy can be built directly from the tar.gz > by: > > rpmbuild -ta haproxy-1.3.16.tar.gz > > When there are two .spec-files, rpmbuild will concatinate both, and fail. OK fine, I wasn't aware of this. I'm even thinking that we can safely remove the "-small" variant right now. Cheers, Willy
