Re: Slow loading

Tue, 30 Mar 2010 17:06:59 -0700 

And pcre static is a separate package for Fedora/RedHat.

For RH/Fedora:

yum install pcre-devel pcre

make TARGET=linux26 ARCH==x86-64 USE_REGPARM=1 USE_PCRE=1

then you can do:
./haproxy -vv

to make sure you have several poll options, just having select seems strange.
Also, nbproc should be 1 I can push 70+ million hits/day and 100Mbps on a VM with nbproc = 1, so you should be ok with nbproc = 1 for non-extreme traffic. 
[Also, Willy will freak out if he sees you using nbproc > 1   :)  j/k]
If there are ip_conntrack problems, you will usually see some log messages in the system log, so when your traffic goes up generally ip_conntrack is the first bottleneck (the ip_conntrack people don't like to admit that, but...). 

what is your uname -a? Are you running a recent kernel?
Also, one common cause of pauses like this is dns/reverse dns lookups, so be sure those are turned off on your web server configs. I don't think this is the main problem in your case, but it is something to note. And for tcpdump, you usually want to turn that off tcpdump -nn... on linux: 
-n Don’t convert host addresses to names.  This can be used to avoid DNS 
lookups.
-nn Don’t convert protocol and port numbers etc. to names either.
You might want to try running the http part of your service in haproxy http mode and use the related http close options. 

On 3/30/10 8:57 AM, Malte Geierhos wrote:

Hi there,

either install a libpcre-dev package so you've got the header files - or
forget about that in the first shot,
and just make TARGET=linux26

regards,
Malte


I tried upgrading with the switches you recommended, but am getting
errors during the make. I've installed pcre. Any suggestions? Thanks!

[r...@rs-globallb1 haproxy-1.4.2]# make TARGET=linux26 USE_REGPARM=1
USE_STATIC_PCRE=1 USE_PCRE=1
gcc -Iinclude -Iebtree -Wall -O2 -g -DTPROXY -DCONFIG_HAP_CRYPT
-DENABLE_POLL -DENABLE_EPOLL -DENABLE_SEPOLL -DNETFILTER
-DUSE_GETSOCKNAME -DCONFIG_REGPARM=3 -DUSE_PCRE -I/usr/local/include
-DCONFIG_HAPROXY_VERSION=\"1.4.2\" -DCONFIG_HAPROXY_DATE=\"2010/03/17\" \
-DBUILD_TARGET='"linux26"' \
-DBUILD_ARCH='""' \
-DBUILD_CPU='"generic"' \
-DBUILD_CC='"gcc"' \
-DBUILD_CFLAGS='"-O2 -g"' \
-DBUILD_OPTIONS='"USE_REGPARM=1 USE_PCRE=1 USE_STATIC_PCRE=1"' \
-c -o src/haproxy.o src/haproxy.c
In file included from include/types/proxy.h:33,
from include/common/cfgparse.h:29,
from src/haproxy.c:54:
include/common/regex.h:28:18: error: pcre.h: No such file or directory
include/common/regex.h:29:23: error: pcreposix.h: No such file or
directory
In file included from include/types/proxy.h:33,
from include/common/cfgparse.h:29,
from src/haproxy.c:54:
include/common/regex.h:45: error: expected â:â, â,â, â;â, â}â or
â__attribute__â before â*â token
include/common/regex.h:51: error: expected â=â, â,â, â;â, âasmâ or
â__attribute__â before âpmatchâ
include/common/regex.h:53: warning: type defaults to âintâ in
declaration of âregmatch_tâ
include/common/regex.h:53: error: expected â;â, â,â or â)â before â*â
token
include/common/regex.h:55: warning: type defaults to âintâ in
declaration of âregex_tâ
include/common/regex.h:55: error: expected â;â, â,â or â)â before â*â
token
In file included from include/types/proxy.h:38,
from include/common/cfgparse.h:29,
from src/haproxy.c:54:
include/types/acl.h:218: error: expected specifier-qualifier-list
before âregex_tâ
src/haproxy.c: In function âdeinitâ:
src/haproxy.c:767: error: âstruct hdr_expâ has no member named âpregâ
src/haproxy.c:768: warning: implicit declaration of function âregfreeâ
src/haproxy.c:768: error: âregex_tâ undeclared (first use in this
function)
src/haproxy.c:768: error: (Each undeclared identifier is reported only
once
src/haproxy.c:768: error: for each function it appears in.)
src/haproxy.c:768: error: expected expression before â)â token
src/haproxy.c:769: error: expected expression before â)â token
src/haproxy.c:772: error: âstruct hdr_expâ has no member named âreplaceâ
src/haproxy.c:772: error: âstruct hdr_expâ has no member named âactionâ
src/haproxy.c:773: error: âstruct hdr_expâ has no member named âreplaceâ
src/haproxy.c:780: error: âstruct hdr_expâ has no member named âpregâ
src/haproxy.c:781: error: expected expression before â)â token
src/haproxy.c:782: error: expected expression before â)â token
src/haproxy.c:785: error: âstruct hdr_expâ has no member named âreplaceâ
src/haproxy.c:785: error: âstruct hdr_expâ has no member named âactionâ
src/haproxy.c:786: error: âstruct hdr_expâ has no member named âreplaceâ


On Tue, Mar 30, 2010 at 11:22 AM, Malte Geierhos <[email protected]
<mailto:[email protected]>> wrote:

    Hi Amanda,

    try - using 1.4.2 - the 1.4.1 has issues with chunked encoding
    under some circumstances ...
    maybe you just hit that issue.
    On the other hand ... there's something weird with your build, too...
    You should at least have more polling systems -> mine looks like
    this :

    Available polling systems :
    sepoll : pref=400, test result OK
    epoll : pref=300, test result OK
    poll : pref=200, test result OK

    select : pref=150, test result OK
    Total: 4 (4 usable), will use sepoll.
    Using sepoll() as the polling mechanism.

    Try building like this :
    make TARGET=linux26 USE_REGPARM=1 USE_STATIC_PCRE=1 USE_PCRE=1

    And with 1.4.2 ...

    nf_conntrack on a loadbalancer can have a huge impact if you're
    serving lots of requests - so
    on a production system i'd try to disable this ( you can define
    acl's in haproxy for disabling certain clients and so on)
    for the rest like ssh use tcpwrapper or let it listen on a
    dedicated ip.

    kind regards,
    Malte


    Am 30.03.10 16:57, schrieb Amanda Machutta:

    Just some more debugging data -- this is debuggng from
    haproxy....I marked where the pause happens here as well:

    Available polling systems :
    select : pref=150, test result OK
    Total: 1 (1 usable), will use select.
    Using select() as the polling mechanism.
    00000000:services.accept(0005)=0006 from [24.99.160.41:52838
    <http://24.99.160.41:52838>]
    00000001:http_proxy.accept(0003)=0008 from [24.99.160.41:52839
    <http://24.99.160.41:52839>]
    00000001:http_proxy.clireq[0008:ffff]: GET /intelliqueue/agentui/
    HTTP/1.1
    00000001:http_proxy.clihdr[0008:ffff]: Host: www.cfpolitical.com
    <http://www.cfpolitical.com>
    00000001:http_proxy.clihdr[0008:ffff]: User-Agent: Mozilla/5.0
    (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.2) Gecko/20100316
    Firefox/3.6.2 (.NET CLR 3.5.30729)
    00000001:http_proxy.clihdr[0008:ffff]: Accept:
    text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    00000001:http_proxy.clihdr[0008:ffff]: Accept-Language:
    en-us,en;q=0.5
    00000001:http_proxy.clihdr[0008:ffff]: Accept-Encoding: gzip,deflate
    00000001:http_proxy.clihdr[0008:ffff]: Accept-Charset:
    ISO-8859-1,utf-8;q=0.7,*;q=0.7
    00000001:http_proxy.clihdr[0008:ffff]: Keep-Alive: 115
    00000001:http_proxy.clihdr[0008:ffff]: Connection: keep-alive
    00000001:http_proxy.clihdr[0008:ffff]: Cookie: CFID=408;
    CFTOKEN=12213899; JSESSIONID=b6302e0e4a8d377c4e6e
    00000001:http_proxy.srvrep[0008:0009]: HTTP/1.1 200 OK
    00000001:http_proxy.srvhdr[0008:0009]: Connection: close
    00000001:http_proxy.srvhdr[0008:0009]: Date: Tue, 30 Mar 2010
    14:55:39 GMT
    00000001:http_proxy.srvhdr[0008:0009]: Server: Microsoft-IIS/6.0
    00000001:http_proxy.srvhdr[0008:0009]: X-Powered-By: ASP.NET
    <http://ASP.NET>
    00000001:http_proxy.srvhdr[0008:0009]: Set-Cookie:
    CFAUTHORIZATION_IntelliQueue=;expires=Mon, 30-Mar-2009 14:55:39
    GMT;path=/
    00000001:http_proxy.srvhdr[0008:0009]: Content-Type: text/html;
    charset=UTF-8
    00000000:services.srvcls[0006:0007]
    00000000:services.clicls[0006:0007]
    00000000:services.closed[0006:0007]

    <!--- 16 second pause --->

    00000001:http_proxy.srvcls[0008:0009]
    00000001:http_proxy.clicls[0008:0009]
    00000001:http_proxy.closed[0008:0009]
    00000002:http_proxy.accept(0003)=0006 from [24.99.160.41:52843
    <http://24.99.160.41:52843>]
    00000002:http_proxy.clireq[0006:ffff]: GET
    /intelliqueue/agentui/config.cfm HTTP/1.1
    00000002:http_proxy.clihdr[0006:ffff]: Host: www.cfpolitical.com
    <http://www.cfpolitical.com>
    00000002:http_proxy.clihdr[0006:ffff]: User-Agent: Mozilla/5.0
    (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.2) Gecko/20100316
    Firefox/3.6.2 (.NET CLR 3.5.30729)
    00000002:http_proxy.clihdr[0006:ffff]: Accept:
    text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    00000002:http_proxy.clihdr[0006:ffff]: Accept-Language:
    en-us,en;q=0.5
    00000002:http_proxy.clihdr[0006:ffff]: Accept-Encoding: gzip,deflate
    00000002:http_proxy.clihdr[0006:ffff]: Accept-Charset:
    ISO-8859-1,utf-8;q=0.7,*;q=0.7
    00000002:http_proxy.clihdr[0006:ffff]: Keep-Alive: 115
    00000002:http_proxy.clihdr[0006:ffff]: Connection: keep-alive
    00000002:http_proxy.clihdr[0006:ffff]: Cookie: CFID=408;
    CFTOKEN=12213899; JSESSIONID=b6302e0e4a8d377c4e6e
    00000002:http_proxy.srvrep[0006:0007]: HTTP/1.1 200 OK
    00000002:http_proxy.srvhdr[0006:0007]: Connection: close
    00000002:http_proxy.srvhdr[0006:0007]: Date: Tue, 30 Mar 2010
    14:55:56 GMT
    00000002:http_proxy.srvhdr[0006:0007]: Server: Microsoft-IIS/6.0
    00000002:http_proxy.srvhdr[0006:0007]: X-Powered-By: ASP.NET
    <http://ASP.NET>
    00000002:http_proxy.srvhdr[0006:0007]: Set-Cookie:
    CFAUTHORIZATION_IntelliQueue=;expires=Mon, 30-Mar-2009 14:55:56
    GMT;path=/
    00000002:http_proxy.srvhdr[0006:0007]: Content-Type: text/xml
    00000002:http_proxy.srvcls[0006:0007]
    00000002:http_proxy.clicls[0006:0007]
    00000002:http_proxy.closed[0006:0007]
    00000003:http_proxy.accept(0003)=0006 from [24.99.160.41:52844
    <http://24.99.160.41:52844>]
    00000003:http_proxy.clireq[0006:ffff]: GET
    /intelliqueue/manage/get_logo.cfm?ui=TRUE HTTP/1.1
    00000003:http_proxy.clihdr[0006:ffff]: Host: www.cfpolitical.com
    <http://www.cfpolitical.com>
    00000003:http_proxy.clihdr[0006:ffff]: User-Agent: Mozilla/5.0
    (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.2) Gecko/20100316
    Firefox/3.6.2 (.NET CLR 3.5.30729)
    00000003:http_proxy.clihdr[0006:ffff]: Accept:
    text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    00000003:http_proxy.clihdr[0006:ffff]: Accept-Language:
    en-us,en;q=0.5
    00000003:http_proxy.clihdr[0006:ffff]: Accept-Encoding: gzip,deflate
    00000003:http_proxy.clihdr[0006:ffff]: Accept-Charset:
    ISO-8859-1,utf-8;q=0.7,*;q=0.7
    00000003:http_proxy.clihdr[0006:ffff]: Keep-Alive: 115
    00000003:http_proxy.clihdr[0006:ffff]: Connection: keep-alive
    00000003:http_proxy.clihdr[0006:ffff]: Cookie: CFID=408;
    CFTOKEN=12213899; JSESSIONID=b6302e0e4a8d377c4e6e
    00000003:http_proxy.srvrep[0006:0007]: HTTP/1.1 200 OK
    00000003:http_proxy.srvhdr[0006:0007]: Connection: close
    00000003:http_proxy.srvhdr[0006:0007]: Date: Tue, 30 Mar 2010
    14:55:56 GMT
    00000003:http_proxy.srvhdr[0006:0007]: Server: Microsoft-IIS/6.0
    00000003:http_proxy.srvhdr[0006:0007]: X-Powered-By: ASP.NET
    <http://ASP.NET>
    00000003:http_proxy.srvhdr[0006:0007]: Content-Type: image/png
    00000003:http_proxy.srvcls[0006:0007]
    00000003:http_proxy.clicls[0006:0007]
    00000003:http_proxy.closed[0006:0007]
    00000004:services.accept(0005)=0006 from [24.99.160.41:52846
    <http://24.99.160.41:52846>]
    00000004:services.srvcls[0006:0007]
    00000004:services.clicls[0006:0007]
    00000004:services.closed[0006:0007]
    00000005:services.accept(0005)=0006 from [24.99.160.41:52847
    <http://24.99.160.41:52847>]

Reply via email to