Hi all,
I am testing haproxy as follows:
System1: 24 Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz, 64 GB. This system
is running 3.19.0 kernel, and hosts the following servers:
1. nginx1 server - cpu 1-2, 1G memory, runs as a Linux
container using cpuset.cpus feature.
2. nginx2 server - cpu 3-4, 1G memory, runs via LXC.
3. nginx3 server - cpu 5-6, 1G memory, runs via LXC.
4. nginx4 server - cpu 7-8, 1G memory, runs via LXC.
5. haproxy - cpu 9-10, 1G memory runs via LXC. Runs haproxy
ver 1.5.8: configured with above 4 container's ip
addresses as the backend.
System2: 56 Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz, 128 GB. This system
is running 3.19.0, and run's 'ab' either to the haproxy node, or
directly to an nginx container. System1 & System2 are locally
connected via a switch with Intel 10G cards.
With very small packets of 64 bytes, I am getting the following results:
A. ab -n 100000 -c 4096 http://nginx1:80/64
-----------------------------------------
Concurrency Level: 4096
Time taken for tests: 3.232 seconds
Complete requests: 100000
Failed requests: 0
Total transferred: 28800000 bytes
HTML transferred: 6400000 bytes
Requests per second: 30943.26 [#/sec] (mean)
Time per request: 132.371 [ms] (mean)
Time per request: 0.032 [ms] (mean, across all concurrent requests)
Transfer rate: 8702.79 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 9 65 137.4 45 1050
Processing: 4 52 25.3 51 241
Waiting: 3 37 19.2 35 234
Total: 16 117 146.1 111 1142
Percentage of the requests served within a certain time (ms)
50% 111 66% 119 75% 122
80% 124 90% 133 95% 215
98% 254 99% 1126 100% 1142 (longest request)
B. ab -n 100000 -c 4096 http://haproxy:80/64
----------------------------------------------
Concurrency Level: 4096
Time taken for tests: 5.503 seconds
Complete requests: 100000
Failed requests: 0
Total transferred: 28800000 bytes
HTML transferred: 6400000 bytes
Requests per second: 18172.96 [#/sec] (mean)
Time per request: 225.390 [ms] (mean)
Time per request: 0.055 [ms] (mean, across all concurrent requests)
Transfer rate: 5111.15 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 134 358.3 23 3033
Processing: 2 61 47.7 51 700
Waiting: 2 50 43.0 42 685
Total: 7 194 366.7 79 3122
Percentage of the requests served within a certain time (ms)
50% 79 66% 105 75% 134
80% 159 90% 318 95% 1076
98% 1140 99% 1240 100% 3122 (longest request)
I expected haproxy to deliver better results with multiple connections,
since
haproxy will round-robin between the 4 servers. I have done no tuning, and
have
used the config file at the end of this mail. With 256K file size, the times
are slightly better for haproxy vs nginx. I notice that %requests served is
similar for both cases till about 90%.
Any help is very much appreciated.
------------------------------------------------------------------------------------------------------
A. haproxy config file:
-----------------------------
global
maxconn 65536
ulimit-n 65536
daemon
quiet
nbproc 2
user haproxy
group haproxy
defaults
#log global
mode http
option dontlognull
retries 3
option redispatch
maxconn 65536
timeout connect 5000
timeout client 50000
timeout server 50000
listen my_ha_proxy 192.168.1.110:80
mode http
stats enable
stats auth someuser:somepassword
balance roundrobin
cookie JSESSIONID prefix
option httpclose
option forwardfor
option httpchk HEAD /check.txt HTTP/1.0
server nginx1 192.168.1.101:80 check
server nginx2 192.168.1.102:80 check
server nginx3 192.168.1.103:80 check
server nginx4 192.168.1.104:80 check
B. haproxy version/build-option:
------------------------------------------
HA-Proxy version 1.5.8 2014/10/31
Copyright 2000-2014 Willy Tarreau <[email protected]>
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat
-Werror=format-security -D_FORTIFY_SOURCE=2
OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.7
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1e 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.30 2012-02-04
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
