OK, I've decided to go a bit easier on the stress tools ;), and limited
concurrency to 500, using siege. That's resulted in some useful data
(attached).
HAProxy HTTP endpoint, with 1 process, handled up to 500K total requests
without breaking a sweat, in just under 30 seconds, achieving an actual
concurrency of 160-170 (I'm guessing the requests finished too fast to go any
higher). For some reason, the longest request took almost all that time; maybe
that's again a limitation of the stress tool? Higher request volumes introduced
a small percentage of timeouts.
Increasing to two HAProxy processes made a marked difference. At 500K requests,
total time halved to approximately 10-15 seconds. At higher volumes, timeouts
were much rarer; I was able to run 5 tests at 1.5M requests each with no
timeouts at all. Just for fun, I increased to 750 concurrency (2.25M requests),
with no trouble (the same test with 1 proc resulted in 57 timeouts and about
30% longer run time).
PHP surprised me, actually; its performance characteristics were somewhat
better than HAProxy with 1 process (though I don't know how the memory usage
compared). However, compared to HAProxy with two processes, it had a somewhat
higher error rate and noticeably slower response time.
Looks like the conclusion is: use HAProxy, but tune nbproc wisely :)
On Thursday, 19 November 2015, 9:49, Willy Tarreau <[email protected]> wrote:
On Wed, Nov 18, 2015 at 10:53:18PM +0000, Thrawn wrote:
> Thanks, Willy.
> I've likewise noticed that ab didn't seem capable of going beyond about 1024
> concurrency. I've also tried siege, since it's available in the Ubuntu
> repositories and has a decent reputation, but it keeps hitting errors, too:
> [error] descriptor table full sock.c:119: Too many open files
> or after setting ulimit -n 10000,
> *** buffer overflow detected ***: siege terminated
"great" ...
> That's with --reps=30 and --concurrent=1000.
> So it may be that HAProxy is actually more capable than the stress tool(s)...
> I'll see whether I can draw some meaningful comparisons between Lua and PHP
> at lower concurrency.
Indeed sometimes that's true. Just yesterday I had to set up a second
machine to host the server during a test because I wasn't able to have
both the client and the server on the same machine to hammer haproxy.
Willy
Date & Time, Trans, Elap Time, Data Trans, Resp Time, Trans Rate, Throughput, Concurrent, OKAY, Failed
--concurrent=500
2015-11-19 12:22:54, 500000, 28.98, 4, 0.01, 17253.28, 0.14, 164.61, 500000, 0
2015-11-19 12:23:27, 500000, 27.32, 4, 0.01, 18301.61, 0.15, 170.11, 500000, 0
2015-11-19 12:29:44, 500000, 27.39, 4, 0.01, 18254.84, 0.15, 159.64, 500000, 0
2015-11-19 12:46:38, 499970, 32.20, 4, 0.01, 15527.02, 0.12, 129.63, 499970, 30
2015-11-19 12:47:28, 500000, 14.40, 4, 0.01, 34722.22, 0.28, 329.47, 500000, 0
2015-11-19 12:24:52, 749985, 31.74, 6, 0.01, 23629.02, 0.19, 209.15, 749985, 15
2015-11-19 12:25:39, 749955, 32.81, 6, 0.01, 22857.51, 0.18, 174.32, 749955, 45
2015-11-19 12:30:27, 750000, 29.08, 6, 0.01, 25790.92, 0.21, 253.69, 750000, 0
2015-11-19 12:44:32, 749989, 31.56, 6, 0.01, 23763.91, 0.19, 194.78, 749989, 11
2015-11-19 12:45:11, 749965, 32.46, 6, 0.01, 23104.28, 0.18, 191.89, 749965, 35
2015-11-19 12:20:41, 999975, 32.20, 8, 0.01, 31055.12, 0.25, 276.73, 999975, 25
2015-11-19 12:21:26, 999994, 33.50, 8, 0.01, 29850.57, 0.24, 285.38, 999994, 6
2015-11-19 12:22:18, 999969, 33.38, 8, 0.01, 29957.13, 0.24, 260.23, 999969, 31
2015-11-19 12:41:46, 999947, 33.63, 8, 0.01, 29733.78, 0.24, 242.28, 999947, 53
2015-11-19 12:42:31, 999999, 31.27, 8, 0.01, 31979.50, 0.26, 317.10, 999999, 1
2015-11-19 13:31:04,1499983, 38.05, 12, 0.01, 39421.37, 0.32, 390.14, 1499983, 17
2015-11-19 13:31:56,1499989, 36.71, 12, 0.01, 40860.50, 0.33, 407.91, 1499989, 11
2015-11-19 13:32:52,1499996, 42.09, 12, 0.01, 35637.82, 0.29, 373.48, 1499996, 4
2015-11-19 13:34:17,1499940, 48.67, 12, 0.01, 30818.58, 0.25, 270.53, 1499940, 60
2015-11-19 13:35:53,1499928, 39.12, 12, 0.01, 38341.72, 0.31, 312.84, 1499928, 72
--concurrent=750
2015-11-19 12:56:51,2249943, 65.67, 19, 0.02, 34261.35, 0.29, 525.41, 2249943, 57
Date & Time, Trans, Elap Time, Data Trans, Resp Time, Trans Rate, Throughput, Concurrent, OKAY, Failed
--concurrent=500
2015-11-19 12:32:19, 500000, 10.61, 4, 0.01, 47125.36, 0.38, 329.42, 500000, 0
2015-11-19 12:32:48, 500000, 14.17, 4, 0.01, 35285.82, 0.28, 253.66, 500000, 0
2015-11-19 12:33:24, 500000, 10.87, 4, 0.01, 45998.16, 0.37, 332.58, 500000, 0
2015-11-19 12:49:27, 500000, 27.08, 4, 0.01, 18463.81, 0.15, 136.43, 500000, 0
2015-11-19 12:49:49, 500000, 14.01, 4, 0.01, 35688.79, 0.29, 246.17, 500000, 0
2015-11-19 12:34:00, 750000, 27.30, 6, 0.01, 27472.53, 0.22, 202.30, 750000, 0
2015-11-19 12:34:41, 749979, 31.67, 6, 0.01, 23681.05, 0.19, 171.51, 749979, 21
2015-11-19 12:35:30, 750000, 26.96, 6, 0.01, 27818.99, 0.22, 215.44, 750000, 0
2015-11-19 12:36:19, 750000, 26.88, 6, 0.01, 27901.79, 0.22, 214.13, 750000, 0
2015-11-19 12:37:00, 750000, 15.16, 6, 0.01, 49472.30, 0.40, 388.68, 750000, 0
2015-11-19 12:37:37,1000000, 27.33, 8, 0.01, 36589.83, 0.29, 313.00, 1000000, 0
2015-11-19 12:38:20, 999991, 31.45, 8, 0.01, 31796.21, 0.25, 260.17, 999991, 9
2015-11-19 12:38:50,1000000, 27.13, 8, 0.01, 36859.57, 0.29, 309.87, 1000000, 0
2015-11-19 12:39:37, 999991, 31.50, 8, 0.01, 31745.75, 0.25, 246.82, 999991, 9
2015-11-19 12:40:24,1000000, 29.23, 8, 0.01, 34211.43, 0.27, 289.80, 1000000, 0
2015-11-19 12:50:39,1500000, 30.78, 12, 0.01, 48732.94, 0.39, 427.32, 1500000, 0
2015-11-19 12:51:17,1500000, 31.29, 12, 0.01, 47938.64, 0.38, 426.87, 1500000, 0
2015-11-19 12:51:52,1500000, 30.28, 12, 0.01, 49537.65, 0.40, 417.86, 1500000, 0
2015-11-19 12:52:33,1500000, 30.57, 12, 0.01, 49067.71, 0.39, 427.68, 1500000, 0
2015-11-19 12:53:07,1500000, 31.49, 12, 0.01, 47634.17, 0.38, 419.04, 1500000, 0
--concurrent=750
2015-11-19 12:54:06,2250000, 48.34, 19, 0.01, 46545.30, 0.39, 671.14, 2250000, 0
2015-11-19 12:55:07,2250000, 49.68, 19, 0.01, 45289.86, 0.38, 666.21, 2250000, 0
Date & Time, Trans, Elap Time, Data Trans, Resp Time, Trans Rate, Throughput, Concurrent, OKAY, Failed
--concurrent=500
2015-11-19 13:00:42, 500000, 27.66, 0, 0.02, 18076.64, 0.00, 305.26, 500000, 0
2015-11-19 13:01:11, 500000, 19.76, 0, 0.02, 25303.64, 0.00, 436.86, 500000, 0
2015-11-19 13:01:35, 500000, 19.89, 0, 0.02, 25138.26, 0.00, 434.78, 500000, 0
2015-11-19 13:02:05, 500000, 25.99, 0, 0.02, 19238.17, 0.00, 342.75, 500000, 0
2015-11-19 13:04:43, 499999, 37.63, 0, 0.02, 13287.24, 0.00, 222.39, 499999, 1
2015-11-19 13:05:41, 749999, 37.84, 0, 0.02, 19820.27, 0.00, 343.39, 749999, 1
2015-11-19 13:06:14, 750000, 30.21, 0, 0.02, 24826.22, 0.00, 433.47, 750000, 0
2015-11-19 13:06:53, 750000, 31.20, 0, 0.02, 24038.46, 0.00, 446.63, 750000, 0
2015-11-19 13:07:39, 749999, 36.88, 0, 0.02, 20336.20, 0.00, 367.93, 749999, 1
2015-11-19 13:08:14, 750000, 31.53, 0, 0.02, 23786.87, 0.00, 427.14, 750000, 0
2015-11-19 13:09:02,1000000, 39.26, 0, 0.02, 25471.22, 0.00, 461.43, 1000000, 0
2015-11-19 13:09:54,1000000, 39.46, 0, 0.02, 25342.12, 0.00, 456.17, 1000000, 0
2015-11-19 13:17:11, 999979, 40.07, 0, 0.02, 24955.80, 0.00, 422.34, 999979, 21
2015-11-19 13:17:53,1000000, 40.12, 0, 0.02, 24925.22, 0.00, 456.94, 1000000, 0
2015-11-19 13:18:44, 999979, 39.59, 0, 0.02, 25258.37, 0.00, 416.96, 999979, 21
2015-11-19 13:20:39,1499997, 60.59, 0, 0.02, 24756.51, 0.00, 464.05, 1499997, 3
2015-11-19 13:21:49,1500000, 59.55, 0, 0.02, 25188.92, 0.00, 467.73, 1500000, 0
2015-11-19 13:23:01,1499990, 59.91, 0, 0.02, 25037.39, 0.00, 455.65, 1499990, 10
2015-11-19 13:24:18,1500000, 60.20, 0, 0.02, 24916.94, 0.00, 458.54, 1500000, 0
2015-11-19 13:25:25,1500000, 60.39, 0, 0.02, 24838.55, 0.00, 468.18, 1500000, 0
--concurrent=750
2015-11-19 13:27:17,2249978, 95.52, 0, 0.03, 23555.05, 0.00, 688.53, 2249978, 22
2015-11-19 13:29:36,2249998, 95.60, 0, 0.03, 23535.54, 0.00, 700.56, 2249998, 2
