Willy Tarreau <w <at> 1wt.eu> writes: > > On Fri, Oct 11, 2013 at 02:43:35AM +0000, steve wrote: > > I have been working on trouble shooting Haproxy 1.5 dev 19 with SSL for the > > last day or so on Cent OS 6.4 64bit. > > > > Lastest OpenSSL compiled 1.0.1e, recompiled haproxy with this make -s > > TARGET=linux2628 USE_EPOLL=1 USE_OPENSSL=1 ARCH=x86_64 clean all > > > > SSL cert wild card, plus godaddy intermediate and our key. > > > > Our current set of issues we are seeing: > > *Massive amounts of connection refused when running the test with ssl > > *Very High usage of CPU on this 8 core 32 gig box with 100 gig ssd and 1gb > > nic > > *Maybe 1/4 the amount of traffic we can push though, compaired to a non ssl > > test > > > > We are using Jmeter to load test and blazemeter to do up to 40k jmeter > > threads for a full hour. > > Are you sure your haproxy settings support these 40k concurrent connections ?
thats a good question, posted above the config.. we are still getting our feet wet with this system. > > > Here is a list of the errors that are spit back after the test is done > > Response codes > > > > response codecountresponse message > > 400 29 Bad request > > This means SSL could pass through but that it's the tester which is sending > bad requests. Quite concerning in fact because from this point it's permitted > to doubt about everything else... > > > Non HTTP response code: javax.net.ssl.SSLPeerUnverifiedException > > 86069 Non HTTP response message: peer not authenticated > > Possibly aborted handshakes. > > > Non HTTP response code: org.apache.http.conn.HttpHostConnectException 27229 > > Non HTTP response message: Connection to https://xxxx.com:2222 refused > > Huh ? did you stop and restart haproxy during the test ? Are you sure > the connectivity between the client and haproxy is OK ? A connection > refused can only happen when either the server is stopped or when there > is one component between the client and the server which explicitly sends > RST packets (eg: a firewall), or some ICMP admin prohibited packets (eg: > a router). > This might of been the case, we are about to run another load test and make sure nothing else is done during this 1 hour test. > > Non HTTP response code: java.net.SocketException > > 88 Non HTTP response message: Connection reset > > > > 4122 Precondition Failed > > Non HTTP response code: org.apache.http.NoHttpResponseException270Non HTTP > > response message: The target server failed to respond > > > > So this is what we are facing and we are not haproxy experts and think we > > have taken it to the best of what we understand about haproxy config and > > settings. > > > > special note: we do not have a web site on the backend, its user server for > > an upcoming game we are working on so the stack is quite simple from haproxy > > -> node.js --> db and back. > > > > Json data is posted to the user server and returned. > > OK. Anyway this is like a website and it must work! > > You need to check haproxy's logs to see if it *receives* the requests that > are reported to fail, or if it logs failed handshakes. > > Since you're reporting a high CPU usage, it is also possible that the client > renegociates a new key for each request, which might or might not match what > you expect from your target. For example, if each of your clients does only > one request and leaves, this is OK. But if you have only 40k concurrent > clients which do a number of requests, they will only negociate once at the > beginning of their session. > So.. If I understand, ip comes in.. it creates a new registration for a user (if they are not already in the game), data is return to the client, after a a few seconds or so a game update is passed to the user server to store the latest changes and a reply is sent back, then its rinse and repeat for this user. Ideally we only need one session for this.. instead of 1 for reg, 1 for each game state or call until they finish.. ? Is that what your getting at, because I *think* we might be doing one for everyone.. not sure tho. We are also looking at terminating the ssl on the backends instead of haproxy and passing it though tcp via haproxy which might spread the ssl load a bit more how ever with our load tests we only have a few ips so it seems that only certain backend servers get hit since its from the same ip.. ? thanks! > Regards, > Willy > >
