RE: Can't get over 100 client connections?
Hi Chris
> -Original Message-
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Sent: Thursday, May 18, 2017 12:06 AM
> To: users@tomcat.apache.org
> Subject: Re: Can't get over 100 client connections?
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> John,
>
> On 5/16/17 1:57 PM, john.e.gr...@wellsfargo.com.INVALID wrote:
> > All,
> >
> > I'm using Tomcat 7.0.75.
> >
> > I cannot get my connection or thread count over 100 no matter how much
> > load I throw at the server. Currently I just have a dummy app
> > deployed that doesn't do much except sleep for about 500ms and return
> > a canned response. Initially I had maxThreads=20 with no explicit
> > maxConnections. I could get the "current threads busy"
> > metric to 20 easily. Then I raised it to 100 and was able to reach
> > that number easily. Additionally, netstat told me I had 100 incoming
> > connections to port 5114. However raising the maxThreads higher
> > doesn't make any difference, nor does explicitly specifying
> > maxConnections=200. In either case, I can only get 100 busy threads
> > and 100 active connections. Once I reach that limit, the response
> > times on the client start going up. The Tomcat server itself isn't
> > stressed at all. CPU and GC are low. The client is on a different
> > server in the same data center.
> >
> > I'm not going through a load balancer. I'm going directly to the
> > connector below:
> >
> >
> > > maxConnections="200" maxThreads="200" maxKeepAliveRequests="100"
> > keepAliveTimeout="1" scheme="https" secure="true"
> > clientAuth="true" sslProtocol="TLS"
> > keystoreFile="/app/comp/myapp/certs/appcerttestkeystore"
> > keystorePass="${keystore.password}" keyAlias="test"
> > truststoreFile="/app/comp/myapp/certs/cacerts"
> > truststorePass="${truststore.password}"
> > allowUnsafeLegacyRenegotiation="false" ciphers="blah blah blah" />
>
> What is the benchmark client and how do you invoke it?
>
> - -chris
LoadRunner --> vendor app using Commons HttpClient 4.x --> my Tomcat server.
Tomcat is running an app I wrote to stub out the backend because we're not
allowed to run load tests against it. I confirmed that Tomcat itself could
handle more than 100 connections by running the vendor app under load, as well
as a test from JMeter directly against my Tomcat server. Worked fine. It's
the vendor app that appears to have hit a 100 connection limit.
Interestingly, JMeter with the option to use HttpClient 4.x opens and closes a
lot of connections. The exact same JMeter test with HttpClient 3.x selected
reuses the connection the way I expect.
If you're on the HttpClient users list, you might see a question from me! I
don't understand why HttpClient is behaving the way it is.
Thanks
Re: Can't get over 100 client connections?
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
John,
On 5/16/17 1:57 PM, john.e.gr...@wellsfargo.com.INVALID wrote:
> All,
>
> I'm using Tomcat 7.0.75.
>
> I cannot get my connection or thread count over 100 no matter how
> much load I throw at the server. Currently I just have a dummy app
> deployed that doesn't do much except sleep for about 500ms and
> return a canned response. Initially I had maxThreads=20 with no
> explicit maxConnections. I could get the "current threads busy"
> metric to 20 easily. Then I raised it to 100 and was able to reach
> that number easily. Additionally, netstat told me I had 100
> incoming connections to port 5114. However raising the maxThreads
> higher doesn't make any difference, nor does explicitly specifying
> maxConnections=200. In either case, I can only get 100 busy
> threads and 100 active connections. Once I reach that limit, the
> response times on the client start going up. The Tomcat server
> itself isn't stressed at all. CPU and GC are low. The client is
> on a different server in the same data center.
>
> I'm not going through a load balancer. I'm going directly to the
> connector below:
>
>
> maxConnections="200" maxThreads="200" maxKeepAliveRequests="100"
> keepAliveTimeout="1" scheme="https" secure="true"
> clientAuth="true" sslProtocol="TLS"
> keystoreFile="/app/comp/myapp/certs/appcerttestkeystore"
> keystorePass="${keystore.password}" keyAlias="test"
> truststoreFile="/app/comp/myapp/certs/cacerts"
> truststorePass="${truststore.password}"
> allowUnsafeLegacyRenegotiation="false" ciphers="blah blah blah" />
What is the benchmark client and how do you invoke it?
- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlkdK6sACgkQHPApP6U8
pFh1zxAAlYXrL7LVfojOlVFfXizjNCC9ioE1ogn2557ga0lpU34D58FxtnSeP+DK
EywaYVNAcWzckFcoTc796GJJ9DVnmZN+rbGosT5jPR42pLYw9N2M6N3MVp0PQ/Tn
hud1X9giCDIqgC3YsuxtMqfwU4haVS7+qARrfpdVf9s+J0NMjTQK1AYmM78CSLeZ
RRe13eQDZFShl4cVrJhuRilu0Bf8vKSSMXFUCPpaZo1yU6Js6IVfS+YZwTpwjtfO
EYQQjQ6t5cplpv7fvGiiiOEf43/tTH+Cyii0M2VmdYnhDh3/WJUClbVtLE2zkVSY
7TG32sSGrjpUOS5/Zucbfjsc8G5E5y9AFdZctGK9A513yiLr+uB+YEG7TehwNelN
Z3wUGKZ6KXeAtjs2HMfg2tZ+6G2VMKUWAcn2+AC/vhEfq2DjdDn4KNmFma1WMVMW
k3J8khO7zbGyWFuzOyFrZ3dQyjGwWhLsmeiALphLkIKSVcUTD93rIidlvwEu2P3x
yUDeo+sNqwu0tr5j3b+CLPSFT8rClEIuG/yw3SFpeB/6XSOwJt5U5zspVVI7HRgd
7f0q9rVCy4mL9241CF0ntV9XN5EIpRi9EnPL2nXCz2eZTWBEGVjyo0CL+z8m5DZD
skw0vv6f6AlKjYrpk5Kt4c3zJFynUf8nP2B1xJ4NFggIYWQS6Ss=
=luv6
-END PGP SIGNATURE-
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Can't get over 100 client connections?
André,
> -Original Message-
> From: André Warnier (tomcat) [mailto:a...@ice-sa.com]
> Sent: Tuesday, May 16, 2017 2:00 PM
> To: users@tomcat.apache.org
> Subject: Re: Can't get over 100 client connections?
>
> On 16.05.2017 19:57, john.e.gr...@wellsfargo.com.INVALID wrote:
> > All,
> >
> > I'm using Tomcat 7.0.75.
> >
> > I cannot get my connection or thread count over 100 no matter how much
> load I throw at the server. Currently I just have a dummy app deployed that
> doesn't do much except sleep for about 500ms and return a canned
> response. Initially I had maxThreads=20 with no explicit maxConnections. I
> could get the "current threads busy" metric to 20 easily. Then I raised it to
> 100 and was able to reach that number easily. Additionally, netstat told me I
> had 100 incoming connections to port 5114. However raising the maxThreads
> higher doesn't make any difference, nor does explicitly specifying
> maxConnections=200. In either case, I can only get 100 busy threads and 100
> active connections. Once I reach that limit, the response times on the client
> start going up. The Tomcat server itself isn't stressed at all. CPU and GC
> are
> low. The client is on a different server in the same data center.
> >
> > I'm not going through a load balancer. I'm going directly to the connector
> below:
> >
> >
> > > protocol="HTTP/1.1"
> > SSLEnabled="true"
> > maxConnections="200"
> > maxThreads="200"
> > maxKeepAliveRequests="100"
> > keepAliveTimeout="1"
> > scheme="https"
> > secure="true"
> > clientAuth="true"
> > sslProtocol="TLS"
> > keystoreFile="/app/comp/myapp/certs/appcerttestkeystore"
> > keystorePass="${keystore.password}"
> > keyAlias="test"
> > truststoreFile="/app/comp/myapp/certs/cacerts"
> > truststorePass="${truststore.password}"
> > allowUnsafeLegacyRenegotiation="false"
> > ciphers="blah blah blah"
> > />
> >
>
> Hi.
> I do not know with what you are testing (as a client).
> But be aware of the following :
>
> 1) > keepAliveTimeout="1"
> means 10 seconds.
> It means that, after the last request which one particular client sends on its
> connection to Tomcat, and Tomcat has responded to it, Tomcat will keep that
> connection open for an additional 10 s., just waiting to see if that same
> client
> has anything more to request.
> Since you are not using an Executor, keeping the connection open will also
> mean keeping the corresponding Tomcat thread alive, also waiting.
> Only once this time is over, will Tomcat close this connection, and "recycle"
> the thread to serve another client connection.
> 2) there may be a limit in the server OS, as to how many connections a
> process can have open at the same time. If that limit is reached at some
> point, that may either crash the process that wants an additional one, or put
> it in some wait queue until one is available again.
> 3) when a client opens a connection to a server (or tries to), and the server
> process does not immediately respond to the "open connection" request,
> the TCP/IP stack on the server will place the connection-open request in a
> wait queue. The size of that queue is an adjustable TCP/IP parameter.
> From the client side, if its connection is not accepted immediately (but not
> rejected right away), the client will just wait, until it is accepted. There
> is
> usually a timeout for this also, on the client side.
>
> Some combination of the above may explain what you see.
>
Thanks for the suggestions. I don't know 100% the cause yet but it appears to
be an issue on the client side. When running multiple clients I can get the
concurrent requests over 100 with no trouble.
Thanks
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Can't get over 100 client connections?
On 16.05.2017 19:57, john.e.gr...@wellsfargo.com.INVALID wrote: All, I'm using Tomcat 7.0.75. I cannot get my connection or thread count over 100 no matter how much load I throw at the server. Currently I just have a dummy app deployed that doesn't do much except sleep for about 500ms and return a canned response. Initially I had maxThreads=20 with no explicit maxConnections. I could get the "current threads busy" metric to 20 easily. Then I raised it to 100 and was able to reach that number easily. Additionally, netstat told me I had 100 incoming connections to port 5114. However raising the maxThreads higher doesn't make any difference, nor does explicitly specifying maxConnections=200. In either case, I can only get 100 busy threads and 100 active connections. Once I reach that limit, the response times on the client start going up. The Tomcat server itself isn't stressed at all. CPU and GC are low. The client is on a different server in the same data center. I'm not going through a load balancer. I'm going directly to the connector below: Hi. I do not know with what you are testing (as a client). But be aware of the following : 1) > keepAliveTimeout="1" means 10 seconds. It means that, after the last request which one particular client sends on its connection to Tomcat, and Tomcat has responded to it, Tomcat will keep that connection open for an additional 10 s., just waiting to see if that same client has anything more to request. Since you are not using an Executor, keeping the connection open will also mean keeping the corresponding Tomcat thread alive, also waiting. Only once this time is over, will Tomcat close this connection, and "recycle" the thread to serve another client connection. 2) there may be a limit in the server OS, as to how many connections a process can have open at the same time. If that limit is reached at some point, that may either crash the process that wants an additional one, or put it in some wait queue until one is available again. 3) when a client opens a connection to a server (or tries to), and the server process does not immediately respond to the "open connection" request, the TCP/IP stack on the server will place the connection-open request in a wait queue. The size of that queue is an adjustable TCP/IP parameter. From the client side, if its connection is not accepted immediately (but not rejected right away), the client will just wait, until it is accepted. There is usually a timeout for this also, on the client side. Some combination of the above may explain what you see. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
