RE: Tomcat Behavior on Multiple HTTP requests from same browser
Hi Nick, Thanks for your valuable feedback. The point I was trying to make when I mentioned that all requests from the same browser results in same thread is not a mere coincidence. The reason why it may not be sheer luck is because we have seen this on production which is accessed by more than 1000 simultaneous users on the site. Currently we trace the user journey by the thread id in the logs and my observation is all 1000 users (across the globe) gets the same thread id from the pool when they access subsequent URLS from the browser. It makes me feel as if the thread is not returned to the pool till the session times out or the user has closed the browser. Although the behavior is weird because tomcat won't should not have a persistent connection as it would mean improper use of the resources (thread). I read the connector documentation and my gathering is threads are allocated per request and they are returned to the pool once the HTTP response in sent back to the client. However, if that's the case my thread ids should be different when the user accesses multiple pages on the site from same browser. Also printing the thread id may not be the right way to trace user journey then. Also I understand there may be difference in milli seconds or seconds between 1 concurrent user requests and it may not be exactly 1 users in system at one go but I can also simulate 10,000 concurrent request at same time even in milli seconds. Google analytics does show 10,000 concurrent users. In that case I assume it will leverage 10,000 concurrent threads. Let me know your thoughts around it. Regards, Saurabh Agrawal Manager Technology | Sapient -Original Message- From: Nick Williams [mailto:nicho...@nicholaswilliams.net] Sent: Wednesday, March 20, 2013 2:31 AM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote: Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. Remember that real users will have some time between each request, ranging from a few seconds to several minutes depending on the page and the user. A good load testing tool like JMeter, The Grinder, or NeoLoad (just examples) will mimic this behavior by putting varying pauses between requests. It sounds like whatever tool you are using is doing this--that's good! So, 8,000 simultaneous users might result in only 400-500 simultaneous requests (just a guess; depends on your application and its user base). In this case, you'd only see 400-500 threads used across the cluster. In your small-scale test, where each request from your browser got the same thread, that could just be pure luck. If you're the only person using the server, it's likely that the Thread pool is extremely small, thus the odds are high that the same thread would get picked over and over again. However, in a high-load environment, this will not be the case, and the number of threads will nearly always be less than the number of simultaneous users. I'm glad to see that you have a working cluster of some sorts and that you are performing load tests to evaluate its performance. This means you are on the right track, making some good decisions, and have obviously done a little reading. I suspect a little more reading of the Connector documentation will help you understand exactly how all of this works. Nick P.S. Please remember to bottom post. Don't top post. -Original Message- From: Nick Williams [mailto:nicho...@nicholaswilliams.net] Sent: Wednesday, March 20, 2013 1:44 AM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote: From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Nick Williams nicho...@nicholaswilliams.net wrote: On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote: Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. The correctness of that statement is highly dependent on the connector used and how it is configured. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. +1 If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. How many servers you need will depend on the app and how those users use it. It is perfectly possible for one server to serve 20,000 simultaneous users. 20,000 simultaneous requests is more likely to require multiple servers but again - it depends. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Mark Thomas wrote: Nick Williams nicho...@nicholaswilliams.net wrote: On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote: Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. The correctness of that statement is highly dependent on the connector used and how it is configured. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. +1 If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. How many servers you need will depend on the app and how those users use it. It is perfectly possible for one server to serve 20,000 simultaneous users. 20,000 simultaneous requests is more likely to require multiple servers but again - it depends. If I may add something : at some point, on each Tomcat server, there is only 1 listening socket for one port (the point being : you could have several, but you won't have 8000). So even if your clients really send 8000 TCP requests to the server at the same moment, they will be serialized at some point, and from the server's point of view, they come in one by one. /Then/ the server (provided it's TCP stack and all the rest is fast enough) can start distributing them over a pool of Threads. I know that this is an absurd example, but just to provide one extreme point of view : unless you have 8000 independent clients each firing one request at the sam time over 8000 cables connecting to 8000 ports, each one being served by one CPU core running its own TCP stack and its own Tomcat, you will never really have 8000 requests being processed really simultaneously. So the simultaneousness is a question of degree, not an absolute value. And the degree of simultaneousness depends on many factors, among which are (but none of them to be considered alone) : the network, the front-end, the client keepalive setting, the server keepalive setting, the number of connections that the client opens simultaneously, the choice of Connector(s), the usage or not of an Executor, the degree to which the requests (and responses) are really similar, the number of configured threads, the time needed to process each request, the CPU speed, the amount of memory etc. etc. And finally, the number of Threads that are started or are running simultaneously depends on all these factors, and the only one who has access to all these factors is you. If you fire 8000 simultaneous requests from clients, and you see only 200 Threads running to process them, then obviously there is a bottleneck somewhere. But it is not necessarily Tomcat which is not allocating all the Threads that it could be allocating. It could just be that Tomcat does not get more requests to allocate a Thread for, because they get slowed down (or discarded) somewhere else along the line. Or it could be that Tomcat just is not getting enough CPU cycles to be able to allocate more Threads and running them. I some configurations, and with some kinds of client requests patterns, a longer keepalive setting can make it so that one Tomcat thread stays allocated to one client connection, waiting for futher requests on that connection (which may never come). In some kinds of use cases, that is efficient, in others it is very inefficient. Like everything else, it depends.. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/19/13 8:55 PM, Saurabh Agrawal wrote: What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Tomcat will assign an arbitrary (i.e. random) thread from the pool. If I look at my server logs for both the requests from client, the thread id remains the same which gives me a feeling the state of HTTP is persistent and till the time browser is not closed, the thread won't be returned to the pool. Please can someone clarify the above behavior. What you are probably observing is a server under very low load (maybe just your two example requests) that just happens to process the two requests with the same thread. Don't worry: the threads go back into the pool right away (after the keepalive timeout, if you are using the BIO connector which is currently the default). Note: I need to configure the maxThreads setting of Tomcat to support 20,000 concurrent users in the system. The above clarification will help me pick the appropriate setting for maxThreads. 20,000 concurrent users is no problem. How often do they all make requests? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJvhAACgkQ9CaO5/Lv0PAkXwCfS5GE/d1yk+w1gN6xNvQ9LLfA y7sAoJjfeun+2Njsu47uBMrWhDl1xzIv =en0N -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Nick, On 3/19/13 10:31 PM, Nick Williams wrote: I'm glad to see that you have a working cluster of some sorts and that you are performing load tests to evaluate its performance. This means you are on the right track, making some good decisions, and have obviously done a little reading. +1000 It's refreshing to see someone actually performing load-tests against their configuration while tweaking their settings rather than just saying I need to support a million users, setting maxThreads=100, setting PermGen to 3GiB, changing the thread stack size, and micro-managing the garbage collector. Well done. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJvvcACgkQ9CaO5/Lv0PDthgCaA+XQogABsObUGwSUYCeOXadA BAEAnA7+BHlzmZkgKgAKwXZ0v+9pGrga =yQdQ -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
There will be a load of 20,000 users for a special festival for around 4-5 hours ONLY. Regards, Saurabh Agrawal Manager Technology | Sapient -Original Message- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Sent: Wednesday, March 20, 2013 1:48 PM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/19/13 8:55 PM, Saurabh Agrawal wrote: What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Tomcat will assign an arbitrary (i.e. random) thread from the pool. If I look at my server logs for both the requests from client, the thread id remains the same which gives me a feeling the state of HTTP is persistent and till the time browser is not closed, the thread won't be returned to the pool. Please can someone clarify the above behavior. What you are probably observing is a server under very low load (maybe just your two example requests) that just happens to process the two requests with the same thread. Don't worry: the threads go back into the pool right away (after the keepalive timeout, if you are using the BIO connector which is currently the default). Note: I need to configure the maxThreads setting of Tomcat to support 20,000 concurrent users in the system. The above clarification will help me pick the appropriate setting for maxThreads. 20,000 concurrent users is no problem. How often do they all make requests? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJvhAACgkQ9CaO5/Lv0PAkXwCfS5GE/d1yk+w1gN6xNvQ9LLfA y7sAoJjfeun+2Njsu47uBMrWhDl1xzIv =en0N -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/20/13 3:27 AM, Saurabh Agrawal wrote: The point I was trying to make when I mentioned that all requests from the same browser results in same thread is not a mere coincidence. The reason why it may not be sheer luck is because we have seen this on production which is accessed by more than 1000 simultaneous users on the site. Currently we trace the user journey by the thread id in the logs and my observation is all 1000 users (across the globe) gets the same thread id from the pool when they access subsequent URLS from the browser. Is the reverse true? Do you ever see thread A used by more than one client? Or is it a one-to-one mapping? It makes me feel as if the thread is not returned to the pool till the session times out or the user has closed the browser. Although the behavior is weird because tomcat won't should not have a persistent connection as it would mean improper use of the resources (thread). Tomcat is doing its job just fine, otherwise half the Java servers in the world would lock-up and stop working. What may be happening is that your browser is maintaining a keep-alive connection long enough (maybe 10 seconds?) that the next request can be sent over the same connection, and will get the same thread. So, if you have fairly quick-responding clients, you may see that each one is kind of monopolizing a thread for a while. Honestly, if that's the case, it's a *good* case as long as you aren't running out of threads. It would be interesting to see how you have your Connector configured, and how you have your load-balancer configured, too. Also I understand there may be difference in milli seconds or seconds between 1 concurrent user requests and it may not be exactly 1 users in system at one go but I can also simulate 10,000 concurrent request at same time even in milli seconds. Google analytics does show 10,000 concurrent users. In that case I assume it will leverage 10,000 concurrent threads. Users != requests. 10,000 concurrent users might generate 1000 simultaneous requests (that is, requests in-progress at a particular instant), unless they are really pounding on your cluster. What do your thread-pool usage metrics look like? Are you collecting that data (hint: do it)? Are you graphing them (hint: do that, too)? Doing time-based graphing can help you visualize your user load (measured in requests-per-second versus total average concurrent users, which is a nearly useless metric unless you have 100MiB sessions or something and you have to use that as part of your capacity plan) and you can even see things like hey, every day at 11:00 PDT we hit our maxThreads for 5 minutes. That lets you know that you need to give yourself a little more breathing room and then watch for another couple of days. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJwI8ACgkQ9CaO5/Lv0PD8QACdHdYnkzzZbcbmq4dmayaguTp/ eDcAn2dGIn63MJRtbllnJcaicIClmJDp =AfQh -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/20/13 6:52 AM, André Warnier wrote: If I may add something : at some point, on each Tomcat server, there is only 1 listening socket for one port (the point being : you could have several, but you won't have 8000). So even if your clients really send 8000 TCP requests to the server at the same moment, they will be serialized at some point, and from the server's point of view, they come in one by one. While that is true, the threaded-dispatch of those requests does mean that many requests can be processed simultaneously. Yes, they are de-queued serially, but that happens very quickly compared to the duration of the actual requests. So, you can have 200 simultaneous requests /in-process/... not that they all arrive at the exact same instant in time, but that they are all being served (i.e. have threads assigned and are actually doing work) simultaneously. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJwWYACgkQ9CaO5/Lv0PAtSQCaA1QrSvIOd4GPEKsA2+RezsxQ RW4AnRBtpcy4LBypi9ShRmproEcerrjd =inUq -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Saurabh,
On 3/19/13 9:46 PM, Saurabh Agrawal wrote:
Executor name=hybrisExecutor namePrefix=hybrisHTTP
maxThreads=${tomcat.maxthreads}
minSpareThreads=${tomcat.minsparethreads}
maxIdleTime=${tomcat.maxidletime}/ Connector
port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200
protocol=org.apache.coyote.ajp.AjpProtocol
executor=hybrisExecutor enableLookups=false acceptCount=100
connectionTimeout=2 URIEncoding=UTF-8
disableUploadTimeout=true /
Connector port=${tomcat.http.port} maxHttpHeaderSize=8192
maxThreads=${tomcat.maxthreads}
protocol=org.apache.coyote.http11.Http11Protocol
executor=hybrisExecutor enableLookups=false acceptCount=100
connectionTimeout=2 URIEncoding=UTF-8
disableUploadTimeout=true /
Note that your Executor has maxThreads=200 and your Connector
uses that Executor: your ${tomcat.maxthreads} is being ignored.
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEAREIAAYFAlFJwdIACgkQ9CaO5/Lv0PDOMwCfUxQ+hIaIIWlVfOok6b2+tawK
eSoAn2Ivy26EMiLbhnaPs6VH35ZECbgB
=6T32
-END PGP SIGNATURE-
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
From: Saurabh Agrawal [mailto:sagra...@sapient.com]
Subject: RE: Tomcat Behavior on Multiple HTTP requests from same browser
We have not set the keep alive explicitly in tomcat's server.xml.
It's on by default.
Connector port=${tomcat.ajp.port}
Connector port=${tomcat.http.port}
You have both HTTP and AJP Connectors defined; are the requests coming in
over both or just one of them? The discussion so far has been primarily
related to HTTP, not AJP.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you received
this in error, please contact the sender and delete the e-mail and its
attachments from all computers.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Christopher Schultz wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Saurabh,
On 3/19/13 9:46 PM, Saurabh Agrawal wrote:
Executor name=hybrisExecutor namePrefix=hybrisHTTP
maxThreads=${tomcat.maxthreads}
minSpareThreads=${tomcat.minsparethreads}
maxIdleTime=${tomcat.maxidletime}/ Connector
port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200
protocol=org.apache.coyote.ajp.AjpProtocol
executor=hybrisExecutor enableLookups=false acceptCount=100
connectionTimeout=2 URIEncoding=UTF-8
disableUploadTimeout=true /
Connector port=${tomcat.http.port} maxHttpHeaderSize=8192
maxThreads=${tomcat.maxthreads}
protocol=org.apache.coyote.http11.Http11Protocol
executor=hybrisExecutor enableLookups=false acceptCount=100
connectionTimeout=2 URIEncoding=UTF-8
disableUploadTimeout=true /
Note that your Executor has maxThreads=200 and your Connector
uses that Executor: your ${tomcat.maxthreads} is being ignored.
That, and the default keepalive setting, are probably the keys here.
And the observation of Chuck about the HTTP and AJP connectors. Over which Connector do
the test requests actually come in ?
And a question : is the simulation with the 1 clients really comparable to what you
expect in the reality ? For example, if the simulation requests one page per client, and
then does nothing else with that page; but the real clients would get a page, and then
immediately request the 50 thumbnail images referenced by that page, conditions would be
really different, and keepalive would have a very different effect.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
-Original Message-
From: André Warnier [mailto:a...@ice-sa.com]
Sent: Wednesday, March 20, 2013 3:27 PM
To: Tomcat Users List
Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
Christopher Schultz wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Saurabh,
On 3/19/13 9:46 PM, Saurabh Agrawal wrote:
Executor name=hybrisExecutor namePrefix=hybrisHTTP
maxThreads=${tomcat.maxthreads}
minSpareThreads=${tomcat.minsparethreads}
maxIdleTime=${tomcat.maxidletime}/ Connector
port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200
protocol=org.apache.coyote.ajp.AjpProtocol
executor=hybrisExecutor enableLookups=false acceptCount=100
connectionTimeout=2 URIEncoding=UTF-8
disableUploadTimeout=true /
Connector port=${tomcat.http.port} maxHttpHeaderSize=8192
maxThreads=${tomcat.maxthreads}
protocol=org.apache.coyote.http11.Http11Protocol
executor=hybrisExecutor enableLookups=false acceptCount=100
connectionTimeout=2 URIEncoding=UTF-8
disableUploadTimeout=true /
Note that your Executor has maxThreads=200 and your Connector
uses that Executor: your ${tomcat.maxthreads} is being ignored.
That, and the default keepalive setting, are probably the keys here.
And the observation of Chuck about the HTTP and AJP connectors. Over which
Connector do
the test requests actually come in ?
Saurabh - The actual front end requests come on AJP port. We are using AJP
protocol for communication between Apache and Tomcat. It helps in load
balancing across the application servers in cluster. There is a separate
internal application (not exposed on internet) used by CMS team which is using
HTTP connector. I hope that clarifies.
And a question : is the simulation with the 1 clients really comparable
to what you
expect in the reality ? For example, if the simulation requests one page per
client, and
then does nothing else with that page; but the real clients would get a page,
and then
immediately request the 50 thumbnail images referenced by that page, conditions
would be
really different, and keepalive would have a very different effect.
Saurabh - The way we have configured our user journeys are as follows:
User 1: Hits homepage, clicks football link on home page, makes a selection,
adds to cart and checkout. So this is one user journey which triggers multiple
requests. All our assets are served from L3 CDN. So the asset requests never
come to the application server. We have not set keep alive explicitly anywhere
in tomcat.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Saurabh Agrawal wrote:
-Original Message-
From: André Warnier [mailto:a...@ice-sa.com]
Sent: Wednesday, March 20, 2013 3:27 PM
To: Tomcat Users List
Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
Christopher Schultz wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Saurabh,
On 3/19/13 9:46 PM, Saurabh Agrawal wrote:
Executor name=hybrisExecutor namePrefix=hybrisHTTP
maxThreads=${tomcat.maxthreads}
minSpareThreads=${tomcat.minsparethreads}
maxIdleTime=${tomcat.maxidletime}/
Connector
port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200
protocol=org.apache.coyote.ajp.AjpProtocol
executor=hybrisExecutor enableLookups=false acceptCount=100
connectionTimeout=2 URIEncoding=UTF-8
disableUploadTimeout=true /
Connector port=${tomcat.http.port} maxHttpHeaderSize=8192
maxThreads=${tomcat.maxthreads}
protocol=org.apache.coyote.http11.Http11Protocol
executor=hybrisExecutor enableLookups=false acceptCount=100
connectionTimeout=2 URIEncoding=UTF-8
disableUploadTimeout=true /
Note that your Executor has maxThreads=200 and your Connector
uses that Executor: your ${tomcat.maxthreads} is being ignored.
That, and the default keepalive setting, are probably the keys here.
And the observation of Chuck about the HTTP and AJP connectors. Over which Connector do
the test requests actually come in ?
Saurabh - The actual front end requests come on AJP port. We are using AJP
protocol for communication between Apache and Tomcat.
Right. So then I suppose that Christopher's note is not applicable. Probablya he misread,
because the way in which you pasted the configuration in the email makes it difficult to
read, after a couple of cut-and-paste.
As far as I can tell, the AJP connector refers to the Executor, and the Executor specifies
maxThreads=${tomcat.maxthreads}.
The main point of Christopher was that you specify a maxThreads parameter in both of
your Connectors, but because they both use the Executor, this parameter is being ignored
in the Connector, and it is only the maxThreads in the Executor that counts.
It helps in load balancing across the application servers in cluster. There is a
separate internal application (not exposed on internet) used by CMS team which is using
HTTP connector. I hope that clarifies.
Yes.
And a question : is the simulation with the 1 clients really comparable to what you
expect in the reality ? For example, if the simulation requests one page per client, and
then does nothing else with that page; but the real clients would get a page, and then
immediately request the 50 thumbnail images referenced by that page, conditions would be
really different, and keepalive would have a very different effect.
Saurabh - The way we have configured our user journeys are as follows:
User 1: Hits homepage, clicks football link on home page, makes a selection, adds to cart and checkout. So this is one user journey which triggers multiple requests.
Hi. Your usage of user journey is a bit obscure (to me at least) in the context of
analysing a matter of tomcat request/response performance.
I kind of understand what you mean, but it does not really provide the answer to the
questions :
- is this what you are using in your tests ?
- are you doing this same series of requests for each of your 1 test
clients ?
- does this represent (more or less) what you are expecting later in
production ?
The point here was to avoid a case where you would be optimising the parameters in
function of a benchmark test, and then find out later that your production case is totally
different, and your optimal benchmark settings are totally inappropriate for the
production case.
All our assets are served from L3 CDN. So the asset requests never come to the
application server.
That, I do not understand. I do not understand what you mean by assets here, and I do
not understand L3 CDN. So I cannot tell of this is relevant or not to the problem.
Have pity for the people trying to help you here, who only know Tomcat and
HTTP/AJP.
Try to use vocabulary that we understand, and you may get better help.
We have not set keep alive explicitly anywhere in tomcat.
What Chuck was telling you in an earlier message, is that even if you do not set it
explicitly, it is set to some default non-zero value by Tomcat.
Look at this page in the on-line documentation :
http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html#Standard_Implementations
So, by default (unless you explicitly define it),
keepAliveTimeout :
The number of milliseconds this Connector will wait for another AJP request before closing
the connection. The default value is to use the value that has been set for the
connectionTimeout attribute.
and
connectionTimeout
The number of milliseconds this Connector will wait, after accepting a connection, for the
request URI line to be presented. The default value for AJP protocol
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/20/13 2:25 PM, André Warnier wrote: Saurabh Agrawal wrote: All our assets are served from L3 CDN. So the asset requests never come to the application server. That, I do not understand. I do not understand what you mean by assets here, and I do not understand L3 CDN. So I cannot tell of this is relevant or not to the problem. CDN = Content Delivery Network. I'm not sure what L3 (probably Level 3, a data center operations company) is, but a CDN is basically a whole bunch of copies of your files geographically distributed such that requesting a file always gets the bits that are closest to you. Kind of a cool thing. ;) The bottom line is that Saurabh expects only dynamic requests to come to Tomcat, so keepalives should be much less useful than if Tomcat were to be serving everything. Imagine httpd out front serving all static content and forwarding dynamic stuff to Tomcat via AJP -- that's almost exactly what's going on here, except that the static stuff is being served very efficiently from a network-topology perspective. Since AJP is in use, keepalive is almost entirely a red herring as typical AJP connections are permanently-connected to the web server. So, by default, the keepAliveTimeout [for AJP] is set to infinite. [snip] And if the client keeps the connection open, but does not send any additional request on that connection, the Thread will wait theoretically forever (because that is what the documentation says about the default value of these parameters). No, the defaults are different for non-AJP connections. Tomcat's default default is 60 seconds but the stock server.xml configures it to 20 seconds. Now your case is a bit different, because - you are not using the HTTP BIO connector (you use AJP) I think you've gotten yourself confused, here, unfortunately. You can use AJP with BIO, NIO, or APR (maybe you mixed-up AJP and APR between your eyes and your brain... the two are honestly too close to each other and it is very easy to do that). He is in fact using the BIO connector because he has specified protocol=org.apache.coyote.ajp.AjpProtocol. - in front of your Tomcat, is an Apache httpd server. This server has its own keep-alive settings which apply to the connection of the client with Apache httpd. And these keep-alive settings are a bit different from the Tomcat ones (for example, there is a keep-alive timeout, but also a MaxKeepAliveRequests) +1 - between Apache httpd and Tomcat, there is the mod_jk module in Apache, and that module uses its own timeouts (as set in workers.properties), and in addition it uses itself a pool of connections to Tomcat, and this pool of connections has its own rules for keeping alive a connection between Apache and Tomcat. But the basic principles above apply, and may explain why you are seeing what appears to be one Thread dedicated to one client, forever. I think there might be a problem with the instrumentation, or just coincidences at a fairly implausible level. The trust of the matter is that Tomcat does not allocate a thread permanently to a remote client until ... whenever the client disconnects (whatever that means, as HTTP is a connection-less protocol). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRSlo8AAoJEBzwKT+lPKRYodAP/Rz73chvzhyYPNWpePlvPaBm x7bFxFCNg7hbHV7X1skgfG4CYbR9AsjFqcgkXovVSBcIZjKoNT/yCqwL20SMdNIo glkx1Gz6gBfM8Ad68VL4xr7nGtwC1WraN2tCszc8dNrBJOjmQwp5tX09OWtpsxqe DNQVfhGk8wtHmPWjXgK8Kr0lZDjfz540bp4OSAwohjixsCTr3C1PqVEiiZyYWazC B/5tEkeJ7YtPQJqmJhgx1uR+CmU3ty5iQpLmr4K9uKTLSTjs3HGFV0nkCKQIXmQs vzjt0eWr8VzbbZnY4QBfBIntgjQiDIoK9Mi+Q3uilYbjCJuFw4jJGICAjaZjlYxW eW29Ag/WGthB6shOwStiLS+/KUenzXY9aNzHQ1sCeB9Pdsy2eI+Yg15TomwcfxcV 2LjtHZO13QQeGcHpr5vBId8x/B9x5W1bzs8gzUALPQhdsYqGixohu4Ad4nqmeBI0 VZ02jiEDQqr4zT6oJwDVPpm8FSS2wBZKURIp80iI4B2+bS1VPcKzY/J/aKCQRQCw KP4VDtNiAOXD4a+8sHuZrOh8Q/splNnbhf2G687PryjfxQuk+HQPY/XfFvY1CXzD ZZD1yVjkSaRfXDmDCfFxfD+a3lOmGhAa8NNR4dtJJo1w1lhrYXvx1ujdS5V3FgO5 xdKlzkg4u30lRrAQq6IO =yLs6 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote: From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.) If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering. Nick - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Hi chuck,
We have not set the keep alive explicitly in tomcat's server.xml. Howeverm
when I print the thread id for all requests from same browser, it prints the
same thread it. It gives me a feeling that all HTTP requests are holding worker
thread of tomcat if they originate from same browser.
My tomcat configuration is as follows:
Server port=${tomcat.internalserver.port} shutdown=SHUTDOWN
Listener className=org.apache.catalina.core.JasperListener /
Listener className=org.apache.catalina.mbeans.ServerLifecycleListener /
Listener
className=org.apache.catalina.mbeans.GlobalResourcesLifecycleListener /
Service name=Catalina
Executor name=hybrisExecutor
namePrefix=hybrisHTTP
maxThreads=${tomcat.maxthreads}
minSpareThreads=${tomcat.minsparethreads}
maxIdleTime=${tomcat.maxidletime}/
Connector port=${tomcat.ajp.port}
maxHttpHeaderSize=8192
maxThreads=200
protocol=org.apache.coyote.ajp.AjpProtocol
executor=hybrisExecutor
enableLookups=false
acceptCount=100
connectionTimeout=2
URIEncoding=UTF-8
disableUploadTimeout=true /
Connector port=${tomcat.http.port}
maxHttpHeaderSize=8192
maxThreads=${tomcat.maxthreads}
protocol=org.apache.coyote.http11.Http11Protocol
executor=hybrisExecutor
enableLookups=false
acceptCount=100
connectionTimeout=2
URIEncoding=UTF-8
disableUploadTimeout=true /
Engine name=Catalina defaultHost=localhost
jvmRoute=${tomcat.ajp.worker}
Valve
className=org.apache.catalina.valves.FastCommonAccessLogValve
directory=${HYBRIS_LOG_DIR}/tomcat
prefix=access.
suffix=.log
pattern=%{X-Forwarded-For}i %l %u %t %T %r %s
%b %{User-Agent}i %{Referer}i %{JSESSIONID}c
/
Host name=localhost
appBase=webapps
unpackWARs=true
autoDeploy=false
xmlValidation=false
xmlNamespaceAware=false
${tomcat.webapps.60}
/Host
/Engine
/Service
/Server
Regards,
Saurabh Agrawal
Manager Technology | Sapient
-Original Message-
From: Caldarale, Charles R [mailto:chuck.caldar...@unisys.com]
Sent: Wednesday, March 20, 2013 1:37 AM
To: Tomcat Users List
Subject: RE: Tomcat Behavior on Multiple HTTP requests from same browser
From: Saurabh Agrawal [mailto:sagra...@sapient.com]
Subject: Tomcat Behavior on Multiple HTTP requests from same browser
Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL,
tomcat will assign one of the worker threads (say Thread 1) from the pool
to the HTTP request which will be processed and then response will be sent
to the client. Now if I hit a link on homepage which is for login, a separate
HTTP request will be initiated from the same browser.
What I want to understand is if the Tomcat will keep Thread 1 as persistent
thread to server the second request (for login) from the same browser or will
it assign a separate thread from the pool ?
Normally, the first available thread from the pool is used for each request.
However, if you are using the BIO Connector with keep-alives enable *and* the
browser is using keep-alives, the same thread as long as the HTTP connection is
active.
See the HTTP Connector doc, especially the Connector Comparison at the end.
http://tomcat.apache.org/tomcat-7.0-doc/config/http.html
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you received
this in error, please contact the sender and delete the e-mail and its
attachments from all computers.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Regards, Saurabh Agrawal Manager Technology | Sapient -Original Message- From: Nick Williams [mailto:nicho...@nicholaswilliams.net] Sent: Wednesday, March 20, 2013 1:44 AM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote: From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.) If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering. Nick - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote: Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. Remember that real users will have some time between each request, ranging from a few seconds to several minutes depending on the page and the user. A good load testing tool like JMeter, The Grinder, or NeoLoad (just examples) will mimic this behavior by putting varying pauses between requests. It sounds like whatever tool you are using is doing this--that's good! So, 8,000 simultaneous users might result in only 400-500 simultaneous requests (just a guess; depends on your application and its user base). In this case, you'd only see 400-500 threads used across the cluster. In your small-scale test, where each request from your browser got the same thread, that could just be pure luck. If you're the only person using the server, it's likely that the Thread pool is extremely small, thus the odds are high that the same thread would get picked over and over again. However, in a high-load environment, this will not be the case, and the number of threads will nearly always be less than the number of simultaneous users. I'm glad to see that you have a working cluster of some sorts and that you are performing load tests to evaluate its performance. This means you are on the right track, making some good decisions, and have obviously done a little reading. I suspect a little more reading of the Connector documentation will help you understand exactly how all of this works. Nick P.S. Please remember to bottom post. Don't top post. -Original Message- From: Nick Williams [mailto:nicho...@nicholaswilliams.net] Sent: Wednesday, March 20, 2013 1:44 AM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote: From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.) If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering. Nick - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h
