RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
-Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: Thursday, March 21, 2013 8:51 AM To: Tomcat Users List Subject: Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser Christopher Schultz wrote: HTTP connections for long periods of time, but that's really abuse of the protocol IMO. You can send bowling balls via carrier pigeon, but there are better ways to send bowling balls. You would need a fairly large, and well-disciplined team of pigeons to do that though. I don't think that this was a good metaphor, You should have chosen a bigger bird and/or a smaller load. Eagles and tennis balls maybe ? Or swallows and coconuts. Jeff (sorry, couldn't resist.) - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
On Fri, Mar 22, 2013 at 10:04 AM, Jeffrey Janner jeffrey.jan...@polydyne.com wrote: -Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: Thursday, March 21, 2013 8:51 AM To: Tomcat Users List Subject: Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser Christopher Schultz wrote: HTTP connections for long periods of time, but that's really abuse of the protocol IMO. You can send bowling balls via carrier pigeon, but there are better ways to send bowling balls. You would need a fairly large, and well-disciplined team of pigeons to do that though. I don't think that this was a good metaphor, You should have chosen a bigger bird and/or a smaller load. Eagles and tennis balls maybe ? Or swallows and coconuts. Jeff (sorry, couldn't resist.) Wow, you all are funny! LOL
RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
From: Jeffrey Janner [mailto:jeffrey.jan...@polydyne.com] Subject: RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser You would need a fairly large, and well-disciplined team of pigeons to do that though. I don't think that this was a good metaphor, You should have chosen a bigger bird and/or a smaller load. Eagles and tennis balls maybe ? Or swallows and coconuts. Someone had to bring that up. African or European? I think we can remove the not from the subject line now... - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
Caldarale, Charles R wrote: From: Jeffrey Janner [mailto:jeffrey.jan...@polydyne.com] Subject: RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser You would need a fairly large, and well-disciplined team of pigeons to do that though. I don't think that this was a good metaphor, You should have chosen a bigger bird and/or a smaller load. Eagles and tennis balls maybe ? Or swallows and coconuts. Someone had to bring that up. African or European? I think we can remove the not from the subject line now... Done. This all reminds me of this (locally) well-known Belgian bird : the oye-oye-oye bird. For those who don't know the species : It is a very strong, sturdy bird. Rather bad-tempered too, you shouldn't mess with it. It is a bit the bird-equivalent of the Belgian horse really. It has a big round head, with a strong beak, say oh a good 5 inches wide and 7 inches long. A really strong beak, he can crush nuts or bones with it. Then it has very strong wings too, strong enough to lift a small sheep for instance (or a bowling ball for that matter). And tough feathers, you can make brooms with them. And also rather short, but very strong sturdy legs; like 2 inches thick and 2 inches long. And then its balls, no kidding, they are the size of coconuts. And each time it lands, it goes oye oye oye. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
On 3/22/2013 9:35 AM, André Warnier wrote: Caldarale, Charles R wrote: From: Jeffrey Janner [mailto:jeffrey.jan...@polydyne.com] Subject: RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser You would need a fairly large, and well-disciplined team of pigeons to do that though. I don't think that this was a good metaphor, You should have chosen a bigger bird and/or a smaller load. Eagles and tennis balls maybe ? Or swallows and coconuts. Someone had to bring that up. African or European? I think we can remove the not from the subject line now... Done. This all reminds me of this (locally) well-known Belgian bird : the oye-oye-oye bird. For those who don't know the species : It is a very strong, sturdy bird. Rather bad-tempered too, you shouldn't mess with it. It is a bit the bird-equivalent of the Belgian horse really. It has a big round head, with a strong beak, say oh a good 5 inches wide and 7 inches long. A really strong beak, he can crush nuts or bones with it. Then it has very strong wings too, strong enough to lift a small sheep for instance (or a bowling ball for that matter). And tough feathers, you can make brooms with them. And also rather short, but very strong sturdy legs; like 2 inches thick and 2 inches long. And then its balls, no kidding, they are the size of coconuts. And each time it lands, it goes oye oye oye. Sounds very much like the African oomy-goomie bird. There's a rugby song about that - something about going off to see a wild west show . . . elephants and kangaroos. Looks like at least one subspecies migrated north. /mde/ - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chuck, On 3/22/13 10:25 AM, Caldarale, Charles R wrote: From: Jeffrey Janner [mailto:jeffrey.jan...@polydyne.com] Subject: RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser You would need a fairly large, and well-disciplined team of pigeons to do that though. I don't think that this was a good metaphor, You should have chosen a bigger bird and/or a smaller load. Eagles and tennis balls maybe ? Or swallows and coconuts. Someone had to bring that up. African or European? I think we can remove the not from the subject line now... I think someone gets thrown off a bridge into an abyss if we keep going. The Spring Generation probably doesn't have any idea what we're tottering on about. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRTNxAAAoJEBzwKT+lPKRY3koQAIkmmGnn4SA1u1VeLkU8ZAre 8YWDktOOvf6J0xJQl8iRajoV2ZZT0EjoHSO9ODopS7BiYRcbuCfLROXaIYLSD/PR nLiporwrtehxW7yTHxtpM3P599E/Gr2vvLhw3/9Wl7BcM1exnQrsDkmTa3bLEIxT XvUaisgwhDquCyfFyPxV5fy46Fn128fyHa6E4ZUaeRGYYH4hcwkAJRc7lM1TaX+y BOmNr0jR60Z2KfSMEWeXR2J4FpYoFUrkS3KDRWi7RruZp28BR7MPLX/49inioy3K a+K7lgus9s6p0QySBREDEpi2oCXEO31m2gRYQ4O7iXOoKaKcOkMS7Vx2hKkbO2eq unJ/pcjkTWYXVx5oahbi9VEOSf3PiXGV3xYA/JFFbG/xYkKeVReofTmCoeHIfVTh H9GKbzGR0jQAWUkZpnuMumg+Dbra6/J3+Nau/ItDy5kQ1e1xIbLyYswEbzTjxnHa 0wNkQkfID/IfOpz9mnM5wkowE8cDrqVFO3YkWDzYo5GsIliGT8o6NapTf7y8ll1D rVyVhPfMUJWMjq6QlJRcPJE3EuluOtRP+73UejZmqssL/V2RJCH+5f2sbBipB1gW X+VwBRVuD0qCGgniDIrXYjLZqTKsp+Y/9d38H1SgZgdTdIOPtqMVJ8IwU1uYopTa L2rZsjHHYmB7+eYyzvMf =Ttie -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/22/13 12:35 PM, André Warnier wrote: Caldarale, Charles R wrote: From: Jeffrey Janner [mailto:jeffrey.jan...@polydyne.com] Subject: RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser You would need a fairly large, and well-disciplined team of pigeons to do that though. I don't think that this was a good metaphor, You should have chosen a bigger bird and/or a smaller load. Eagles and tennis balls maybe ? Or swallows and coconuts. Someone had to bring that up. African or European? I think we can remove the not from the subject line now... Done. This all reminds me of this (locally) well-known Belgian bird : the oye-oye-oye bird. For those who don't know the species : It is a very strong, sturdy bird. Rather bad-tempered too, you shouldn't mess with it. It is a bit the bird-equivalent of the Belgian horse really. It has a big round head, with a strong beak, say oh a good 5 inches wide and 7 inches long. A really strong beak, he can crush nuts or bones with it. Then it has very strong wings too, strong enough to lift a small sheep for instance (or a bowling ball for that matter). And tough feathers, you can make brooms with them. And also rather short, but very strong sturdy legs; like 2 inches thick and 2 inches long. And then its balls, no kidding, they are the size of coconuts. And each time it lands, it goes oye oye oye. That sounds absolutely terrifying. A 7-inch-long beak that's 5-inches wide? That's like an oil tunnel for a car. I don't even want to hear about the Belgian Horse... - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRTNzSAAoJEBzwKT+lPKRY79EP/0xwu/G4cTHvKoJldQRdFYUX cT9vBgr/xrdkNuV8gO/+cwBhiAGVAw2LOtkT6aF85zdv0Da6PALk6DOrvx4PLcnz KVAGG2mp6KHngISB09ti1I/QycNfKFQUkA3ogHFB8N3LDxQoTc1ZfbcWm1+UcRYu BMvXrzAcAapx0ZKYP4ZId7Z9vtIyB5mpGmCXu88x0bJs+D/shzLTPGiE7tYzZfpi R/VJhE4TezajaRFvnCNLLooeHAGEx3qJ/FNLaynr1QB8X7QRhOomulr5tR4n5+Za ELoGLkeIfcYCrXiSDANJAEGrLUf5/4ub/yZqPhXuL2dgSI5FvI4+Np7PfscfJ0xi BtTZ8JYB6I0FQfJqRHnB0krpvnBhncmnyqdk+OKuxoSqYeHabaX8PlqGMQu6mlpR ay71I8qrHYsBvOjTF+rexstcNEkzWeEON06Q+AzlWElp/NAyI+OT6lpHXAew7A5e wX8RcVU7/5vC14RnpUVwaA4OSvqQpMH29hk9n0s5rEjR8zro+fm5ZFUCHxsg6j51 Xmgq0AwENY2hm6EVducDs2aFUK1n1xoz0z09Vc9Jb9dIiXuz0DgPW/Ls7YcoiRce kh04Jn+M8oQl1nZsteb3egXasExm/S8EoCg/HrFpenLje30z+riW7mBPRpwx4v5p WenLW/6m51hgZNpuccxx =QmXY -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: [totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same browser Or swallows and coconuts. Someone had to bring that up. African or European? I think we can remove the not from the subject line now... I think someone gets thrown off a bridge into an abyss if we keep going. The Spring Generation probably doesn't have any idea what we're tottering on about. The rules say the Spring Generation goes over if they can't answer the questions... Agh. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
Christopher, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/20/13 2:25 PM, André Warnier wrote: Saurabh Agrawal wrote: All our assets are served from L3 CDN. So the asset requests never come to the application server. That, I do not understand. I do not understand what you mean by assets here, and I do not understand L3 CDN. So I cannot tell of this is relevant or not to the problem. CDN = Content Delivery Network. I'm not sure what L3 (probably Level 3, a data center operations company) is, but a CDN is basically a whole bunch of copies of your files geographically distributed such that requesting a file always gets the bits that are closest to you. Kind of a cool thing. ;) Thank your for the above Rosetta Stone. This computer business os so full of acronyms of all kinds - some of them with multiple interpretations - that it is sometimes difficult to grasp the meaning. And I really don't feel like having to use Wikipedia every 3 words of a post on the list. Not when static content is delivered by the Apache front-end would have done it. The bottom line is that Saurabh expects only dynamic requests to come to Tomcat, so keepalives should be much less useful than if Tomcat were to be serving everything. Imagine httpd out front serving all static content and forwarding dynamic stuff to Tomcat via AJP -- that's almost exactly what's going on here, except that the static stuff is being served very efficiently from a network-topology perspective. Since AJP is in use, keepalive is almost entirely a red herring as typical AJP connections are permanently-connected to the web server. Well, I would say so indeed forthe case of a html page wit embedded images e.g. Butit may not be so in the benchmark case that Saurabh explained, with each of the 1 clients making multiple requests to non-static content, presumably served by Tomcat. A human user may have delays, that his testcase might not have. So, by default, the keepAliveTimeout [for AJP] is set to infinite. [snip] And if the client keeps the connection open, but does not send any additional request on that connection, the Thread will wait theoretically forever (because that is what the documentation says about the default value of these parameters). No, the defaults are different for non-AJP connections. Tomcat's default default is 60 seconds but the stock server.xml configures it to 20 seconds. Right. But my explanation was meant as an example only, to point out the interlinked effects of the various attributes and timeouts. And 20 seconds is still an incredibly long time, in the context of 1 simultaneous clients sending multiple requests each. Now your case is a bit different, because - you are not using the HTTP BIO connector (you use AJP) I think you've gotten yourself confused, here, unfortunately. You can use AJP with BIO, NIO, or APR (maybe you mixed-up AJP and APR between your eyes and your brain... the two are honestly too close to each other and it is very easy to do that). Yes, I was confused and thank you for making me see the light. I though that the AJP Connector was a beast all of it's own, and did not use these different underlying layers. It is clear fom the documentation that it does though, I don't know how I could have overlooked that for so long. He is in fact using the BIO connector because he has specified protocol=org.apache.coyote.ajp.AjpProtocol. - in front of your Tomcat, is an Apache httpd server. This server has its own keep-alive settings which apply to the connection of the client with Apache httpd. And these keep-alive settings are a bit different from the Tomcat ones (for example, there is a keep-alive timeout, but also a MaxKeepAliveRequests) +1 - between Apache httpd and Tomcat, there is the mod_jk module in Apache, and that module uses its own timeouts (as set in workers.properties), and in addition it uses itself a pool of connections to Tomcat, and this pool of connections has its own rules for keeping alive a connection between Apache and Tomcat. But the basic principles above apply, and may explain why you are seeing what appears to be one Thread dedicated to one client, forever. I think there might be a problem with the instrumentation, or just coincidences at a fairly implausible level. The trust of the matter is that Tomcat does not allocate a thread permanently to a remote client until ... whenever the client disconnects (whatever that means, as HTTP is a connection-less protocol). (See the nitpick (*) below) Possible, but see above again with the httpd/tomcat connections managed by the mod_jk module. It does have and manage its own pool of connections, with each connection potentially staying alive for a time much longer than any individual client request. I do not deny that. But what I am not so sure of (and maybe Rainer could comment here) is this scenario : - a client, via httpd+mod_jk,
Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/21/13 7:15 AM, André Warnier wrote: Christopher Schultz wrote: I think there might be a problem with the instrumentation, or just coincidences at a fairly implausible level. The trust of the matter is that Tomcat does not allocate a thread permanently to a remote client until ... whenever the client disconnects (whatever that means, as HTTP is a connection-less protocol). (See the nitpick (*) below) Possible, but see above again with the httpd/tomcat connections managed by the mod_jk module. It does have and manage its own pool of connections, with each connection potentially staying alive for a time much longer than any individual client request. I do not deny that. Right, but the AJP connections are managed in a connection pool. I haven't really checked-into this, but I suspect that two requests coming from the same keepalive connection have no guarantee of being sent across the same AJP connection to Tomcat, and thus no guarantee that they will be served by the same JVM thread. mod_jk is aware that the client/httpd connection is keepalive, and it does not have any way to know that this client is not going to send another request to Tomcat. So what does mod_jk really do ? Does it relinquish the one connection that he had to Tomcat back to the pool immediately after the first response has been served ? or does it keep its handle on that pool connection until the client/httpd timeout has expired ? It would be a mistake for mod_jk to retain control of the AJP connection for that keepalive request -- there's no guarantee that the /next/ request across that connection would even be routed through mod_jk: it might be for some other resource that another module handles. There is also kind of a weird question here : what is really the purpose of the keepAliveTimeout attribute on the Tomcat AJP Connector ? +1 (*) nitpick about HTTP being connection-less : that may be true in the sense that each request+response is supposedly independent from any other request+response. But HTTP 1.1 explicitly introduces persistent TCP connections. Yes, and HTTP sessions are standard fare these days, too. But the fact is that HTTP itself is connection-less. We as engineers can make it feel like it's not and do stupid things like put JDBC connections into HttpSession objects and try to tie everything together to make the user feel like they have a permanent connection. We can even hold-open HTTP connections for long periods of time, but that's really abuse of the protocol IMO. You can send bowling balls via carrier pigeon, but there are better ways to send bowling balls. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRSwfNAAoJEBzwKT+lPKRYAMgQAKyONnliLXad6HDwO9raxN+d evcEl8zAeYr6vbevZJ/KJK/FALoFVOmHdMDj+twGEWM+zrIOkHel2y9LHKzxR+St 6Fz1466yDeHM45D/PBcyMow2WaOSR9a6Ewj8uDprJLuVjT20qRlaiw0pQjvfB5M2 rPfX0rEP6NPMQNaTTdaTiq56LP4j/kNMiIhNZyZrq3Gu+9hP/LGEZgKW4j9PDPRF wvNUnWrHwhl4cUJwtRF1CuyynmJmnsrglQWpLgj1cYvBnzHp/19I3CKUevut5JUj NtcCmZ+u/is9zsJbWn6g1yRpyVFNForyV2XF2UFMDm/4UO+Ofyq1lVsGtvkMu3b2 2PQ7vMPqMM34JBySI3ZWVMFxD3GZUMm+Bqc4H5iKIzcGxRg0MgQn5z6DHniuIOmw lUxsjiwHiJ8xF/W3cdN1cxzPPzG92MOp4FWpsnMF+XcAN8ctGr5MuRJzDJKfct1o VcQojNqhmDyBHlJd3188aAz94KUXIGyXkmsLNdvnYhSZLZWxjFwBNxYm/4RzmHeA Dm/Heoe+qpxsk868YGKvJcIAk/1Rdxje8ZEJv44YRNXyCqfDkq0t9HUCduzyNBJF 4H/RVCSSS6OEXvdXUMywS2JJElcss560fSZ1ZF45YSW6NiLMIyl5PjFR1mb0Kfsf YYwN2L9egDE8ZDibeON2 =Bxsy -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/21/13 7:15 AM, André Warnier wrote: Christopher Schultz wrote: I think there might be a problem with the instrumentation, or just coincidences at a fairly implausible level. The trust of the matter is that Tomcat does not allocate a thread permanently to a remote client until ... whenever the client disconnects (whatever that means, as HTTP is a connection-less protocol). (See the nitpick (*) below) Possible, but see above again with the httpd/tomcat connections managed by the mod_jk module. It does have and manage its own pool of connections, with each connection potentially staying alive for a time much longer than any individual client request. I do not deny that. Right, but the AJP connections are managed in a connection pool. I haven't really checked-into this, but I suspect that two requests coming from the same keepalive connection have no guarantee of being sent across the same AJP connection to Tomcat, and thus no guarantee that they will be served by the same JVM thread. mod_jk is aware that the client/httpd connection is keepalive, and it does not have any way to know that this client is not going to send another request to Tomcat. So what does mod_jk really do ? Does it relinquish the one connection that he had to Tomcat back to the pool immediately after the first response has been served ? or does it keep its handle on that pool connection until the client/httpd timeout has expired ? It would be a mistake for mod_jk to retain control of the AJP connection for that keepalive request -- there's no guarantee that the /next/ request across that connection would even be routed through mod_jk: it might be for some other resource that another module handles. On the other hand, if there were 10 successive requests for Tomcat from the same client on the same connection, then it might be argued that it would be counterproductive to return the connection to the pool each time, just to go obtain another one right after, and this 10 times in a row. May be there should be an adaptative or predictive algorithm here : if this client in the recent past has always sent several requests in short succession, then maybe I'll keep this connection for now, just in case he does it again. I can already hear Rainer saying patches are always welcome. ;-) But the real point is : does mod_jk keep the connection, or does it return it to the pool at the end of each response ? Barring Rainer reading this, I guess that only looking at the code would tell. Note that Apache httpd already maintains the client/httpd connection, and keeps a count of how many requests have been received over this connection. It has to, for MaxKeepAliveRequests. So it would not be too much of a complication for mod_jk to keep its own count, of how many requests forwarded to Tomcat have been received over this same connection. That would already be a good predictor of whether the same is likely in the future. a = time for which this client connection has been alive b = number of requests forwarded to tomcat over this connection c = a / b = average time between 2 requests forwarded to tomcat if c is lower than the overhead for obtaining and returning a connection from the pool, then keep the connection. It would be self-adaptative, because if the client slows down its request rate, then c would become larger, and the connection would be returned to the pool; and vice-versa. There is also kind of a weird question here : what is really the purpose of the keepAliveTimeout attribute on the Tomcat AJP Connector ? +1 (*) nitpick about HTTP being connection-less : that may be true in the sense that each request+response is supposedly independent from any other request+response. But HTTP 1.1 explicitly introduces persistent TCP connections. Yes, and HTTP sessions are standard fare these days, too. But the fact is that HTTP itself is connection-less. We as engineers can make it feel like it's not and do stupid things like put JDBC connections into HttpSession objects and try to tie everything together to make the user feel like they have a permanent connection. We can even hold-open HTTP connections for long periods of time, but that's really abuse of the protocol IMO. You can send bowling balls via carrier pigeon, but there are better ways to send bowling balls. You would need a fairly large, and well-disciplined team of pigeons to do that though. I don't think that this was a good metaphor, You should have chosen a bigger bird and/or a smaller load. Eagles and tennis balls maybe ? I should also probably remind you of RFC 1149 : http://en.wikipedia.org/wiki/IP_over_Avian_Carriers - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Hi Nick, Thanks for your valuable feedback. The point I was trying to make when I mentioned that all requests from the same browser results in same thread is not a mere coincidence. The reason why it may not be sheer luck is because we have seen this on production which is accessed by more than 1000 simultaneous users on the site. Currently we trace the user journey by the thread id in the logs and my observation is all 1000 users (across the globe) gets the same thread id from the pool when they access subsequent URLS from the browser. It makes me feel as if the thread is not returned to the pool till the session times out or the user has closed the browser. Although the behavior is weird because tomcat won't should not have a persistent connection as it would mean improper use of the resources (thread). I read the connector documentation and my gathering is threads are allocated per request and they are returned to the pool once the HTTP response in sent back to the client. However, if that's the case my thread ids should be different when the user accesses multiple pages on the site from same browser. Also printing the thread id may not be the right way to trace user journey then. Also I understand there may be difference in milli seconds or seconds between 1 concurrent user requests and it may not be exactly 1 users in system at one go but I can also simulate 10,000 concurrent request at same time even in milli seconds. Google analytics does show 10,000 concurrent users. In that case I assume it will leverage 10,000 concurrent threads. Let me know your thoughts around it. Regards, Saurabh Agrawal Manager Technology | Sapient -Original Message- From: Nick Williams [mailto:nicho...@nicholaswilliams.net] Sent: Wednesday, March 20, 2013 2:31 AM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote: Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. Remember that real users will have some time between each request, ranging from a few seconds to several minutes depending on the page and the user. A good load testing tool like JMeter, The Grinder, or NeoLoad (just examples) will mimic this behavior by putting varying pauses between requests. It sounds like whatever tool you are using is doing this--that's good! So, 8,000 simultaneous users might result in only 400-500 simultaneous requests (just a guess; depends on your application and its user base). In this case, you'd only see 400-500 threads used across the cluster. In your small-scale test, where each request from your browser got the same thread, that could just be pure luck. If you're the only person using the server, it's likely that the Thread pool is extremely small, thus the odds are high that the same thread would get picked over and over again. However, in a high-load environment, this will not be the case, and the number of threads will nearly always be less than the number of simultaneous users. I'm glad to see that you have a working cluster of some sorts and that you are performing load tests to evaluate its performance. This means you are on the right track, making some good decisions, and have obviously done a little reading. I suspect a little more reading of the Connector documentation will help you understand exactly how all of this works. Nick P.S. Please remember to bottom post. Don't top post. -Original Message- From: Nick Williams [mailto:nicho...@nicholaswilliams.net] Sent: Wednesday, March 20, 2013 1:44 AM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote: From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Nick Williams nicho...@nicholaswilliams.net wrote: On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote: Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. The correctness of that statement is highly dependent on the connector used and how it is configured. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. +1 If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. How many servers you need will depend on the app and how those users use it. It is perfectly possible for one server to serve 20,000 simultaneous users. 20,000 simultaneous requests is more likely to require multiple servers but again - it depends. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Mark Thomas wrote: Nick Williams nicho...@nicholaswilliams.net wrote: On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote: Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. The correctness of that statement is highly dependent on the connector used and how it is configured. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. +1 If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. How many servers you need will depend on the app and how those users use it. It is perfectly possible for one server to serve 20,000 simultaneous users. 20,000 simultaneous requests is more likely to require multiple servers but again - it depends. If I may add something : at some point, on each Tomcat server, there is only 1 listening socket for one port (the point being : you could have several, but you won't have 8000). So even if your clients really send 8000 TCP requests to the server at the same moment, they will be serialized at some point, and from the server's point of view, they come in one by one. /Then/ the server (provided it's TCP stack and all the rest is fast enough) can start distributing them over a pool of Threads. I know that this is an absurd example, but just to provide one extreme point of view : unless you have 8000 independent clients each firing one request at the sam time over 8000 cables connecting to 8000 ports, each one being served by one CPU core running its own TCP stack and its own Tomcat, you will never really have 8000 requests being processed really simultaneously. So the simultaneousness is a question of degree, not an absolute value. And the degree of simultaneousness depends on many factors, among which are (but none of them to be considered alone) : the network, the front-end, the client keepalive setting, the server keepalive setting, the number of connections that the client opens simultaneously, the choice of Connector(s), the usage or not of an Executor, the degree to which the requests (and responses) are really similar, the number of configured threads, the time needed to process each request, the CPU speed, the amount of memory etc. etc. And finally, the number of Threads that are started or are running simultaneously depends on all these factors, and the only one who has access to all these factors is you. If you fire 8000 simultaneous requests from clients, and you see only 200 Threads running to process them, then obviously there is a bottleneck somewhere. But it is not necessarily Tomcat which is not allocating all the Threads that it could be allocating. It could just be that Tomcat does not get more requests to allocate a Thread for, because they get slowed down (or discarded) somewhere else along the line. Or it could be that Tomcat just is not getting enough CPU cycles to be able to allocate more Threads and running them. I some configurations, and with some kinds of client requests patterns, a longer keepalive setting can make it so that one Tomcat thread stays allocated to one client connection, waiting for futher requests on that connection (which may never come). In some kinds of use cases, that is efficient, in others it is very inefficient. Like everything else, it depends.. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/19/13 8:55 PM, Saurabh Agrawal wrote: What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Tomcat will assign an arbitrary (i.e. random) thread from the pool. If I look at my server logs for both the requests from client, the thread id remains the same which gives me a feeling the state of HTTP is persistent and till the time browser is not closed, the thread won't be returned to the pool. Please can someone clarify the above behavior. What you are probably observing is a server under very low load (maybe just your two example requests) that just happens to process the two requests with the same thread. Don't worry: the threads go back into the pool right away (after the keepalive timeout, if you are using the BIO connector which is currently the default). Note: I need to configure the maxThreads setting of Tomcat to support 20,000 concurrent users in the system. The above clarification will help me pick the appropriate setting for maxThreads. 20,000 concurrent users is no problem. How often do they all make requests? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJvhAACgkQ9CaO5/Lv0PAkXwCfS5GE/d1yk+w1gN6xNvQ9LLfA y7sAoJjfeun+2Njsu47uBMrWhDl1xzIv =en0N -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Nick, On 3/19/13 10:31 PM, Nick Williams wrote: I'm glad to see that you have a working cluster of some sorts and that you are performing load tests to evaluate its performance. This means you are on the right track, making some good decisions, and have obviously done a little reading. +1000 It's refreshing to see someone actually performing load-tests against their configuration while tweaking their settings rather than just saying I need to support a million users, setting maxThreads=100, setting PermGen to 3GiB, changing the thread stack size, and micro-managing the garbage collector. Well done. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJvvcACgkQ9CaO5/Lv0PDthgCaA+XQogABsObUGwSUYCeOXadA BAEAnA7+BHlzmZkgKgAKwXZ0v+9pGrga =yQdQ -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
There will be a load of 20,000 users for a special festival for around 4-5 hours ONLY. Regards, Saurabh Agrawal Manager Technology | Sapient -Original Message- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Sent: Wednesday, March 20, 2013 1:48 PM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/19/13 8:55 PM, Saurabh Agrawal wrote: What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Tomcat will assign an arbitrary (i.e. random) thread from the pool. If I look at my server logs for both the requests from client, the thread id remains the same which gives me a feeling the state of HTTP is persistent and till the time browser is not closed, the thread won't be returned to the pool. Please can someone clarify the above behavior. What you are probably observing is a server under very low load (maybe just your two example requests) that just happens to process the two requests with the same thread. Don't worry: the threads go back into the pool right away (after the keepalive timeout, if you are using the BIO connector which is currently the default). Note: I need to configure the maxThreads setting of Tomcat to support 20,000 concurrent users in the system. The above clarification will help me pick the appropriate setting for maxThreads. 20,000 concurrent users is no problem. How often do they all make requests? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJvhAACgkQ9CaO5/Lv0PAkXwCfS5GE/d1yk+w1gN6xNvQ9LLfA y7sAoJjfeun+2Njsu47uBMrWhDl1xzIv =en0N -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/20/13 3:27 AM, Saurabh Agrawal wrote: The point I was trying to make when I mentioned that all requests from the same browser results in same thread is not a mere coincidence. The reason why it may not be sheer luck is because we have seen this on production which is accessed by more than 1000 simultaneous users on the site. Currently we trace the user journey by the thread id in the logs and my observation is all 1000 users (across the globe) gets the same thread id from the pool when they access subsequent URLS from the browser. Is the reverse true? Do you ever see thread A used by more than one client? Or is it a one-to-one mapping? It makes me feel as if the thread is not returned to the pool till the session times out or the user has closed the browser. Although the behavior is weird because tomcat won't should not have a persistent connection as it would mean improper use of the resources (thread). Tomcat is doing its job just fine, otherwise half the Java servers in the world would lock-up and stop working. What may be happening is that your browser is maintaining a keep-alive connection long enough (maybe 10 seconds?) that the next request can be sent over the same connection, and will get the same thread. So, if you have fairly quick-responding clients, you may see that each one is kind of monopolizing a thread for a while. Honestly, if that's the case, it's a *good* case as long as you aren't running out of threads. It would be interesting to see how you have your Connector configured, and how you have your load-balancer configured, too. Also I understand there may be difference in milli seconds or seconds between 1 concurrent user requests and it may not be exactly 1 users in system at one go but I can also simulate 10,000 concurrent request at same time even in milli seconds. Google analytics does show 10,000 concurrent users. In that case I assume it will leverage 10,000 concurrent threads. Users != requests. 10,000 concurrent users might generate 1000 simultaneous requests (that is, requests in-progress at a particular instant), unless they are really pounding on your cluster. What do your thread-pool usage metrics look like? Are you collecting that data (hint: do it)? Are you graphing them (hint: do that, too)? Doing time-based graphing can help you visualize your user load (measured in requests-per-second versus total average concurrent users, which is a nearly useless metric unless you have 100MiB sessions or something and you have to use that as part of your capacity plan) and you can even see things like hey, every day at 11:00 PDT we hit our maxThreads for 5 minutes. That lets you know that you need to give yourself a little more breathing room and then watch for another couple of days. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJwI8ACgkQ9CaO5/Lv0PD8QACdHdYnkzzZbcbmq4dmayaguTp/ eDcAn2dGIn63MJRtbllnJcaicIClmJDp =AfQh -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/20/13 6:52 AM, André Warnier wrote: If I may add something : at some point, on each Tomcat server, there is only 1 listening socket for one port (the point being : you could have several, but you won't have 8000). So even if your clients really send 8000 TCP requests to the server at the same moment, they will be serialized at some point, and from the server's point of view, they come in one by one. While that is true, the threaded-dispatch of those requests does mean that many requests can be processed simultaneously. Yes, they are de-queued serially, but that happens very quickly compared to the duration of the actual requests. So, you can have 200 simultaneous requests /in-process/... not that they all arrive at the exact same instant in time, but that they are all being served (i.e. have threads assigned and are actually doing work) simultaneously. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJwWYACgkQ9CaO5/Lv0PAtSQCaA1QrSvIOd4GPEKsA2+RezsxQ RW4AnRBtpcy4LBypi9ShRmproEcerrjd =inUq -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/19/13 9:46 PM, Saurabh Agrawal wrote: Executor name=hybrisExecutor namePrefix=hybrisHTTP maxThreads=${tomcat.maxthreads} minSpareThreads=${tomcat.minsparethreads} maxIdleTime=${tomcat.maxidletime}/ Connector port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200 protocol=org.apache.coyote.ajp.AjpProtocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Connector port=${tomcat.http.port} maxHttpHeaderSize=8192 maxThreads=${tomcat.maxthreads} protocol=org.apache.coyote.http11.Http11Protocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Note that your Executor has maxThreads=200 and your Connector uses that Executor: your ${tomcat.maxthreads} is being ignored. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlFJwdIACgkQ9CaO5/Lv0PDOMwCfUxQ+hIaIIWlVfOok6b2+tawK eSoAn2Ivy26EMiLbhnaPs6VH35ZECbgB =6T32 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: RE: Tomcat Behavior on Multiple HTTP requests from same browser We have not set the keep alive explicitly in tomcat's server.xml. It's on by default. Connector port=${tomcat.ajp.port} Connector port=${tomcat.http.port} You have both HTTP and AJP Connectors defined; are the requests coming in over both or just one of them? The discussion so far has been primarily related to HTTP, not AJP. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/19/13 9:46 PM, Saurabh Agrawal wrote: Executor name=hybrisExecutor namePrefix=hybrisHTTP maxThreads=${tomcat.maxthreads} minSpareThreads=${tomcat.minsparethreads} maxIdleTime=${tomcat.maxidletime}/ Connector port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200 protocol=org.apache.coyote.ajp.AjpProtocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Connector port=${tomcat.http.port} maxHttpHeaderSize=8192 maxThreads=${tomcat.maxthreads} protocol=org.apache.coyote.http11.Http11Protocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Note that your Executor has maxThreads=200 and your Connector uses that Executor: your ${tomcat.maxthreads} is being ignored. That, and the default keepalive setting, are probably the keys here. And the observation of Chuck about the HTTP and AJP connectors. Over which Connector do the test requests actually come in ? And a question : is the simulation with the 1 clients really comparable to what you expect in the reality ? For example, if the simulation requests one page per client, and then does nothing else with that page; but the real clients would get a page, and then immediately request the 50 thumbnail images referenced by that page, conditions would be really different, and keepalive would have a very different effect. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
-Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: Wednesday, March 20, 2013 3:27 PM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/19/13 9:46 PM, Saurabh Agrawal wrote: Executor name=hybrisExecutor namePrefix=hybrisHTTP maxThreads=${tomcat.maxthreads} minSpareThreads=${tomcat.minsparethreads} maxIdleTime=${tomcat.maxidletime}/ Connector port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200 protocol=org.apache.coyote.ajp.AjpProtocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Connector port=${tomcat.http.port} maxHttpHeaderSize=8192 maxThreads=${tomcat.maxthreads} protocol=org.apache.coyote.http11.Http11Protocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Note that your Executor has maxThreads=200 and your Connector uses that Executor: your ${tomcat.maxthreads} is being ignored. That, and the default keepalive setting, are probably the keys here. And the observation of Chuck about the HTTP and AJP connectors. Over which Connector do the test requests actually come in ? Saurabh - The actual front end requests come on AJP port. We are using AJP protocol for communication between Apache and Tomcat. It helps in load balancing across the application servers in cluster. There is a separate internal application (not exposed on internet) used by CMS team which is using HTTP connector. I hope that clarifies. And a question : is the simulation with the 1 clients really comparable to what you expect in the reality ? For example, if the simulation requests one page per client, and then does nothing else with that page; but the real clients would get a page, and then immediately request the 50 thumbnail images referenced by that page, conditions would be really different, and keepalive would have a very different effect. Saurabh - The way we have configured our user journeys are as follows: User 1: Hits homepage, clicks football link on home page, makes a selection, adds to cart and checkout. So this is one user journey which triggers multiple requests. All our assets are served from L3 CDN. So the asset requests never come to the application server. We have not set keep alive explicitly anywhere in tomcat. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Saurabh Agrawal wrote: -Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: Wednesday, March 20, 2013 3:27 PM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Saurabh, On 3/19/13 9:46 PM, Saurabh Agrawal wrote: Executor name=hybrisExecutor namePrefix=hybrisHTTP maxThreads=${tomcat.maxthreads} minSpareThreads=${tomcat.minsparethreads} maxIdleTime=${tomcat.maxidletime}/ Connector port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200 protocol=org.apache.coyote.ajp.AjpProtocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Connector port=${tomcat.http.port} maxHttpHeaderSize=8192 maxThreads=${tomcat.maxthreads} protocol=org.apache.coyote.http11.Http11Protocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Note that your Executor has maxThreads=200 and your Connector uses that Executor: your ${tomcat.maxthreads} is being ignored. That, and the default keepalive setting, are probably the keys here. And the observation of Chuck about the HTTP and AJP connectors. Over which Connector do the test requests actually come in ? Saurabh - The actual front end requests come on AJP port. We are using AJP protocol for communication between Apache and Tomcat. Right. So then I suppose that Christopher's note is not applicable. Probablya he misread, because the way in which you pasted the configuration in the email makes it difficult to read, after a couple of cut-and-paste. As far as I can tell, the AJP connector refers to the Executor, and the Executor specifies maxThreads=${tomcat.maxthreads}. The main point of Christopher was that you specify a maxThreads parameter in both of your Connectors, but because they both use the Executor, this parameter is being ignored in the Connector, and it is only the maxThreads in the Executor that counts. It helps in load balancing across the application servers in cluster. There is a separate internal application (not exposed on internet) used by CMS team which is using HTTP connector. I hope that clarifies. Yes. And a question : is the simulation with the 1 clients really comparable to what you expect in the reality ? For example, if the simulation requests one page per client, and then does nothing else with that page; but the real clients would get a page, and then immediately request the 50 thumbnail images referenced by that page, conditions would be really different, and keepalive would have a very different effect. Saurabh - The way we have configured our user journeys are as follows: User 1: Hits homepage, clicks football link on home page, makes a selection, adds to cart and checkout. So this is one user journey which triggers multiple requests. Hi. Your usage of user journey is a bit obscure (to me at least) in the context of analysing a matter of tomcat request/response performance. I kind of understand what you mean, but it does not really provide the answer to the questions : - is this what you are using in your tests ? - are you doing this same series of requests for each of your 1 test clients ? - does this represent (more or less) what you are expecting later in production ? The point here was to avoid a case where you would be optimising the parameters in function of a benchmark test, and then find out later that your production case is totally different, and your optimal benchmark settings are totally inappropriate for the production case. All our assets are served from L3 CDN. So the asset requests never come to the application server. That, I do not understand. I do not understand what you mean by assets here, and I do not understand L3 CDN. So I cannot tell of this is relevant or not to the problem. Have pity for the people trying to help you here, who only know Tomcat and HTTP/AJP. Try to use vocabulary that we understand, and you may get better help. We have not set keep alive explicitly anywhere in tomcat. What Chuck was telling you in an earlier message, is that even if you do not set it explicitly, it is set to some default non-zero value by Tomcat. Look at this page in the on-line documentation : http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html#Standard_Implementations So, by default (unless you explicitly define it), keepAliveTimeout : The number of milliseconds this Connector will wait for another AJP request before closing the connection. The default value is to use the value that has been set for the connectionTimeout attribute. and connectionTimeout The number of milliseconds this Connector will wait, after accepting a connection, for the request URI line to be presented. The default value for AJP protocol
Re: Tomcat Behavior on Multiple HTTP requests from same browser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/20/13 2:25 PM, André Warnier wrote: Saurabh Agrawal wrote: All our assets are served from L3 CDN. So the asset requests never come to the application server. That, I do not understand. I do not understand what you mean by assets here, and I do not understand L3 CDN. So I cannot tell of this is relevant or not to the problem. CDN = Content Delivery Network. I'm not sure what L3 (probably Level 3, a data center operations company) is, but a CDN is basically a whole bunch of copies of your files geographically distributed such that requesting a file always gets the bits that are closest to you. Kind of a cool thing. ;) The bottom line is that Saurabh expects only dynamic requests to come to Tomcat, so keepalives should be much less useful than if Tomcat were to be serving everything. Imagine httpd out front serving all static content and forwarding dynamic stuff to Tomcat via AJP -- that's almost exactly what's going on here, except that the static stuff is being served very efficiently from a network-topology perspective. Since AJP is in use, keepalive is almost entirely a red herring as typical AJP connections are permanently-connected to the web server. So, by default, the keepAliveTimeout [for AJP] is set to infinite. [snip] And if the client keeps the connection open, but does not send any additional request on that connection, the Thread will wait theoretically forever (because that is what the documentation says about the default value of these parameters). No, the defaults are different for non-AJP connections. Tomcat's default default is 60 seconds but the stock server.xml configures it to 20 seconds. Now your case is a bit different, because - you are not using the HTTP BIO connector (you use AJP) I think you've gotten yourself confused, here, unfortunately. You can use AJP with BIO, NIO, or APR (maybe you mixed-up AJP and APR between your eyes and your brain... the two are honestly too close to each other and it is very easy to do that). He is in fact using the BIO connector because he has specified protocol=org.apache.coyote.ajp.AjpProtocol. - in front of your Tomcat, is an Apache httpd server. This server has its own keep-alive settings which apply to the connection of the client with Apache httpd. And these keep-alive settings are a bit different from the Tomcat ones (for example, there is a keep-alive timeout, but also a MaxKeepAliveRequests) +1 - between Apache httpd and Tomcat, there is the mod_jk module in Apache, and that module uses its own timeouts (as set in workers.properties), and in addition it uses itself a pool of connections to Tomcat, and this pool of connections has its own rules for keeping alive a connection between Apache and Tomcat. But the basic principles above apply, and may explain why you are seeing what appears to be one Thread dedicated to one client, forever. I think there might be a problem with the instrumentation, or just coincidences at a fairly implausible level. The trust of the matter is that Tomcat does not allocate a thread permanently to a remote client until ... whenever the client disconnects (whatever that means, as HTTP is a connection-less protocol). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRSlo8AAoJEBzwKT+lPKRYodAP/Rz73chvzhyYPNWpePlvPaBm x7bFxFCNg7hbHV7X1skgfG4CYbR9AsjFqcgkXovVSBcIZjKoNT/yCqwL20SMdNIo glkx1Gz6gBfM8Ad68VL4xr7nGtwC1WraN2tCszc8dNrBJOjmQwp5tX09OWtpsxqe DNQVfhGk8wtHmPWjXgK8Kr0lZDjfz540bp4OSAwohjixsCTr3C1PqVEiiZyYWazC B/5tEkeJ7YtPQJqmJhgx1uR+CmU3ty5iQpLmr4K9uKTLSTjs3HGFV0nkCKQIXmQs vzjt0eWr8VzbbZnY4QBfBIntgjQiDIoK9Mi+Q3uilYbjCJuFw4jJGICAjaZjlYxW eW29Ag/WGthB6shOwStiLS+/KUenzXY9aNzHQ1sCeB9Pdsy2eI+Yg15TomwcfxcV 2LjtHZO13QQeGcHpr5vBId8x/B9x5W1bzs8gzUALPQhdsYqGixohu4Ad4nqmeBI0 VZ02jiEDQqr4zT6oJwDVPpm8FSS2wBZKURIp80iI4B2+bS1VPcKzY/J/aKCQRQCw KP4VDtNiAOXD4a+8sHuZrOh8Q/splNnbhf2G687PryjfxQuk+HQPY/XfFvY1CXzD ZZD1yVjkSaRfXDmDCfFxfD+a3lOmGhAa8NNR4dtJJo1w1lhrYXvx1ujdS5V3FgO5 xdKlzkg4u30lRrAQq6IO =yLs6 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat Behavior on Multiple HTTP requests from same browser
Hi, I have tried to find the simplest of answer from so many tomcat forums but have not got concrete answer to my query. The query I have is around management of multiple HTTP requests triggering from same browser one after other and being served by Tomcat container. Let me take an example. Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? If I look at my server logs for both the requests from client, the thread id remains the same which gives me a feeling the state of HTTP is persistent and till the time browser is not closed, the thread won't be returned to the pool. Please can someone clarify the above behavior. Note: I need to configure the maxThreads setting of Tomcat to support 20,000 concurrent users in the system. The above clarification will help me pick the appropriate setting for maxThreads. Saurabh Agrawal Manager Technology | Sapient Aachvis Softech Private Limited SEZ, Oxygen, (Tower C), Ground - 3rd Floor, Plot No. 7, Sector 144 Expressway, Noida 201 301 Uttar Pradesh, India Tel: +91 (120) 479 5000 Mobile: +91 981 866 4383 Fax: +91 (120) 479 5001 Email: sagra...@sapient.commailto:sagra...@sapient.com sapient.com The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any (your) computer. ***Please consider the environment before printing this email.***
RE: Tomcat Behavior on Multiple HTTP requests from same browser
From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote: From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.) If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering. Nick - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Hi chuck, We have not set the keep alive explicitly in tomcat's server.xml. Howeverm when I print the thread id for all requests from same browser, it prints the same thread it. It gives me a feeling that all HTTP requests are holding worker thread of tomcat if they originate from same browser. My tomcat configuration is as follows: Server port=${tomcat.internalserver.port} shutdown=SHUTDOWN Listener className=org.apache.catalina.core.JasperListener / Listener className=org.apache.catalina.mbeans.ServerLifecycleListener / Listener className=org.apache.catalina.mbeans.GlobalResourcesLifecycleListener / Service name=Catalina Executor name=hybrisExecutor namePrefix=hybrisHTTP maxThreads=${tomcat.maxthreads} minSpareThreads=${tomcat.minsparethreads} maxIdleTime=${tomcat.maxidletime}/ Connector port=${tomcat.ajp.port} maxHttpHeaderSize=8192 maxThreads=200 protocol=org.apache.coyote.ajp.AjpProtocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Connector port=${tomcat.http.port} maxHttpHeaderSize=8192 maxThreads=${tomcat.maxthreads} protocol=org.apache.coyote.http11.Http11Protocol executor=hybrisExecutor enableLookups=false acceptCount=100 connectionTimeout=2 URIEncoding=UTF-8 disableUploadTimeout=true / Engine name=Catalina defaultHost=localhost jvmRoute=${tomcat.ajp.worker} Valve className=org.apache.catalina.valves.FastCommonAccessLogValve directory=${HYBRIS_LOG_DIR}/tomcat prefix=access. suffix=.log pattern=%{X-Forwarded-For}i %l %u %t %T %r %s %b %{User-Agent}i %{Referer}i %{JSESSIONID}c / Host name=localhost appBase=webapps unpackWARs=true autoDeploy=false xmlValidation=false xmlNamespaceAware=false ${tomcat.webapps.60} /Host /Engine /Service /Server Regards, Saurabh Agrawal Manager Technology | Sapient -Original Message- From: Caldarale, Charles R [mailto:chuck.caldar...@unisys.com] Sent: Wednesday, March 20, 2013 1:37 AM To: Tomcat Users List Subject: RE: Tomcat Behavior on Multiple HTTP requests from same browser From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Regards, Saurabh Agrawal Manager Technology | Sapient -Original Message- From: Nick Williams [mailto:nicho...@nicholaswilliams.net] Sent: Wednesday, March 20, 2013 1:44 AM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote: From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.) If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering. Nick - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote: Hi Nick, We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less. My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users. Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. Remember that real users will have some time between each request, ranging from a few seconds to several minutes depending on the page and the user. A good load testing tool like JMeter, The Grinder, or NeoLoad (just examples) will mimic this behavior by putting varying pauses between requests. It sounds like whatever tool you are using is doing this--that's good! So, 8,000 simultaneous users might result in only 400-500 simultaneous requests (just a guess; depends on your application and its user base). In this case, you'd only see 400-500 threads used across the cluster. In your small-scale test, where each request from your browser got the same thread, that could just be pure luck. If you're the only person using the server, it's likely that the Thread pool is extremely small, thus the odds are high that the same thread would get picked over and over again. However, in a high-load environment, this will not be the case, and the number of threads will nearly always be less than the number of simultaneous users. I'm glad to see that you have a working cluster of some sorts and that you are performing load tests to evaluate its performance. This means you are on the right track, making some good decisions, and have obviously done a little reading. I suspect a little more reading of the Connector documentation will help you understand exactly how all of this works. Nick P.S. Please remember to bottom post. Don't top post. -Original Message- From: Nick Williams [mailto:nicho...@nicholaswilliams.net] Sent: Wednesday, March 20, 2013 1:44 AM To: Tomcat Users List Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote: From: Saurabh Agrawal [mailto:sagra...@sapient.com] Subject: Tomcat Behavior on Multiple HTTP requests from same browser Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser. What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ? Normally, the first available thread from the pool is used for each request. However, if you are using the BIO Connector with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active. See the HTTP Connector doc, especially the Connector Comparison at the end. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html - Chuck I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.) If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering. Nick - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h