Re: Tomcat threads
Thanks Folks. On Wed, Mar 3, 2010 at 1:03 PM, Peter Crowther wrote: > On 3 March 2010 18:24, Bharath Vasudevan wrote: > > > Hmmm... > > > > "No, the server will allocate maxThreads request handlers; the other > > requests would sit in the TCP stack's queue (not in the JVM), up to the > > configured acceptCount value - which you can set as high as your OS > > allows." > > > > I was assuming that the tomcat main thread is going to pick it up from > the > > TCP stack and let the web server thread handle the request. Wont tomcat > > reject the request when max threads have been reached (since it would > have > > already picked up tcp layer)? > > > > No, because the container doesn't know when a thread might become > available. Serving content after a delay is considered better than not > serving content at all, so the requests are simply queued in the TCP stack. > > - Peter >
Re: Tomcat threads
Hmmm... "No, the server will allocate maxThreads request handlers; the other requests would sit in the TCP stack's queue (not in the JVM), up to the configured acceptCount value - which you can set as high as your OS allows." I was assuming that the tomcat main thread is going to pick it up from the TCP stack and let the web server thread handle the request. Wont tomcat reject the request when max threads have been reached (since it would have already picked up tcp layer)? On Wed, Mar 3, 2010 at 9:45 AM, Caldarale, Charles R < chuck.caldar...@unisys.com> wrote: > > From: Bharath Vasudevan [mailto:bharath@gmail.com] > > Subject: Re: Tomcat threads > > > > When tomcat gets a request, it does a socket send to some > > other process to handle the request and then respond. > > Tomcat doesn't do that - your webapp does. > > You have now introduced a previously unmentioned dependency - that the > response depends on the behavior of an external resource, unrelated to > Tomcat. That changes the picture considerably. The neat thing the servlet > 3.0 AsyncContext does is handle this case by letting you decouple the > original Request and Response objects from the original processing thread. > > > But assuming 20k client requests come in at the same time, > > the server would try to allocate 20k threads and handle it. > > No, the server will allocate maxThreads request handlers; the other > requests would sit in the TCP stack's queue (not in the JVM), up to the > configured acceptCount value - which you can set as high as your OS allows. > > > Mostly the system would trash and go down. > > Which is my point about why you don't want to set maxThreads to arbitrarily > large (or small) values. > > > But if it was to be handled gracefully, we can have the > > threads pick up the request throw it in a queue and let > > the worker threads work and respond. > > Which you can do in your webapp by sending an interim response to the > client and having the client poll for the final one. You'd have to disable > keep-alives as well. The 3.0 spec allows the container to perform this > task, rather than requiring the webapp to do so. > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you received > this in error, please contact the sender and delete the e-mail and its > attachments from all computers. > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat threads
That would be the last go. Trying to figure out if there is techniques to handle such scenarios. Looks like comet can asynchronously push data to the user. If this is going to be seamless to the client, it might as well look like a response for their request. On Wed, Mar 3, 2010 at 9:25 AM, Mark Thomas wrote: > On 03/03/2010 17:21, Bharath Vasudevan wrote: > > Hi Charles, > > > > Let me explain the scenario. When tomcat gets a request, it does a socket > > send to some other process to handle the request and then respond. This > > would happen fast. But assuming 20k client requests come in at the same > > time, the server would try to allocate 20k threads and handle it. Mostly > the > > system would trash and go down. > > > > But if it was to be handled gracefully, we can have the threads pick up > the > > request throw it in a queue and let the worker threads work and respond. > Let > > me know if I am missing something here. > > Like the fact that Tomncat is open source and you could just look at the > source code for the various connectors and figure this all out for > yourself? You don't even have to read the source. The connectors > documentation covers this pretty well. > > Mark > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat threads
Thanks Bill. Comet is something that I can dig into :). On Tue, Mar 2, 2010 at 7:21 PM, Bill Barker wrote: > > > "Caldarale, Charles R" wrote in message > news:99c8b2929b39c24493377ac7a121e21f96cb817...@usea-exch8.na.uis.unisys.com. > .. > > From: Bharath Vasudevan [mailto:bharath@gmail.com] >>> Subject: Re: Tomcat threads >>> >>> If we get a request on a thread, let some other thread do >>> the work for it and store the response info. The thread >>> which does the work writes the response on that request. >>> >> >> If the processing is fast, why would you go to the complexity and overhead >> of switching to another thread to process the request? >> >> You should also read the servlet spec, in particular SRV.2.3.3.3: >> >> "Implementations of the request and response objects are not guaranteed to >> be thread >> safe. This means that they should only be used within the scope of the >> request handling >> thread. >> >> "References to the request and response objects should not be given to >> objects >> executing in other threads as the resulting behavior may be >> nondeterministic. If >> the thread created by the application uses the container-managed objects, >> such as >> the request or response object, those objects must be accessed only within >> the >> servlet's service life cycle and such thread itself should have a life >> cycle within >> the life cycle of the servlet's service method because accessing those >> objects >> after the service method ends may cause undeterministic problems." >> >> The illogical behavior you're asking for simply isn't allowed. >> >> > At least not until Tomcat 7 ;). In there, you can do exactly what the user > is asking for using the AsyncContext. The details are in the Servlet 3.0 > spec, but basically the request is handed off to another thread and the > original one returns to the pool. This also allows for two-way > communication instead of relying on polling. > > Of course Comet offers similar functionality in Tomcat 6 if you don't mind > being bound to Tomcat. > > > - Chuck >> >> >> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY >> MATERIAL and is thus for use only by the intended recipient. If you received >> this in error, please contact the sender and delete the e-mail and its >> attachments from all computers. >> > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat threads
Hi Charles, Let me explain the scenario. When tomcat gets a request, it does a socket send to some other process to handle the request and then respond. This would happen fast. But assuming 20k client requests come in at the same time, the server would try to allocate 20k threads and handle it. Mostly the system would trash and go down. But if it was to be handled gracefully, we can have the threads pick up the request throw it in a queue and let the worker threads work and respond. Let me know if I am missing something here. Bharath On Tue, Mar 2, 2010 at 7:22 PM, Caldarale, Charles R < chuck.caldar...@unisys.com> wrote: > > From: Bharath Vasudevan [mailto:bharath@gmail.com] > > Subject: Re: Tomcat threads > > > > Why is it illlogical? > > 40+ years of system architecture experience. > > > If the number of requests increases, the number of threads > > that can be handled by the system goes down. > > You'll have to explain that one, since it doesn't make sense as written. > > > The context switches and the pain to handle the switches makes > > handling of the requests in lesser threads which is scalable. > > Nor does that. What are you categorizing as a "lesser thread"? What makes > those any more scalable than any other kind of thread? You're free to set > the thread pool limit to any value you want, including infinity. Setting > the limit to larger than what the logical and physical resources of the > system you're running on can handle will induce performance problems (e.g., > page thrashing, excessive GCs) or outright failures as soon as someone > decides to start pounding on your server. > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you received > this in error, please contact the sender and delete the e-mail and its > attachments from all computers. > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat threads
Why is it illlogical? Fast is a relative term. If the number of requests increases, the number of threads that can be handled by the system goes down . The context switches and the pain to handle the switches makes handling of the requests in lesser threads which is scalable. On Tue, Mar 2, 2010 at 4:34 PM, Caldarale, Charles R < chuck.caldar...@unisys.com> wrote: > > From: Bharath Vasudevan [mailto:bharath@gmail.com] > > Subject: Re: Tomcat threads > > > > If we get a request on a thread, let some other thread do > > the work for it and store the response info. The thread > > which does the work writes the response on that request. > > If the processing is fast, why would you go to the complexity and overhead > of switching to another thread to process the request? > > You should also read the servlet spec, in particular SRV.2.3.3.3: > > "Implementations of the request and response objects are not guaranteed to > be thread > safe. This means that they should only be used within the scope of the > request handling > thread. > > "References to the request and response objects should not be given to > objects > executing in other threads as the resulting behavior may be > nondeterministic. If > the thread created by the application uses the container-managed objects, > such as > the request or response object, those objects must be accessed only within > the > servlet's service life cycle and such thread itself should have a life > cycle within > the life cycle of the servlet's service method because accessing those > objects > after the service method ends may cause undeterministic problems." > > The illogical behavior you're asking for simply isn't allowed. > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you received > this in error, please contact the sender and delete the e-mail and its > attachments from all computers. > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat threads
Thanks Chuck. One thing I am not able to get is why would the user care if the response was done from a different thread? Is it not going to be opaque to the user as long as its fast? If we get a request on a thread, let some other thread do the work for it and store the response info. The thread which does the work writes the response on that request. Regards, Bharath On Tue, Mar 2, 2010 at 3:57 PM, Caldarale, Charles R < chuck.caldar...@unisys.com> wrote: > > From: Bharath Vasudevan [mailto:bharath@gmail.com] > > Subject: Tomcat threads > > > > Are they user level threads or kernel threads? > > Depends on the particular JVM being used, not on Tomcat. Pretty much all > modern JVMs use a kernel thread for each java.lang.Thread instance on which > start() has been called. > > > I see that the requests are kicked off through the threads from a pool > > (which has a configured size), how can this be scalable? > > Because you can make the limit as large as you want. Making it too large > leaves you open to DOS attacks and running out of resources on the server. > > > Also, from my understanding in servlets, the response for a > > request in has to be done then and there (as in I cant respond > > for a request later in future from some random thread) . > > That's the nature of HTTP, not servlets in particular. A properly designed > application facing a long-duration situation will usually send an interim > status back to the client, and kick off an auxiliary thread to do whatever > is going to run for a while. The auxiliary thread places the final response > in some location associated with the client when it's ready. The client > polls for status and the final response using JavaScript or some other > mechanism. > > > If there is contention for a resource which might take time > > (causing the response to take time), wont tomcat run out of > > threads? > > Yes; Tomcat won't care, but your users might. Don't use poorly designed > applications. > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you received > this in error, please contact the sender and delete the e-mail and its > attachments from all computers. > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Tomcat threads
Hi, I would like to know more on the threads created in tomcat. Are they user level threads or kernel threads? I see that the requests are kicked off through the threads from a pool (which has a configured size), how can this be scalable? Also, from my understanding in servlets, the response for a request in has to be done then and there (as in I cant respond for a request later in future from some random thread) . If there is contention for a resource which might take time (causing the response to take time), wont tomcat run out of threads? Regards, Bharath
