On 9/13/2017 10:25 PM, Yasser Zamani wrote: > > > On 9/13/2017 9:49 PM, Mark Thomas wrote: >> On 05/09/2017 19:56, Yasser Zamani wrote: >>> Thanks a lot Mark! >>> >>> Yes I knew these and before tested that a tomcat with 400 max threads >>> "scalabilitaly is equal" to a tomcat with 200 max threads but with >>> servlet 3's async API including application's thread pool with size 200. >>> >>> However so far I thought Oracle's docs are like standards and tomcat >>> have to satisfy them :) >> >> Tomcat implements the Servlet, JSP, UEL, EL and JASPIC specifications. >> >> The document you refer to is not part of those specs and, as I said, it >> is misleading at best. >> >>>> That does increase scalability >>>> because rather than having a bunch of threads waiting for these >>>> non-blocking operations to complete, those threads can do useful work. >>> >>> But tomcat blocks another thread from container's thread pool for >>> waiting or locking on that non-blocking operation's response! >> >> As I said, if the async API is used to move a blocking operation from >> one thread to another, that won't improve scalability. >> >> You are only going to improve scalability if you move non-blocking >> operations from the Servlet.service() method (which has to block waiting >> for the non-blocking operation to complete) to the async API. >> Essentially, if you leave it in the service() method you have one thread >> allocated to each non-blocking operation. >> >> If the Servlet async API is used, the non-blocking operation is started >> and the container thread continues to complete the service method. The >> container thread is now free to do other useful work and the >> non-blocking operation isn't using any thread at all - and won't until >> the operation completes at which point it will require a thread to >> perform the dispatch back to the container and then to process that >> dispatch. >> > > :S I apologize as I know that I fails to understand some points again. > Please excuse me bothering you. > > The non-blocking operation isn't using any thread at all ?! So which > thread executes my Runnable passed to startAsync? At my first initial > mail, I see Tomcat passes that to it's thread pool. If I define my > non-blocking operation (that Runnable) as > System.out.println(Thread.getCurrentThread.getName()), I'll see > "http-nio-exec-XX which means Tomcat's thread pool has been allocated a > thread for my Runnable non-blocking operation. However, maybe you mean a > real IO wait as non-blocking operation. > > Let forget scalablity. Could you please define an example business > operational and functional scenario which using Servlet's async API > improves or resolve it in such way that could not being improved or > resolved with NIO and increasing maxThreads and maxConnections? >
Thank you Mark. I wish to close and conclude this thread as below: To whom concern, I found an excellent article at [1] which nicely describes what makes difference between increasing pool sizes or using Servlet 3's Async API. He says: > We have solved the problem of the HTTP thread pool exhaustion, but the number > of required threads to handle the requests has not improved: we are just > spawning background threads to handle the requests. In terms of simultaneous > running thread count this should be equivalent to simply increase the HTTP > thread pool size: under heavy load the system will not scale. > > Effective usage of Asynchronous Servlets > In order to demonstrate the powerful features offered by asynchronous > servlets, we will implement the following use case: > > * There is a file which size is 100 bytes that may be streamed to remote > clients > > * We will have a background thread pool with a predefined number of threads > that will be responsible to stream the file to remote clients > > * The HTTP threads will handle incoming requests and immediately pass them to > the background thread pool > > * The background threads will send chunks of 10 bytes to the remote clients > in a round robin fashion Best Regards, Yasser. [1] http://www.byteslounge.com/tutorials/asynchronous-servlets-in-java >> In this case a container thread is only required up to the point where >> there non-blocking operation starts and form the point where it >> completes. While the non-blocking operation is in progress, the >> container thread is free to do other useful work. >> >> Mark >> >> >>> so I do >>> not agree that "because those threads can do useful work" then "does >>> increase scalability". I think Servlet 3's async API here may increase >>> scalability if and only if the released thread also releases some >>> resources which other threads may are blocked on them. and if and only >>> if the new thread does not lock more resources than the original one. >>> **Actually as I understand, using Servlet 3's async API compared with >>> tomcat's nio with greater max threads, does not have any gain except >>> what I wrote above and also preventing deadlocks. wdyt?** >>> >>> On 9/5/2017 11:57 AM, Mark Thomas wrote: >>>> On 03/09/17 09:01, Yasser Zamani wrote: >>>>> Hi there, >>>>> >>>>> At [1] we read: >>>>> >>>>>> Web containers in application servers normally use a server >>>>>> thread >>>>>> per client request. Under heavy load conditions, containers >>>>>> need a >>>>>> large amount of threads to serve all the client requests. >>>>>> Scalability limitations include running out of memory or >>>>>> *exhausting the pool of container threads*. To create >>>>>> scalable web >>>>>> applications, you must ensure that no threads associated with a >>>>>> request are sitting idle, so *the container can use them to >>>>>> process new requests*. Asynchronous processing refers to >>>>>> *assigning these blocking operations to a new thread and >>>>>> returning >>>>>> the thread associated with the request immediately to the >>>>>> container*. >>>>>> >>>>> I could not achieve this scalability in tomcat via calling >>>>> `javax.servlet.AsyncContext.start(Runnable)`! I investigated the cause >>>>> and found it at [2]: >>>>> >>>>> public synchronized void asyncRun(Runnable runnable) { >>>>> ... >>>>> processor.execute(runnable); >>>>> >>>>> I mean `processor.execute(runnable)` uses same thread pool which it's >>>>> also it's duty to process new requests! Such usage made things worse! >>>>> i.e. not only does not make thread pool more free to process new >>>>> requests, but also has an overhead via thread switching! >>>>> >>>>> I think Tomcat must use another thread pool for such blocking >>>>> operations >>>>> and keep current thread pool free for new requests; It's the >>>>> philosophy >>>>> of Servlet 3.0's asynchronous support according to Oracle's >>>>> documentation. wdyt? >>>> >>>> I think this is a good question that highlights a lot of >>>> misunderstanding in this area. The quote above is misleading at best. >>>> >>>> There is no way that moving a blocking operation from the container >>>> thread pool to some other thread will increase scalability any more >>>> then >>>> simply increasing the size of the container thread pool. >>>> >>>> Consider the following: >>>> >>>> - If the system is not at capacity then scalability can be increased by >>>> increasing the size of the container thread pool >>>> >>>> - If the system as at capacity, the container thread pool will need to >>>> be reduced to create capacity for these 'other' blocking threads. >>>> >>>> - If too many resources are allocated to these 'other' blocking threads >>>> then scalability will be reduced because there will be idle 'other' >>>> blocking threads that could be doing useful work elsewhere such as >>>> processing incoming requests. >>>> >>>> - If too few resources are allocated to these 'other' blocking threads >>>> then scalability will be reduced because a bottleneck will have been >>>> introduced. >>>> >>>> - The 'right' level of resources to allocate to these 'other' blocking >>>> threads will vary over time. >>>> >>>> - Rather than try and solve the complex problem of balancing resources >>>> across multiple thread pools, it is far simpler to use a single >>>> thread >>>> pool, as Tomcat does. >>>> >>>> >>>> Servlet 3 async can only increase scalability where the Servlet >>>> needs to >>>> perform a genuinely non-blocking operation. Prior to the >>>> availability of >>>> the async API, the Servlet thread would have to block until the >>>> non-blocking operation completed. That is inefficient. That does limit >>>> scalability. The async API allows this the thread to be released while >>>> this non-blocking operation completes. That does increase scalability >>>> because rather than having a bunch of threads waiting for these >>>> non-blocking operations to complete, those threads can do useful work. >>>> >>>> Mark >>>> >>>> >>>>> >>>>> [1] https://docs.oracle.com/javaee/7/tutorial/servlets012.htm >>>>> [2] >>>>> https://github.com/apache/tomcat/blob/trunk/java/org/apache/coyote/AsyncStateMachine.java#L451 >>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
