On 05/09/2017 19:56, Yasser Zamani wrote:
> Thanks a lot Mark!
>
> Yes I knew these and before tested that a tomcat with 400 max threads
> "scalabilitaly is equal" to a tomcat with 200 max threads but with
> servlet 3's async API including application's thread pool with size 200.
>
> However so far I thought Oracle's docs are like standards and tomcat
> have to satisfy them :)
Tomcat implements the Servlet, JSP, UEL, EL and JASPIC specifications.
The document you refer to is not part of those specs and, as I said, it
is misleading at best.
>> That does increase scalability
>> because rather than having a bunch of threads waiting for these
>> non-blocking operations to complete, those threads can do useful work.
>
> But tomcat blocks another thread from container's thread pool for
> waiting or locking on that non-blocking operation's response!
As I said, if the async API is used to move a blocking operation from
one thread to another, that won't improve scalability.
You are only going to improve scalability if you move non-blocking
operations from the Servlet.service() method (which has to block waiting
for the non-blocking operation to complete) to the async API.
Essentially, if you leave it in the service() method you have one thread
allocated to each non-blocking operation.
If the Servlet async API is used, the non-blocking operation is started
and the container thread continues to complete the service method. The
container thread is now free to do other useful work and the
non-blocking operation isn't using any thread at all - and won't until
the operation completes at which point it will require a thread to
perform the dispatch back to the container and then to process that
dispatch.
In this case a container thread is only required up to the point where
there non-blocking operation starts and form the point where it
completes. While the non-blocking operation is in progress, the
container thread is free to do other useful work.
Mark
> so I do
> not agree that "because those threads can do useful work" then "does
> increase scalability". I think Servlet 3's async API here may increase
> scalability if and only if the released thread also releases some
> resources which other threads may are blocked on them. and if and only
> if the new thread does not lock more resources than the original one.
> **Actually as I understand, using Servlet 3's async API compared with
> tomcat's nio with greater max threads, does not have any gain except
> what I wrote above and also preventing deadlocks. wdyt?**
>
> On 9/5/2017 11:57 AM, Mark Thomas wrote:
>> On 03/09/17 09:01, Yasser Zamani wrote:
>>> Hi there,
>>>
>>> At [1] we read:
>>>
>>>> Web containers in application servers normally use a server thread
>>>> per client request. Under heavy load conditions, containers need a
>>>> large amount of threads to serve all the client requests.
>>>> Scalability limitations include running out of memory or
>>>> *exhausting the pool of container threads*. To create scalable web
>>>> applications, you must ensure that no threads associated with a
>>>> request are sitting idle, so *the container can use them to
>>>> process new requests*. Asynchronous processing refers to
>>>> *assigning these blocking operations to a new thread and returning
>>>> the thread associated with the request immediately to the container*.
>>>>
>>> I could not achieve this scalability in tomcat via calling
>>> `javax.servlet.AsyncContext.start(Runnable)`! I investigated the cause
>>> and found it at [2]:
>>>
>>> public synchronized void asyncRun(Runnable runnable) {
>>> ...
>>> processor.execute(runnable);
>>>
>>> I mean `processor.execute(runnable)` uses same thread pool which it's
>>> also it's duty to process new requests! Such usage made things worse!
>>> i.e. not only does not make thread pool more free to process new
>>> requests, but also has an overhead via thread switching!
>>>
>>> I think Tomcat must use another thread pool for such blocking operations
>>> and keep current thread pool free for new requests; It's the philosophy
>>> of Servlet 3.0's asynchronous support according to Oracle's
>>> documentation. wdyt?
>>
>> I think this is a good question that highlights a lot of
>> misunderstanding in this area. The quote above is misleading at best.
>>
>> There is no way that moving a blocking operation from the container
>> thread pool to some other thread will increase scalability any more then
>> simply increasing the size of the container thread pool.
>>
>> Consider the following:
>>
>> - If the system is not at capacity then scalability can be increased by
>> increasing the size of the container thread pool
>>
>> - If the system as at capacity, the container thread pool will need to
>> be reduced to create capacity for these 'other' blocking threads.
>>
>> - If too many resources are allocated to these 'other' blocking threads
>> then scalability will be reduced because there will be idle 'other'
>> blocking threads that could be doing useful work elsewhere such as
>> processing incoming requests.
>>
>> - If too few resources are allocated to these 'other' blocking threads
>> then scalability will be reduced because a bottleneck will have been
>> introduced.
>>
>> - The 'right' level of resources to allocate to these 'other' blocking
>> threads will vary over time.
>>
>> - Rather than try and solve the complex problem of balancing resources
>> across multiple thread pools, it is far simpler to use a single thread
>> pool, as Tomcat does.
>>
>>
>> Servlet 3 async can only increase scalability where the Servlet needs to
>> perform a genuinely non-blocking operation. Prior to the availability of
>> the async API, the Servlet thread would have to block until the
>> non-blocking operation completed. That is inefficient. That does limit
>> scalability. The async API allows this the thread to be released while
>> this non-blocking operation completes. That does increase scalability
>> because rather than having a bunch of threads waiting for these
>> non-blocking operations to complete, those threads can do useful work.
>>
>> Mark
>>
>>
>>>
>>> [1] https://docs.oracle.com/javaee/7/tutorial/servlets012.htm
>>> [2]
>>> https://github.com/apache/tomcat/blob/trunk/java/org/apache/coyote/AsyncStateMachine.java#L451
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
