Hi Guus,

I agree there is some inconsistency, and I need to check why.

Regarding the initial pb, it seems like an exception causes the worker to be removed, and this exception seems to be due to some TLS handshake failure.

I suspect that TLS 1.3 is being used on the client side, which is not currently supported properly by MINA (just an hypothesis).

Regardless, there is something puzzling:

private class Worker implements Runnable {

    public void run() {

        try {
            for (;;) {
                IoSession session = fetchSession();

                idleWorkers.decrementAndGet();

                if (session == null) {
                    synchronized (workers) {
                        if (workers.size() > getCorePoolSize()) {
                            // Remove now to prevent duplicate exit.
                            workers.remove(this);   <<<-------------
                            break;
                        }
            ...


            } finally {
                synchronized (workers) {
                    workers.remove(this); <<------------------------


In the case where we reach the max pool size, we try to remove it twice. Ok, it's not really critical, because the first call will succeed, and the second will do nothing.

And yes, if there is an exception during the runTasks, we will have an increment of the idleWorkers despite the fact the worker will be removed.


Bottom line, there is a need to get this fixed.

Can you open a JIRA for tracability?


On 11/01/2022 17:26, Guus der Kinderen wrote:
Although I'm not sure if this is the source of my issue, I can reproduce the state by creating a unit test that causes a RuntimeException and/or Error to be thrown by a filter that is processing an event.

In OrderedThreadPoolExecutor's Worker, this snippet causes the Worker to be removed, but the idleWorker count to be increased in such a scenario. Should the first try/finally block be removed perhaps (causing idleWorkers to be increased only if runTasks didn't throw anything?

         try {
             if (session !=null) {
                 runTasks(getSessionTasksQueue(session));
             }
         }finally {
             idleWorkers.incrementAndGet();
         }
     }
}finally {
     synchronized (workers) {
         workers.remove(this);
         OrderedThreadPoolExecutor.this.completedTaskCount 
+=completedTaskCount.get();
         workers.notifyAll();
     }
}


On Tue, 11 Jan 2022 at 15:41, Guus der Kinderen <guus.der.kinde...@gmail.com <mailto:guus.der.kinde...@gmail.com>> wrote:

    Hello,

    This was found with MINA 2.1.3. Is this a bug? I stumbled upon this
    while performing analysis of heap dumps of a JVMs that suffered from
    an issue where no user was able to establish a connection to our
    server software anymore.

    Our application uses an ExecutorFilter. The
    OrderedThreadPoolExecutor of the affected server seems to be in an
    inconsistent state:

      * idleWorkers: 2
      * waitingSessions.count: 2293
      * workers.map.size: 0

    I will attempt to include a screenshot of the profiler that holds
    more data (if the mailinglist allows me).

    What strikes me as odd is:

      * No workers, while there are sessions waiting to be processed
      * No workers, but a non-zero idle workers count

    Servers that are unaffected by the issue have an idle worker count
    that is equal to the amount of workers (I assume that the snapshots
    were taken when that server was not actively processing data).

    For reference: the issue that we are experiencing (possibly caused
    by this) is tracked in
    https://igniterealtime.atlassian.net/browse/OF-2367
    <https://igniterealtime.atlassian.net/browse/OF-2367>. Multiple
    people are reporting the issue, but I've been unable to reproduce it
    locally.

    Kind regards,

       Guus

    image.png


--
*Emmanuel Lécharny - CTO* 205 Promenade des Anglais – 06200 NICE
T. +33 (0)4 89 97 36 50
P. +33 (0)6 08 33 32 61
emmanuel.lecha...@busit.com https://www.busit.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
For additional commands, e-mail: users-h...@mina.apache.org

Reply via email to