On 20/01/2023 20:28, Marc Boorshtein wrote:
Here's an additional datapoint.  Enabling TLS eliminates the issue entirely (not LDAP+StartTLS, but just straight LDAPS).  Once I enabled TLS I was able to hammer MyVD with 300+ inbound connections and 300 outbound connections and it worked great!


This is extra weird...

TLS is managed entirely by MINA, and I can't see how enabling something that eats CPU can make the server running faster :/

At this point, a thread dump could help...



So while I'd love to say "we'll just use LDAPS", the customer would prefer not to so I need to keep digging into this.

On Fri, Jan 20, 2023 at 1:29 PM Marc Boorshtein <[email protected] <mailto:[email protected]>> wrote:


        I would say that we only have a limited number of threads
        dedicated to
        process the incoming messages, and this number is computed based
        on the
        number of core you have on your machine.


    Is this configurable?  I'd like to be able to adjust to figure out
    if it makes an impact.


        First thing: is your server toping at 100% CPU? With Z GC it
        should not
        stop anything...


    No, not even close


        Have you profiled the server while running under your stress
        test (not
        tracing, but sampling)?


    Do you have a tool you could recommend?

        Second thing: are you sure that it occurs when the GC kick on?


    It's a guess.

        On 20/01/2023 15:57, Marc Boorshtein wrote:
         > We're using ApacheDS as a frontend for MyVD,
         > running 2.0.0.AM27-SNAPSHOT.  We're finding that under heavy
        load (~300
         > concurrent connections) we'll periodically get "broken pipe"
        errors from
         > the client.  i can reproduce this pretty easily with jmeter's
        LDAP
         > module.  The errors tend to come in bunches and when there is
        a garbage
         > collection event (under really heavy loads you can see the
        logs slow
         > down momentarily and then the errors occur.
         >
         > My test bed is a mac m2 running java 17, however the server
        is an amazon
         > m5a. running the correta java 18 jvm with the ZGC garbage
        collector.
         > here;s the JVM switches:
         >
         > -Xms4g -Xmx4g -XX:+UnlockExperimentalVMOptions -XX:+UseZGC
         > -Dsun.net.client.defaultConnectTimeout=10000
         > -Dsun.net.client.defaultReadTimeout=20000
         >
         >
         >   We're not seeing any issues in the myvd portion of the
        system, now in
         > the down stream directories being proxied.  Also, if we add an
         > artificial bottleneck by dropping the connection pool size
        from 300 to
         > 50, but still maintaining 300 clients, the issue decreases
        dramatically.
         >
         > Any thoughts as to where i can start debugging this issue?  A
        thread
         > dump analysis doesn't show any deadlocks.
         >
         > Thanks
         > Marc

-- *Emmanuel Lécharny - CTO* 205 Promenade des Anglais – 06200 NICE
        T. +33 (0)4 89 97 36 50
        P. +33 (0)6 08 33 32 61
        [email protected] <mailto:[email protected]>
        https://www.busit.com/ <https://www.busit.com/>

        ---------------------------------------------------------------------
        To unsubscribe, e-mail: [email protected]
        <mailto:[email protected]>
        For additional commands, e-mail: [email protected]
        <mailto:[email protected]>


--
*Emmanuel Lécharny - CTO* 205 Promenade des Anglais – 06200 NICE
T. +33 (0)4 89 97 36 50
P. +33 (0)6 08 33 32 61
[email protected] https://www.busit.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to