We recently upgraded to 2.2.3 from 2.1.4 to pick up TLS 1.3 support. We ran a test and noticed that performance was quite a bit worse trying to get clients connected during an inrush of connection attempts. A thread dump showed that most of the clients were blocked waiting to acquire a lock on the SslFilter due to one of the clients holding it while doing a reverse name lookup:
at java.net.Inet4AddressImpl.getHostByAddr([email protected]/Native Method) at java.net.InetAddress$PlatformResolver.lookupByAddress([email protected]/InetAddress.java:1225) at java.net.InetAddress.getHostFromNameService([email protected]/InetAddress.java:840) at java.net.InetAddress.getHostName([email protected]/InetAddress.java:782) at java.net.InetAddress.getHostName([email protected]/InetAddress.java:754) at java.net.InetSocketAddress$InetSocketAddressHolder.getHostName([email protected]/InetSocketAddress.java:83) at java.net.InetSocketAddress.getHostName([email protected]/InetSocketAddress.java:367) at org.apache.mina.filter.ssl.SslFilter.createEngine(SslFilter.java:314) at org.apache.mina.filter.ssl.SslFilter.onConnected(SslFilter.java:278) - locked <0x000000068b8337a0> (a org.apache.mina.filter.ssl.SslFilter) at org.apache.mina.filter.ssl.SslFilter.onPostAdd(SslFilter.java:250) at org.apache.mina.core.filterchain.DefaultIoFilterChain.register(DefaultIoFilterChain.java:473) at org.apache.mina.core.filterchain.DefaultIoFilterChain.addLast(DefaultIoFilterChain.java:234) - locked <0x000000070740b0f8> (a org.apache.mina.core.filterchain.DefaultIoFilterChain) at org.apache.mina.core.filterchain.DefaultIoFilterChainBuilder.buildFilterChain(DefaultIoFilterChainBuilder.java:553) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.addNow(AbstractPollingIoProcessor.java:832) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.handleNewSessions(AbstractPollingIoProcessor.java:752) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:652) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1144) at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:642) at java.lang.Thread.runWith([email protected]/Thread.java:1596) at java.lang.Thread.run([email protected]/Thread.java:1583) It seems like the reverse name lookup is attempted while the common SslFilter lock is held whereas previously it wasn't. The clients that connect in come from the public internet so the IPs can be unpredictable and we can't rely on their name servers being responsive, so waiting on them to be done serially would take too long. Thanks!Chris
