[
https://issues.apache.org/jira/browse/ARTEMIS-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17285376#comment-17285376
]
Sebastian T commented on ARTEMIS-3117:
--------------------------------------
It looks like, despite the fact that the amqp connection count on our Artemis
instance is stable, we get around 20 TCP connection attempts per second on the
amqps port 5671 from somewhere in our cloud environment. As far as I understand
this results in the same amount of initialization of a new SSL context per
second in NettyAcceptor. Since SSL context initialization is apparently
considerably slower or more expensive in JDK11 than in JDK8, we only now (after
the JDK switch) see this affecting the overall broker performance.
{noformat}
$ sudo tcpdump "tcp[tcpflags] & (tcp-syn|tcp-ack) == (tcp-syn|tcp-ack)" | pv
--line-mode --rate > /dev/null
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
[19.8 /s]
{noformat}
> Performance degradation when switching from JDK8 to JDK11
> ---------------------------------------------------------
>
> Key: ARTEMIS-3117
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3117
> Project: ActiveMQ Artemis
> Issue Type: Improvement
> Components: Broker
> Affects Versions: 2.16.0
> Environment: Amazon Linux 2, Amazon Corretto (OpenJDK 11), AMQP over
> TLS via BoringSSL
> Reporter: Sebastian T
> Priority: Major
> Attachments: broker.xml, image-2021-02-12-21-39-32-185.png,
> image-2021-02-12-21-40-21-125.png, image-2021-02-12-21-44-26-271.png,
> image-2021-02-12-21-46-52-006.png, image-2021-02-12-21-47-02-387.png,
> image-2021-02-12-21-47-57-301.png, image-2021-02-12-22-01-07-044.png
>
>
> Since it was announced that probably Artemis 2.18.0 will require Java 11 we
> upgraded the JVM of one of our broker clusters from OpenJDK 8 to OpenJDK 11
> and are seeing a noticable performance degradation which results in higher
> CPU usage and higher latency.
> We are monitoring request/reply round trip duration with a custom distributed
> qpid-jms based healthcheck applications. Here is a graphic that shows the
> effect when we switched the JDK:
> !image-2021-02-12-21-39-32-185.png!
> CPU Usage of the broker process:
> !image-2021-02-12-22-01-07-044.png|width=874,height=262!
>
> The broker itself is also monitored via Dynatrace, there I can see that after
> upgrading to JDK 11 the broker process spend 21% of CPU time locking while in
> JDK it only spent 3.2%.
> *JDK 8:*
> !image-2021-02-12-21-40-21-125.png|width=1247,height=438!
>
> *JDK 11:*
> *!image-2021-02-12-21-44-26-271.png|width=1197,height=420!*
>
> *A method hotspot breakdown reveals this:*
> !image-2021-02-12-21-47-02-387.png|width=1271,height=605!
> !image-2021-02-12-21-47-57-301.png|width=1059,height=627!
> Maybe I am misinterpreting the charts but the root cause seems to be
> somewhere in {{org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1}}
> and/or in
> {{org.apache.activemq.artemis.core.remoting.impl.netty.NettyAcceptor.getSslHandler}}
> I currently cannot pinpoint the exact line number.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
