Hi,
We have used httpcore-NIO to implement a non-blocking server and with a
recent JAVA upgrade, we are finding a high CPU usage in our servers. While
we debugged, we found out that when we close the HTTPS connection on the
server-side, the WRITE event is getting fired continuously and the
connection close is unsuccessful.
To explain the code, we have extended DefaultNHttpConnectionBase with our
own ServerConnection, and inside close() we simply just call the close()
method of the NHttpConnectionBase[1]. If we look at the code below, the
close method sets the status as CLOSING and after that either sets the
WRITE event in the session or tries to close the session (SSLIOSession in
our case).
@Override
public void close() throws IOException {
if (this.status != ACTIVE) {
return;
}
this.status = CLOSING;
if (this.outbuf.hasData()) {
this.session.setEvent(EventMask.WRITE);
} else {
this.session.close();
this.status = CLOSED;
}
}
Inside SSLIOSession close(), the updateEventMask() [2] is called.
@Override
public synchronized void close() {
if (this.status >= CLOSING) {
return;
}
this.status = CLOSING;
if (this.session.getSocketTimeout() == 0) {
this.session.setSocketTimeout(1000);
}
try {
updateEventMask();
} catch (final CancelledKeyException ex) {
shutdown();
}
}
Inside updateEventMask() method, we get the HandshakeStatus from the
JAVA SSLEngine, and set a new Mask based on the status.
switch (this.sslEngine.getHandshakeStatus()) {
case NEED_WRAP:
newMask = EventMask.READ_WRITE;
break;
case NEED_UNWRAP:
newMask = EventMask.READ;
break;
case NOT_HANDSHAKING:
newMask = this.appEventMask;
break;
case NEED_TASK:
break;
case FINISHED:
break;
}
Before JDK upgrade, when outbound is closed, we used to get the status
as NEED_UNWRAP which sets the newMask as READ mask which closes the
connection in our server perfectly.
But with a recent JDK commit[3], the new status is returned as NOT_HANDSHAKING.
Since mask is not set to READ because of the NOT_HANDSHAKING status,
WRITE event is continuously getting fired which makes the Server
connection close unsuccessful.
When the close is called multiple times for multiple connections, we
could observe that CPU usage increased continuosly and reached 100
percent and stays there continuously.
When I look at the stack trace, the execution flow was completely
inside the httpcore-nio component, which made our attempt to clear the
Event from the server implementation, unsuccessful.
[image: Screenshot 2020-10-11 at 21.53.29.png]
Earlier we were using Httpcore-NIO version 4.3.3. With this issue, we
tried upgrading to 4.4.13. But still the issue exists. The issue
exists only for HTTPS connection (where we use SSLIOSession) and not
for HTTP connections.
We observed the issue with the following JDK versions (which the
contains the commit[3]).
* Oracle JDK 1.8.0_261 (released in July 2020)
* AdoptOpenJDK (build 11.0.8+10)
* All versions after Oracle JDK 11.0.1
It would be much helpful for us, if any help can be provided to
resolve this issue.
[1] -
https://github.com/apache/httpcomponents-core/blob/rel/v4.4.13/httpcore-nio/src/main/java/org/apache/http/impl/nio/NHttpConnectionBase.java#L507
[2] -
https://github.com/apache/httpcomponents-core/blob/rel/v4.4.13/httpcore-nio/src/main/java/org/apache/http/nio/reactor/ssl/SSLIOSession.java#L642
[3] -
https://github.com/AdoptOpenJDK/openjdk-jdk11u/commit/8d1b63a4db2c6348a97b3cf45bd4d2caa7cad6b5#diff-e3c7d4bd31fed54c9607b6e0b6b2d43cL589
Thanks in Advance.
Regards,
Arunan
--
*Sugunakumar Arunan*
Software Engineer | WSO2
Email : [email protected]
Mobile : +94766016272
Web : http://wso2.com
<https://wso2.com/signature>
