Arunan, I confirm this is a bug in HttpCore likely affecting all versions (4.4.x, 5.0.x and 5.1.x).
Please raise a JIRA for this defect. Please try out this fix and let me know if that resolves the issue for you. https://github.com/ok2c/httpcomponents-core/commit/bf12c7dd9eabf0cd75bd6c70ac6b285b0d4430a5 Oleg On Wed, 2020-10-21 at 12:09 +0200, Oleg Kalnichevski wrote: > On Wed, 2020-10-21 at 00:25 +0530, Arunan Sugunakumar wrote: > > Hi Oleg, > > > > I was able to write a simple Test Server that reproduces this > > issue. > > I changed the NHttpFileServer example to come up with an example. > > If > > you send a POST request with a dummy payload and a > > "Connection:close" > > header to this server in 8443 port, you can see that the connection > > will not be closed (JDK11 & JDK8_261) and the CPU will be spinning > > continuously processing Write Events. If you start the same server > > in > > JDK8 (JDK8_251), the session will close normally. > > > > Please go through the example and provide your suggestions. > > > > Regards, > > Arunan > > > > Hi Arunan > > I was able to reproduce the issue with Oracle Java 11.0.8 which is a > reasonably up to date Java 11 runtime. However at the moment I do not > know whether this is a bug in Oracle's TLSv1.3 implementation or that > in HttpCore. It will likely take me a few days to complete the > analysis > and come up a fix. > > Oleg > > > > On Tue, Oct 13, 2020 at 11:07 AM Oleg Kalnichevski < > > [email protected]> > > wrote: > > > On Tue, 2020-10-13 at 08:42 +0530, Arunan Sugunakumar wrote: > > > > Hi Oleg, > > > > > > > > I upgraded to 4.4.13. Still the issue is there. > > > > > > > > > > Provide a reproducer. > > > > > > Oleg > > > > > > > Regards, > > > > Arunan > > > > > > > > On Mon, Oct 12, 2020, 20:56 Oleg Kalnichevski <[email protected] > > > > > > > > > wrote: > > > > > > > > > On Sun, 2020-10-11 at 22:16 +0530, Arunan Sugunakumar wrote: > > > > > > Hi, > > > > > > > > > > > > We have used httpcore-NIO to implement a non-blocking > > > > > > server > > > and > > > > > > with > > > > > > a recent JAVA upgrade, we are finding a high CPU usage in > > > > > > our > > > > > > servers. While we debugged, we found out that when we close > > > the > > > > > > HTTPS > > > > > > connection on the server-side, the WRITE event is getting > > > > > > fired continuously and the connection close is > > > > > > unsuccessful. > > > > > > > > > > > > To explain the code, we have extended > > > DefaultNHttpConnectionBase > > > > > > with > > > > > > our own ServerConnection, and inside close() we simply just > > > call > > > > > > the > > > > > > close() method of the NHttpConnectionBase[1]. If we look at > > > the > > > > > > code > > > > > > below, the close method sets the status as CLOSING and > > > > > > after > > > that > > > > > > either sets the WRITE event in the session or tries to > > > > > > close > > > the > > > > > > session (SSLIOSession in our case). > > > > > > > > > > > > @Override > > > > > > public void close() throws IOException { > > > > > > if (this.status != ACTIVE) { > > > > > > return; > > > > > > } > > > > > > this.status = CLOSING; > > > > > > if (this.outbuf.hasData()) { > > > > > > this.session.setEvent(EventMask.WRITE); > > > > > > } else { > > > > > > this.session.close(); > > > > > > this.status = CLOSED; > > > > > > } > > > > > > } > > > > > > > > > > > > Inside SSLIOSession close(), the updateEventMask() [2] is > > > called. > > > > > > @Override > > > > > > public synchronized void close() { > > > > > > if (this.status >= CLOSING) { > > > > > > return; > > > > > > } > > > > > > this.status = CLOSING; > > > > > > if (this.session.getSocketTimeout() == 0) { > > > > > > this.session.setSocketTimeout(1000); > > > > > > } > > > > > > try { > > > > > > updateEventMask(); > > > > > > } catch (final CancelledKeyException ex) { > > > > > > shutdown(); > > > > > > } > > > > > > } > > > > > > > > > > > > Inside updateEventMask() method, we get the HandshakeStatus > > > from > > > > > > the > > > > > > JAVA SSLEngine, and set a new Mask based on the status. > > > > > > switch (this.sslEngine.getHandshakeStatus()) { > > > > > > case NEED_WRAP: > > > > > > newMask = EventMask.READ_WRITE; > > > > > > break; > > > > > > case NEED_UNWRAP: > > > > > > newMask = EventMask.READ; > > > > > > break; > > > > > > case NOT_HANDSHAKING: > > > > > > newMask = this.appEventMask; > > > > > > break; > > > > > > case NEED_TASK: > > > > > > break; > > > > > > case FINISHED: > > > > > > break; > > > > > > } > > > > > > Before JDK upgrade, when outbound is closed, we used to get > > > the > > > > > > status as NEED_UNWRAP which sets the newMask as READ mask > > > which > > > > > > closes the connection in our server perfectly. > > > > > > But with a recent JDK commit[3], the new status is returned > > > as > > > > > > NOT_HANDSHAKING. > > > > > > Since mask is not set to READ because of the > > > > > > NOT_HANDSHAKING > > > > > > status, > > > > > > WRITE event is continuously getting fired which makes the > > > Server > > > > > > connection close unsuccessful. > > > > > > When the close is called multiple times for multiple > > > connections, > > > > > > we > > > > > > could observe that CPU usage increased continuosly and > > > reached > > > > > > 100 > > > > > > percent and stays there continuously. > > > > > > When I look at the stack trace, the execution flow was > > > completely > > > > > > inside the httpcore-nio component, which made our attempt > > > > > > to > > > > > > clear > > > > > > the Event from the server implementation, unsuccessful. > > > > > > > > > > > > > > > > > > Earlier we were using Httpcore-NIO version 4.3.3. With this > > > > > > issue, we > > > > > > tried upgrading to 4.4.13. But still the issue exists. > > > > > > > > > > I have seen no evidence supporting that. Please upgrade to > > > 4.4.13. > > > > > Oleg > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------- > > > ---- > > > > > ---- > > > > > To unsubscribe, e-mail: [email protected] > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > ----------------------------------------------------------------- > > > ---- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > ----------------------------------------------------------------- > > ---- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
