[
https://issues.apache.org/jira/browse/HTTPCORE-764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruce Hamilton updated HTTPCORE-764:
------------------------------------
Description:
Using the apache http async client 4.1.5, we are making calls to the Open AI
server-side streaming endpoint. After some period of making requests (an hour
or so), the service will hit 100% CPU and invariably the client will invariably
be stuck in the same spot in the SSLIOSession class, trying to complete the
handshake.
We believe the server is dropping connections during the TLS handshake, which
manifests as connection timeouts in other clients. This seems to happen for
both TLS 1.2 and 1.3, and I can see there's been some closely related issues in
the past regarding this loop.
Here is the stack trace from the thread dump:
"Ktor-client-apache (server): llm-" #53 daemon prio=5 os_prio=0
cpu=20726438.17ms elapsed=62011.90s tid=0x00007f7f28074010 nid=0x41 runnable
[0x00007f7e934fd000]
java.lang.Thread.State: RUNNABLE
at
sun.security.ssl.SSLEngineImpl.wrap([email protected]/SSLEngineImpl.java:123)
at javax.net.ssl.SSLEngine.wrap([email protected]/SSLEngine.java:524)
at
org.apache.http.nio.reactor.ssl.SSLIOSession.doWrap(SSLIOSession.java:270)
at
org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:316)
at
org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:547)
- locked <0x00000000f7e699b0> (a
org.apache.http.nio.reactor.ssl.SSLIOSession)
at
org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
at java.lang.Thread.run([email protected]/Thread.java:833)
was:
Using the apache http async client 4.1.5, we are making calls to the Open AI
server-side streaming endpoint. After some period of making requests (an hour
or so), the service will hit 100% CPU and invariably the client will invariably
be stuck in the same spot in the SSLIOSession class, trying to complete the
handshake.
We believe the server is dropping connections during the TLS handshake, which
manifests as connection timeouts in other clients. This seems to happen for
both TLS 1.2 and 1.3, and I can see there's been some closely related issues in
the past regarding this loop.
Here is the stack trace from the thread dump:
"Ktor-client-apache (server): llm-" #53 daemon prio=5 os_prio=0
cpu=20726438.17ms elapsed=62011.90s tid=0x00007f7f28074010 nid=0x41 runnable
[0x00007f7e934fd000]
java.lang.Thread.State: RUNNABLE
at sun.security.ssl.SSLEngineImpl.wrap([email protected]/SSLEngineImpl.java:123)
at javax.net.ssl.SSLEngine.wrap([email protected]/SSLEngine.java:524)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doWrap(SSLIOSession.java:270)
at
org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:316)
at
org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:547)
- locked <0x00000000f7e699b0> (a org.apache.http.nio.reactor.ssl.SSLIOSession)
at
org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
at java.lang.Thread.run([email protected]/Thread.java:833)
> Apparent infinite loop in SSLIOSession
> --------------------------------------
>
> Key: HTTPCORE-764
> URL: https://issues.apache.org/jira/browse/HTTPCORE-764
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore NIO
> Affects Versions: 4.4.15
> Reporter: Bruce Hamilton
> Priority: Major
>
> Using the apache http async client 4.1.5, we are making calls to the Open AI
> server-side streaming endpoint. After some period of making requests (an
> hour or so), the service will hit 100% CPU and invariably the client will
> invariably be stuck in the same spot in the SSLIOSession class, trying to
> complete the handshake.
> We believe the server is dropping connections during the TLS handshake, which
> manifests as connection timeouts in other clients. This seems to happen for
> both TLS 1.2 and 1.3, and I can see there's been some closely related issues
> in the past regarding this loop.
>
> Here is the stack trace from the thread dump:
> "Ktor-client-apache (server): llm-" #53 daemon prio=5 os_prio=0
> cpu=20726438.17ms elapsed=62011.90s tid=0x00007f7f28074010 nid=0x41 runnable
> [0x00007f7e934fd000]
> java.lang.Thread.State: RUNNABLE
> at
> sun.security.ssl.SSLEngineImpl.wrap([email protected]/SSLEngineImpl.java:123)
> at javax.net.ssl.SSLEngine.wrap([email protected]/SSLEngine.java:524)
> at
> org.apache.http.nio.reactor.ssl.SSLIOSession.doWrap(SSLIOSession.java:270)
> at
> org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:316)
> at
> org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:547)
> - locked <0x00000000f7e699b0> (a
> org.apache.http.nio.reactor.ssl.SSLIOSession)
> at
> org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
> at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
> at java.lang.Thread.run([email protected]/Thread.java:833)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
