Just a note to say I haven't forgotten this. I hope to look at this this
week unless someone beats me to it.
Mark
On 16/10/2019 17:55, George Stanchev wrote:
>
> On 15/10/2019 22:15, George Stanchev wrote:
>> Hi,
>>
>> I would need some help with tracking an issue with TC 8.5.47 (windows x64,
>> java: azul 1.8.0_222) configured with [1] and tcnative-1.dll. When a simple
>> client tries to connect to the server, the server hangs on SSL handshake
>> until either the client times out on read or the server times out (if I set
>> the HttpsURLConnection#setConnectTimeout(0) and ...#setReadTimeout(0)). I
>> have enabled the client side SSL trace and everything goes well until ECDH
>> key exchange (for brevity I have enabled only one TLS suite
>> "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA"). I can provide the debug logs if
>> requested. The cacerts we use is whatever comes with Azul's Java distro
>> which has ~150 entries + the custom cert for testing. The issue comes from
>> how the connector deals with trusted certs because if I reduce the cacerts
>> to contain only the test certificate, the request is being served without an
>> issue. Also if I remove the tcnative-1.dll from TC there is no issue either.
>>
>> Perhaps I am missing something. Any help is appreciated.
>
> <Mark wrote>
> This sounds like it is repeatable and that you have a system you can test
> with. On that basis here are a few things to try:
>
> 1. Take a 3 thread dumps ~5s apart of the Tomcat process when TLS handshake
> is hanging.
>
> 2. Try a binary search to try and determine if the issue is the number of
> certificates in the trust store or is caused by a specific certificate.
>
> It sounds like there might be an issue converting one or more of the trusted
> certs in the trust store to a format OpenSSL can work with.
>
> </Mark wrote>
>
> So the thread dumps didn't prove to be very useful - at least I couldn't see
> anything. I attached one of them (dump3.txt). But the trial and error in the
> binary searched proved it is not any certificate but a magic number of
> certificates in cacerts - 125 hang and 124 work. I don't know if it is a
> memory issue or buffer size issue - I tried removing and adding random certs
> around the 124/125 boundary and which one is out and in doesn't make a
> difference. I realize that a cert is rather large blob so it still could be
> memory issue with the size of the cert doesn't matter that much. In
> 8.5.47-src\java\org\apache\tomcat\util\net\openssl\OpenSSLContext.java#init(kms,
> tms, sr) we pass the DER-encoded certs to OpenSSL:
>
> // Pass along the DER encoded certificates of the accepted
> client
> // certificate issuers, so that their subjects can be
> presented
> // by the server during the handshake to allow the client
> choosing
> // an acceptable certificate
> for (X509Certificate caCert :
> x509TrustManager.getAcceptedIssuers()) {
> SSLContext.addClientCACertificateRaw(ctx,
> caCert.getEncoded());
> if (log.isDebugEnabled())
> log.debug(sm.getString("openssl.addedClientCaCert",
> caCert.toString()));
> }
>
> And this is not where it hangs. I stepped through the code through the
> handshaker but still haven't been able to figure out the hang point as I am
> not familiar with the details of that portion of TC code . I've attached two
> cacerts that you can plug in as truststore to try out the issue.
>
> George
>
>
>
>>
>> George
>>
>> [1]
>>
>> <Connector
>> port="8443" SSLEnabled="true" maxHttpHeaderSize="8192"
>> maxThreads="150" acceptCount="100" enableLookups="false"
>> disableUploadTimeout="true"
>> scheme="https" secure="true" clientAuth="true" sslProtocol="TLS"
>> sslEnabledProtocols="+TLSv1 +TLSv1.1 +TLSv1.2"
>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>> keystoreType="JKS"
>> keystoreFile="${server.conf.dir}/serena.keystore" keystorePass="changeit"
>> keyAlias="jboss" URIEncoding="UTF-8" useServerCipherSuitesOrder="true"
>> ciphers="TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384,
>> TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
>> TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
>> TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
>> TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
>> TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,
>> TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,
>> TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,
>> TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,
>> TLS_DHE_RSA_WITH_AES_256_CBC_SHA256,
>> TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256,
>> TLS_DHE_DSS_WITH_AES_128_GCM_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
>> TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,
>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
>> TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256,
>> TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_CBC_SHA256,
>> TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA,
>> TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA,
>> TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_SHA_GCM_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_SHA_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_AES_128_CCM, TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
>> TLS_ECDHE_ECDSA_WITH_AES_256_GCM_CCM_8,
>> TLS_ECDHE_ECDSA_WITH_AES_256_CCM, TLS_ECDHE_ECDSA_WITH_AES_256_SHA,
>> TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
>> TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,
>> TLS_RSA_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_RSA_WITH_CAMELLIA_256_CBC_SHA256,
>> TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
>> TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
>> TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384" />
>>
>>
>
> B‹KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB•È [œÝXœØÜšX™KK[XZ[
> ˆ \Ù\œË][œÝXœØÜšX™P ÛXØ]
> ˜\XÚK›Ü™ÃB‘›ÜˆY][Û˜[ÛÛ[X[™ ËK[XZ[
> ˆ \Ù\œËZ[ ÛXØ]
> ˜\XÚK›Ü™ÃBƒ
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
