RE: TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key format must be RAW
Jose, > -Original Message- > From: Jose María Zaragoza [mailto:demablo...@gmail.com] > Sent: Wednesday, September 21, 2016 11:46 AM > To: Tomcat Users List > Subject: Re: TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key > format must be RAW > > 2016-09-21 19:16 GMT+02:00 André Warnier (tomcat) : > > On 21.09.2016 18:49, Christopher Schultz wrote: > >> > >> -BEGIN PGP SIGNED MESSAGE- > >> Hash: SHA256 > >> > >> Ron, > >> > >> On 9/21/16 11:58 AM, Roskens, Ronald wrote: > > -Original Message- From: Christopher Schultz > [mailto:ch...@christopherschultz.net] Sent: Wednesday, September > 21, 2016 9:40 AM To: Tomcat Users List Subject: Re: TLS 1.2 > Handshake on Tomcat 7.0.39 Getting Internal Error: Key format must > be RAW > > >>> > > > > Thanks also, but does this explain fully the symptoms seen by the OP ? > > As I recall, he had 3 apparently similar servers, configured > > similarly, but where > > 2 were seeing the problem and the third one not. > > Or was there another difference which he did not tell us about, and where ? > > > > > > I'd try to run > > cat /proc/sys/crypto/fips_enabled Thank you and below is the output on Production and staging: Production (CentOS 6.4): [Wed Sep 21 20:36:01 root@ip-##-###-##-##:~]$ cat /proc/sys/crypto/fips_enabled 0 Staging (CentOS 6.5): [root@stagedas cp-hosted-downloads]# cat /proc/sys/crypto/fips_enabled 0 > > > > > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org
RE: TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key format must be RAW
Ron, > -Original Message- > From: André Warnier (tomcat) [mailto:a...@ice-sa.com] > Sent: Wednesday, September 21, 2016 10:17 AM > To: users@tomcat.apache.org > Subject: Re: TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key > format must be RAW > > On 21.09.2016 18:49, Christopher Schultz wrote: > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA256 > > > > Ron, > > > > On 9/21/16 11:58 AM, Roskens, Ronald wrote: > >>> -Original Message- From: Christopher Schultz > >>> [mailto:ch...@christopherschultz.net] Sent: Wednesday, September 21, > >>> 2016 9:40 AM To: Tomcat Users List Subject: Re: TLS 1.2 Handshake on > >>> Tomcat 7.0.39 Getting Internal Error: Key format must be RAW > >>> > >> > >> > >> > >>> This may be the most promising page on the Internet, but of course > >>> Red Hat wants you to pay to read it: > >>> > >>> https://access.redhat.com/solutions/1309153 > >>> > >>> I can't see the "verified solution", or I'd reprint it here without > >>> permission :) > >> We came across the above link as well, but it requires RedHat login credentials to see the solution :( > >> The resolution says to either disable TLS 1.2 or FIPS mode. > >> > >> The root cause is the PKCS#11 implementation included in Java 7 and > >> 8 does not support TLS 1.2 when in FIPS mode as documented in OpenJDK > >> bug JDK-8029661 > >> (https://bugs.openjdk.java.net/browse/JDK-8029661) > >> > >> See also: > >> https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ > >> F > > IPS.html > > > > Thanks > >> > > for posting this. > > > > Good old FIPS: hobbling real security since 1994. > > Thank you for posting this. Will read through that posting. A quick cat on java.security on Production and staging server indicate no SunPKCS11-NSS is specified for provider #4: Production: [Wed Sep 21 20:35:54 root@ip-##-##-##-##:~]$ cat /usr/java/latest/lib/security/java.security | grep -E '^security\.provider' security.provider.1=sun.security.provider.Sun security.provider.2=sun.security.rsa.SunRsaSign security.provider.3=sun.security.ec.SunEC security.provider.4=com.sun.net.ssl.internal.ssl.Provider security.provider.5=com.sun.crypto.provider.SunJCE security.provider.6=sun.security.jgss.SunProvider security.provider.7=com.sun.security.sasl.Provider security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI security.provider.9=sun.security.smartcardio.SunPCSC Staging: [root@stagedas cp-hosted-downloads]# cat /usr/java/latest/lib/security/java.security | grep -E '^security\.provider' security.provider.1=sun.security.provider.Sun security.provider.2=sun.security.rsa.SunRsaSign security.provider.3=sun.security.ec.SunEC security.provider.4=com.sun.net.ssl.internal.ssl.Provider security.provider.5=com.sun.crypto.provider.SunJCE security.provider.6=sun.security.jgss.SunProvider security.provider.7=com.sun.security.sasl.Provider security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI security.provider.9=sun.security.smartcardio.SunPCSC > > Thanks also, but does this explain fully the symptoms seen by the OP ? As I > recall, he had 3 apparently similar servers, configured similarly, but where 2 > were seeing the problem and the third one not. > Or was there another difference which he did not tell us about, and where ? > Correct, we did make sure Tomcat and Java version are the same across all 3 servers. The CentOS version on all 3 servers are different: 6.4 (Production/AWS, TLS 1.2 works), 6.5 (Staging, no TLS 1.2), and 6.7 (Staging, no TLS 1.2) Appreciate all the help. We will continue our investigation and once resolved we will post the resolution in this forum. Thank you. > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org
RE: TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key format must be RAW
Hi André,
> -Original Message-
> From: André Warnier (tomcat) [mailto:a...@ice-sa.com]
> Sent: Tuesday, September 20, 2016 12:13 AM
> To: users@tomcat.apache.org
> Subject: Re: TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key
> format must be RAW
>
> On 20.09.2016 09:06, André Warnier (tomcat) wrote:
> > On 19.09.2016 18:45, Dono Harjanto wrote:
> >> Hi All,
> >>
> >>
> >> We have a web app deployed on 3 different servers, all running Tomcat
> >> 7.0.39 and Java 8 (update 101/102). Here is the operating system on each
> server:
> >>
> >> - Production: CentOS 6.4
> >>
> >> - Staging 1: CentOS 6.5
> >>
> >> - Staging 2: CentOS 6.7
> >>
> >>
> >
> > Java versions ?
>
> Sorry for the noise, did not read the above carefully enough.
> Are you sure they are really using the same Java version, though ?
> (/etc/alternatives and all that)
>
Result from running "ps -ef | grep tomcat" command (truncated) on all instances:
Production:
502 29119 1 2 Sep14 ?03:08:08 /usr/java/latest/bin/java
-Djava.util.logging.config.file=/var/www/tomcat/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms1024m
-Xmx20
Staging:
502 25138 1 3 Sep15 ?03:30:29 /usr/java/latest/bin/java
-Djava.util.logging.config.file=/var/www/tomcat/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms1024m
-Xmx2048m -XX:MaxPermS
The content of /usr/java/ folder which shows latest is pointing to jre1.8.0_102
instead of jre1.7.0_21.
Production:
lrwxrwxrwx. 1 root root 16 Apr 26 2013 default -> /usr/java/latest
drwxr-xr-x. 6 root root 4096 Apr 26 2013 jre1.7.0_21
drwxr-xr-x. 7 root root 4096 Aug 1 20:43 jre1.8.0_102
lrwxrwxrwx. 1 root root 22 Sep 17 00:22 latest -> /usr/java/jre1.8.0_102
Staging:
lrwxrwxrwx. 1 root root 16 Aug 14 2014 default -> /usr/java/latest
drwxr-xr-x. 9 root root 4096 Sep 7 18:53 jdk1.8.0_60
drwxr-xr-x. 6 root root 4096 Aug 14 2014 jre1.7.0_60
drwxr-xr-x. 7 root root 4096 Sep 14 21:25 jre1.8.0_102
drwxr-xr-x. 7 root root 4096 Sep 7 18:51 jre1.8.0_60
lrwxrwxrwx. 1 root root 22 Sep 14 21:55 latest -> /usr/java/jre1.8.0_102
So it's definitely using Java 8 instead of Java 7.
> >
> >
> >> When we accessed the web app on Production server, we were able to
> >> connect and connected over TLS 1.2 (as expected). However, when we
> >> accessed the web app on both Staging servers we were able to connect,
> >> but it was connected over TLS 1.1 not TLS 1.2 as TLS
> >> 1.2 handshake failed and server sent an Alert (Level: Fatal,
> >> Description: Internal
> >> Error) response.
> >>
> >>
> >> We enabled SSL debugging on Tomcat and we saw Tomcat threw
> >> InvalidAlgorithmParameterException exception in catalina.out as shown
> below:
> >>
> >>
> >> http-bio-8443-exec-1, READ: TLSv1.2 Handshake, length = 70
> >> *** ECDHClientKeyExchange
> >> ECDH Public value: { 4, 245, 39, 156, 56, 88, 62, 108, 141, 237, 93,
> >> 240, 210, 228, 91, 60, 14, 109, 138, 121, 126, 100, 36, 194, 93, 101,
> >> 131, 119, 120, 57, 120, 222, 73, 123, 122, 218, 253, 91, 170, 240,
> >> 251, 73, 214, 29, 192, 234, 109, 189, 40, 249, 161, 176, 172, 179,
> >> 36, 162, 229, 69, 160, 221, 242, 53, 100, 34, 215 } SESSION KEYGEN:
> >>
> >> PreMaster Secret:
> >> (key bytes not available)
> >> RSA master secret generation error:
> >> java.security.InvalidAlgorithmParameterException: Key format must be
> RAW
> >> at
> >> com.sun.crypto.provider.TlsMasterSecretGenerator.engineInit(TlsMaster
> >> SecretGenerator.java:67)
> >>
> >> at javax.crypto.KeyGenerator.init(KeyGenerator.java:454)
> >> at javax.crypto.KeyGenerator.init(KeyGenerator.java:430)
> >> at sun.security.ssl.Handshaker.calculateMasterSecret(Unknown
> Source)
> >> at sun.security.ssl.Handshaker.calculateKeys(Unknown Source)
> >> at sun.security.ssl.ServerHandshaker.processMessage(Unknown
> Source)
> >> at sun.security.ssl.Handshaker.processLoop(Unknown Source)
> >> at sun.security.ssl.Handshaker.process_record(Unknown Source)
> >> at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
> >> at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown
> Source)
> >> at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
> >>
RE: TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key format must be RAW
Hi Chris,
> -Original Message-
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Sent: Monday, September 19, 2016 1:24 PM
> To: Tomcat Users List
> Subject: Re: TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key
> format must be RAW
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Dono,
>
> On 9/19/16 12:45 PM, Dono Harjanto wrote:
> > Hi All,
> >
> >
> > We have a web app deployed on 3 different servers, all running Tomcat
> > 7.0.39 and Java 8 (update 101/102). Here is the operating system on
> > each server:
> >
> > - Production: CentOS 6.4
> >
> > - Staging 1: CentOS 6.5
> >
> > - Staging 2: CentOS 6.7
> >
> >
> > When we accessed the web app on Production server, we were able to
> > connect and connected over TLS 1.2 (as expected). However, when we
> > accessed the web app on both Staging servers we were able to connect,
> > but it was connected over TLS 1.1 not TLS 1.2 as TLS 1.2 handshake
> > failed and server sent an Alert (Level: Fatal,
> > Description: Internal Error) response.
> >
> >
> > We enabled SSL debugging on Tomcat and we saw Tomcat threw
> > InvalidAlgorithmParameterException exception in catalina.out as shown
> > below:
> >
> >
> > http-bio-8443-exec-1, READ: TLSv1.2 Handshake, length = 70 ***
> > ECDHClientKeyExchange ECDH Public value: { 4, 245, 39, 156, 56, 88,
> > 62, 108, 141, 237, 93, 240, 210, 228, 91, 60, 14, 109, 138, 121, 126,
> > 100, 36, 194, 93, 101, 131, 119, 120, 57, 120, 222, 73, 123, 122, 218,
> > 253, 91, 170, 240, 251, 73, 214, 29, 192, 234, 109, 189, 40, 249, 161,
> > 176, 172, 179, 36, 162, 229, 69, 160, 221, 242, 53, 100, 34, 215 }
> > SESSION KEYGEN:
> >
> > PreMaster Secret: (key bytes not available) RSA master secret
> > generation error: java.security.InvalidAlgorithmParameterException:
> > Key format must be RAW at
> > com.sun.crypto.provider.TlsMasterSecretGenerator.engineInit(TlsMasterS
> ecretGenerator.java:67)
> >
> >
> at javax.crypto.KeyGenerator.init(KeyGenerator.java:454)
> > at javax.crypto.KeyGenerator.init(KeyGenerator.java:430) at
> > sun.security.ssl.Handshaker.calculateMasterSecret(Unknown Source) at
> > sun.security.ssl.Handshaker.calculateKeys(Unknown Source) at
> > sun.security.ssl.ServerHandshaker.processMessage(Unknown Source) at
> > sun.security.ssl.Handshaker.processLoop(Unknown Source) at
> > sun.security.ssl.Handshaker.process_record(Unknown Source) at
> > sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source) at
> > sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown
> > Source) at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown
> > Source) at sun.security.ssl.SSLSocketImpl.getSession(Unknown
> > Source) at
> > org.apache.tomcat.util.net.jsse.JSSESocketFactory.handshake(JSSESocket
> Factory.java:215)
> >
> >
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.j
> ava:298)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> > Source) at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at
> > java.lang.Thread.run(Unknown Source) http-bio-8443-exec-1, handling
> > exception: java.security.ProviderException:
> > java.security.InvalidAlgorithmParameterException: Key format must be
> > RAW %% Invalidated: [Session-1,
> > TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256] http-bio-8443-exec-1,
> SEND
> > TLSv1.2 ALERT: fatal, description = internal_error
> > http-bio-8443-exec-1, WRITE: TLSv1.2 Alert, length = 2 [Raw write]:
> > length = 7 : 15 03 03 00 02 02 50
> > ..P http-bio-8443-exec-1, called closeSocket()
> > http-bio-8443-exec-1, IOException in getSession():
> > javax.net.ssl.SSLException: java.security.ProviderException:
> > java.security.InvalidAlgorithmParameterException: Key format must be
> > RAW http-bio-8443-exec-1, called close() http-bio-8443-exec-1, called
> > closeInternal(true)
> >
> >
> >
> > Below is the server.xml configuration we have on all servers:
> >
> >
> > > protocol="org.apache.coyote.http11.Http11Protocol"
> >
> > SSLEnabled="true" scheme="https" secure="true" clientAuth="false"
> > sslProtocol="TLS"
> >
> > maxHttpHeaderSize="8192" maxThreads="150" minSpareThreads="25"
> > enableLookups="false" disableUploadTimeout="true"
> > acceptCount="100" useBodyEncodingForURI="true"
>
TLS 1.2 Handshake on Tomcat 7.0.39 Getting Internal Error: Key format must be RAW
Hi All,
We have a web app deployed on 3 different servers, all running Tomcat 7.0.39
and Java 8 (update 101/102). Here is the operating system on each server:
- Production: CentOS 6.4
- Staging 1: CentOS 6.5
- Staging 2: CentOS 6.7
When we accessed the web app on Production server, we were able to connect and
connected over TLS 1.2 (as expected). However, when we accessed the web app on
both Staging servers we were able to connect, but it was connected over TLS 1.1
not TLS 1.2 as TLS 1.2 handshake failed and server sent an Alert (Level: Fatal,
Description: Internal Error) response.
We enabled SSL debugging on Tomcat and we saw Tomcat threw
InvalidAlgorithmParameterException exception in catalina.out as shown below:
http-bio-8443-exec-1, READ: TLSv1.2 Handshake, length = 70
*** ECDHClientKeyExchange
ECDH Public value: { 4, 245, 39, 156, 56, 88, 62, 108, 141, 237, 93, 240, 210,
228, 91, 60, 14, 109, 138, 121, 126, 100, 36, 194, 93, 101, 131, 119, 120, 57,
120, 222, 73, 123, 122, 218, 253, 91, 170, 240, 251, 73, 214, 29, 192, 234,
109, 189, 40, 249, 161, 176, 172, 179, 36, 162, 229, 69, 160, 221, 242, 53,
100, 34, 215 }
SESSION KEYGEN:
PreMaster Secret:
(key bytes not available)
RSA master secret generation error:
java.security.InvalidAlgorithmParameterException: Key format must be RAW
at
com.sun.crypto.provider.TlsMasterSecretGenerator.engineInit(TlsMasterSecretGenerator.java:67)
at javax.crypto.KeyGenerator.init(KeyGenerator.java:454)
at javax.crypto.KeyGenerator.init(KeyGenerator.java:430)
at sun.security.ssl.Handshaker.calculateMasterSecret(Unknown Source)
at sun.security.ssl.Handshaker.calculateKeys(Unknown Source)
at sun.security.ssl.ServerHandshaker.processMessage(Unknown Source)
at sun.security.ssl.Handshaker.processLoop(Unknown Source)
at sun.security.ssl.Handshaker.process_record(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown
Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.getSession(Unknown Source)
at
org.apache.tomcat.util.net.jsse.JSSESocketFactory.handshake(JSSESocketFactory.java:215)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
http-bio-8443-exec-1, handling exception: java.security.ProviderException:
java.security.InvalidAlgorithmParameterException: Key format must be RAW
%% Invalidated: [Session-1, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256]
http-bio-8443-exec-1, SEND TLSv1.2 ALERT: fatal, description = internal_error
http-bio-8443-exec-1, WRITE: TLSv1.2 Alert, length = 2
[Raw write]: length = 7
: 15 03 03 00 02 02 50 ..P
http-bio-8443-exec-1, called closeSocket()
http-bio-8443-exec-1, IOException in getSession(): javax.net.ssl.SSLException:
java.security.ProviderException:
java.security.InvalidAlgorithmParameterException: Key format must be RAW
http-bio-8443-exec-1, called close()
http-bio-8443-exec-1, called closeInternal(true)
Below is the server.xml configuration we have on all servers:
Any idea why Tomcat not able to do TLS 1.2 handshake and throwing "Key format
must be RAW" exception? Did we miss anything here?
Thanks for your help,
Don
