Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Aurélien Terrestris]

George,

I'm not sure we can find any solution, but can we have a look at a pcap
capture ?
Esmond Pitt was posting sometimes, that would be a challenge for him.

2015-10-15 4:33 GMT+02:00 Christopher Schultz 
:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Aurélien,
>
> On 10/14/15 5:59 PM, Aurélien Terrestris wrote:
> > Still no solutions, I suppose..
> >
> > Did you enable the SSLv2 Hello as suggested by Chris, and what's
> > the result ? I tested a small client with Java 8, by adding
> > -Djdk.tls.client.protocols="SSLv2Hello,TLSv1.2" at the command
> > line, and I get my SSLv2 Hello.
>
> It looks like if you add SSLv2Hello to the list of protocols you'll
> accept, you'll get an SSLv2Hello in there (abridged output):
>
> Allow unsafe renegotiation: false
> Allow legacy hello messages: true
> Is initial handshake: true
> Is secure renegotiation: false
> ...
> main, WRITE: TLSv1.2 Handshake, length = 221
> main, WRITE: SSLv2 client hello message, length = 140
> main, READ: TLSv1.2 Handshake, length = 81
> main, READ: TLSv1.2 Handshake, length = 2779
> main, READ: TLSv1.2 Handshake, length = 589
> main, READ: TLSv1.2 Handshake, length = 4
> main, WRITE: TLSv1.2 Handshake, length = 70
> main, WRITE: TLSv1.2 Change Cipher Spec, length = 1
> main, WRITE: TLSv1.2 Handshake, length = 40
> main, READ: TLSv1.2 Change Cipher Spec, length = 1
> main, READ: TLSv1.2 Handshake, length = 40
>
> You just have to use a custom SSLSocketFactory that sets the protocols
> you want to enable on the (client) socket. If one of the protocols you
> use is "SSLv2Hello".
>
> Oddly enough, when *not* specifying SSLv2Hello, you'll get this
> (abridged output):
>
> Allow unsafe renegotiation: false
> Allow legacy hello messages: true
> Is initial handshake: true
> Is secure renegotiation: false
> ...
> main, WRITE: TLSv1.2 Handshake, length = 221
> main, READ: TLSv1.2 Handshake, length = 89
> main, READ: TLSv1.2 Handshake, length = 2779
> main, READ: TLSv1.2 Handshake, length = 589
> main, READ: TLSv1.2 Handshake, length = 4
> main, WRITE: TLSv1.2 Handshake, length = 70
> main, WRITE: TLSv1.2 Change Cipher Spec, length = 1
> main, WRITE: TLSv1.2 Handshake, length = 40
> main, READ: TLSv1.2 Change Cipher Spec, length = 1
> main, READ: TLSv1.2 Handshake, length = 40
>
> When the SSLv2Hello "protocol" isn't enabled, you don't get the "main,
> WRITE" and "main, READ"
>
> Note that I'm not trying anything with a client certificate, here. I
> hope that helps somewhat.
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
>
> iQIcBAEBCAAGBQJWHxCIAAoJEBzwKT+lPKRYCNQQAMJx3cHj3Rl8ieX+2cANmXfW
> fHr0MPkHNIcbzpX5WWJaEqfhnYqQTk9TiY7rKxwjo3OtJtEG1bkm9tqeq4pzHJcX
> oQ03/wMOKrNqqGoILcpdWgRpc0jylsx1GouJ2qmmCNvZO1fBdBhtAE49dvg4Hd+c
> uOzet5CizkTIfbu/i2Rb/szC9T/mopvicOsoS7oe1EE7sJZKL4BU3ayun5KvFXvr
> 0KbDRU0Btp3M0YcPP4R2MtExYROW9pwwb5UYJdmK8ZxHAsmhJsG8DzDQnywFEx3+
> cm2e0W5v5FMAAh3PBNqfl5VN/8uIlHkeLtCjDU0JCMCfguwTQbitPpyhatnRlE7z
> K8FfdZUC2zBprX1HnJl5aT02u3STzRsyL5DWlVAKPC/OAUEYFO26Ira1K86ACpww
> O7t6phwHfXdGIkT/GdT9i2DgGippj6/mAhgq6XUsAkVr9usK33pNP8q/jf/ORwq/
> Njf4d4vjRNw3W7UZ0w0NCgZ7dKdepC/x2sT6zugQugiLNQ+gHGQWfcOhrQsRsj8f
> qHGU1E+94g5oQCqb14KWoZv8bAA2WYAqgUK3DK2icsiCEFqWd6Yb6gYcvIGsbV9t
> g+Mtxfm5qjncCwHeyONd3uBWTjakZb7fIvk4di0pZcnZB7HFYx7/r0ndS+IRzUVS
> LJxWiHhKQZ32QvVKtBxe
> =zKZ4
> -END PGP SIGNATURE-
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

Aurélien,

I added good_run.pcap and bad_run.pcap to that dropbox location [1]. 

I also think this needs to be looked at by MS engineers. I am following up on 
my support case but really not getting anywhere...

George

[1] https://www.dropbox.com/sh/az1r3agxx4w8r7e/AACRGedBG3G5oh4-qE9652WNa?dl=0
-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com] 
Sent: Thursday, October 15, 2015 7:43 AM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

George,

I'm not sure we can find any solution, but can we have a look at a pcap capture 
?
Esmond Pitt was posting sometimes, that would be a challenge for him.

2015-10-15 4:33 GMT+02:00 Christopher Schultz <ch...@christopherschultz.net>
:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Aurélien,
>
> On 10/14/15 5:59 PM, Aurélien Terrestris wrote:
> > Still no solutions, I suppose..
> >
> > Did you enable the SSLv2 Hello as suggested by Chris, and what's the 
> > result ? I tested a small client with Java 8, by adding 
> > -Djdk.tls.client.protocols="SSLv2Hello,TLSv1.2" at the command line, 
> > and I get my SSLv2 Hello.
>
> It looks like if you add SSLv2Hello to the list of protocols you'll 
> accept, you'll get an SSLv2Hello in there (abridged output):
>
> Allow unsafe renegotiation: false
> Allow legacy hello messages: true
> Is initial handshake: true
> Is secure renegotiation: false
> ...
> main, WRITE: TLSv1.2 Handshake, length = 221 main, WRITE: SSLv2 client 
> hello message, length = 140 main, READ: TLSv1.2 Handshake, length = 81 
> main, READ: TLSv1.2 Handshake, length = 2779 main, READ: TLSv1.2 
> Handshake, length = 589 main, READ: TLSv1.2 Handshake, length = 4 
> main, WRITE: TLSv1.2 Handshake, length = 70 main, WRITE: TLSv1.2 
> Change Cipher Spec, length = 1 main, WRITE: TLSv1.2 Handshake, length 
> = 40 main, READ: TLSv1.2 Change Cipher Spec, length = 1 main, READ: 
> TLSv1.2 Handshake, length = 40
>
> You just have to use a custom SSLSocketFactory that sets the protocols 
> you want to enable on the (client) socket. If one of the protocols you 
> use is "SSLv2Hello".
>
> Oddly enough, when *not* specifying SSLv2Hello, you'll get this 
> (abridged output):
>
> Allow unsafe renegotiation: false
> Allow legacy hello messages: true
> Is initial handshake: true
> Is secure renegotiation: false
> ...
> main, WRITE: TLSv1.2 Handshake, length = 221 main, READ: TLSv1.2 
> Handshake, length = 89 main, READ: TLSv1.2 Handshake, length = 2779 
> main, READ: TLSv1.2 Handshake, length = 589 main, READ: TLSv1.2 
> Handshake, length = 4 main, WRITE: TLSv1.2 Handshake, length = 70 
> main, WRITE: TLSv1.2 Change Cipher Spec, length = 1 main, WRITE: 
> TLSv1.2 Handshake, length = 40 main, READ: TLSv1.2 Change Cipher Spec, 
> length = 1 main, READ: TLSv1.2 Handshake, length = 40
>
> When the SSLv2Hello "protocol" isn't enabled, you don't get the "main, 
> WRITE" and "main, READ"
>
> Note that I'm not trying anything with a client certificate, here. I 
> hope that helps somewhat.
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
>
> iQIcBAEBCAAGBQJWHxCIAAoJEBzwKT+lPKRYCNQQAMJx3cHj3Rl8ieX+2cANmXfW
> fHr0MPkHNIcbzpX5WWJaEqfhnYqQTk9TiY7rKxwjo3OtJtEG1bkm9tqeq4pzHJcX
> oQ03/wMOKrNqqGoILcpdWgRpc0jylsx1GouJ2qmmCNvZO1fBdBhtAE49dvg4Hd+c
> uOzet5CizkTIfbu/i2Rb/szC9T/mopvicOsoS7oe1EE7sJZKL4BU3ayun5KvFXvr
> 0KbDRU0Btp3M0YcPP4R2MtExYROW9pwwb5UYJdmK8ZxHAsmhJsG8DzDQnywFEx3+
> cm2e0W5v5FMAAh3PBNqfl5VN/8uIlHkeLtCjDU0JCMCfguwTQbitPpyhatnRlE7z
> K8FfdZUC2zBprX1HnJl5aT02u3STzRsyL5DWlVAKPC/OAUEYFO26Ira1K86ACpww
> O7t6phwHfXdGIkT/GdT9i2DgGippj6/mAhgq6XUsAkVr9usK33pNP8q/jf/ORwq/
> Njf4d4vjRNw3W7UZ0w0NCgZ7dKdepC/x2sT6zugQugiLNQ+gHGQWfcOhrQsRsj8f
> qHGU1E+94g5oQCqb14KWoZv8bAA2WYAqgUK3DK2icsiCEFqWd6Yb6gYcvIGsbV9t
> g+Mtxfm5qjncCwHeyONd3uBWTjakZb7fIvk4di0pZcnZB7HFYx7/r0ndS+IRzUVS
> LJxWiHhKQZ32QvVKtBxe
> =zKZ4
> -END PGP SIGNATURE-
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Aurélien Terrestris]

Still no solutions, I suppose..

Did you enable the SSLv2 Hello as suggested by Chris, and what's the result
? I tested a small client with Java 8, by adding
-Djdk.tls.client.protocols="SSLv2Hello,TLSv1.2" at the command line, and I
get my SSLv2 Hello.

>From the TLS 1.2 RFC ( https://www.ietf.org/rfc/rfc5246.txt ) we know we
must expect some problems like yours (Appendix E.1 & E.2). I'm not saying
it's that kind of things for sure, but this is what they suggest :

"Interoperability with such buggy servers is a complex topic beyond the
scope of this document, and may require multiple connection attempts by the
client."

And that's what HttpUrl class does, a retry.


I am not sure that we can blindly trust the JSSE Ref Guide , but it's
saying that the renegotiation (for your client cert) has to happen after
some data was already been sent to each other. My question is : are you
sure you are not doing 2 handshakes without sending data between them ?

See :
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization
*"Encrypted data*
The client and the server communicate using the symmetric encryption
algorithm and the cryptographic hash function negotiated during the client
hello and server hello, and using the secret key that the client sent to
the server during the client key exchange. The handshake can be
renegotiated at this time. See the next section for details."






2015-10-14 4:56 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> So this time I spoke too soon. I was relying on Java to dump in its debug
> trace. Running rawcap/wireshark does reveal that Java indeed wraps the
> TLSv1.2 hello payload in SSLv2Hello record envelope which that legacy
> protocol enables. The Java piece of code that I quoted was for the content
> of the ClientHello record, not how it is wrapped which happens later when
> the record is being serialized to the socket...
>
> Anyways, thanks to all for the tip but it doesn't make a
> difference...still bad mac record...
>
> George
>
> -Original Message-
> From: George Stanchev [mailto:gstanc...@serena.com]
> Sent: Tuesday, October 13, 2015 3:56 PM
> To: Tomcat Users List
> Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
> description = bad_record_mac
>
> May be I am mistaken. I will give jtouch a try, thanks for the
> pointers...at this point I am grasping at straws :)
>
> Thanks Aurelien!
>
> -Original Message-
> From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
> Sent: Tuesday, October 13, 2015 3:52 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
> description = bad_record_mac
>
> "Another route to try the SSLv2Hello is to hack and recompile the JSSE.
> It's on my agenda to learn to do that if possible without rebuilding the
> whole JRE since for some obscure reason Oracle compiles JSSE with no debug
> info (and it is not included in the srcs distributed with the JRE) on it
> which makes debugging and inspecting local vars really hard..."
>
> No, it's not needed. I use jtouch.sourceforge.net and it's working well
> for trying SSLv2 hello's. From the network captures, I see that it's
> working.
> Not making advertisement for my software here, but,.. ;)
>
>
>
>
>
>
>
> 2015-10-13 23:20 GMT+02:00 George Stanchev <gstanc...@serena.com>:
>
> > Just as a side note, https.protocols is read by HttpsUrlConnection
> > which feeds it down through setEnabledProtocols() on the SSL socket. "
> > jdk.tls.client.protocols" is read directly by JSSE and does the same
> > thing...
> >
> > Another route to try the SSLv2Hello is to hack and recompile the JSSE.
> > It's on my agenda to learn to do that if possible without rebuilding
> > the whole JRE since for some obscure reason Oracle compiles JSSE with
> > no debug info (and it is not included in the srcs distributed with the
> > JRE) on it which makes debugging and inspecting local vars really hard...
> >
> > George
> >
> > -Original Message-
> > From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
> > Sent: Tuesday, October 13, 2015 3:13 PM
> > To: Tomcat Users List
> > Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
> > description = bad_record_mac
> >
> > Maybe writing too fast..
> >
> > "How do you force Java 8 to use SSLv2Hello?"
> >
> > As suggested before, by writing your own client OR by trying this hack.
> > From the JSSE Ref Guide 5, we know how to the customize the protocol (
> > https://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefG
> > uide.h

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

Yeah it's in my msg below. Java logs it sequentially and I was looking at where 
the payload was created, not wher it was serialized [1]. I realized my mistake 
when I took a pcap and then had a closer look at the logs. I also know how this 
is handled in the JSSE code - the hello wrapper is handled by I/O layer, not by 
the handshaker...


I don't think https://www.ietf.org/rfc/rfc5246.txt 's Appendices E.1 and E.2 
refer to my situation. They talk about initial handshakes, which in my case go 
just fine.

Regarding the last remark, about 2 handshakes without data inbetween - it's not 
like I am handling raw sockets. In one case the clinet is HttpsUrlConnection, 
the other is the Apache HttpClient 3. Also, since I have put together an 
SChannel server I can see the data in between the handshakes making it 
through...

George


[1] 

*** ClientHello, TLSv1 <* I was looking here
RandomCookie:  
Session ID:  {}
Cipher Suites: [TLS_RSA_WITH_AES_256_CBC_SHA]
Compression Methods:  { 0 }
Extension renegotiation_info, renegotiated_connection: 
***
[write] MD5 and SHA1 hashes:  len = 52
...
main, WRITE: TLSv1 Handshake, length = 52
[write] MD5 and SHA1 hashes:  len = 47
...
main, WRITE: SSLv2 client hello message, length = 47  <* 
But it went out here where it got serialized
[Raw write]: length = 49
... 
[Raw read]: length = 5
: 16 03 01 02 E7 .
[Raw read]: length = 743
...main, READ: TLSv1 Handshake, length = 743
*** ServerHello, TLSv1

-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com] 
Sent: Wednesday, October 14, 2015 3:59 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

Still no solutions, I suppose..

Did you enable the SSLv2 Hello as suggested by Chris, and what's the result ? I 
tested a small client with Java 8, by adding 
-Djdk.tls.client.protocols="SSLv2Hello,TLSv1.2" at the command line, and I get 
my SSLv2 Hello.

From the TLS 1.2 RFC ( https://www.ietf.org/rfc/rfc5246.txt ) we know we must 
expect some problems like yours (Appendix E.1 & E.2). I'm not saying it's that 
kind of things for sure, but this is what they suggest :

"Interoperability with such buggy servers is a complex topic beyond the scope 
of this document, and may require multiple connection attempts by the client."

And that's what HttpUrl class does, a retry.


I am not sure that we can blindly trust the JSSE Ref Guide , but it's saying 
that the renegotiation (for your client cert) has to happen after some data was 
already been sent to each other. My question is : are you sure you are not 
doing 2 handshakes without sending data between them ?

See :
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#InstallationAndCustomization
*"Encrypted data*
The client and the server communicate using the symmetric encryption algorithm 
and the cryptographic hash function negotiated during the client hello and 
server hello, and using the secret key that the client sent to the server 
during the client key exchange. The handshake can be renegotiated at this time. 
See the next section for details."






2015-10-14 4:56 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> So this time I spoke too soon. I was relying on Java to dump in its 
> debug trace. Running rawcap/wireshark does reveal that Java indeed 
> wraps the
> TLSv1.2 hello payload in SSLv2Hello record envelope which that legacy 
> protocol enables. The Java piece of code that I quoted was for the 
> content of the ClientHello record, not how it is wrapped which happens 
> later when the record is being serialized to the socket...
>
> Anyways, thanks to all for the tip but it doesn't make a 
> difference...still bad mac record...
>
> George
>
> -Original Message-
> From: George Stanchev [mailto:gstanc...@serena.com]
> Sent: Tuesday, October 13, 2015 3:56 PM
> To: Tomcat Users List
> Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
> description = bad_record_mac
>
> May be I am mistaken. I will give jtouch a try, thanks for the 
> pointers...at this point I am grasping at straws :)
>
> Thanks Aurelien!
>
> -Original Message-
> From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
> Sent: Tuesday, October 13, 2015 3:52 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
> description = bad_record_mac
>
> "Another route to try the SSLv2Hello is to hack and recompile the JSSE.
> It's on my agenda to learn to do that if possible without rebuilding 
> the whole JRE since for some obscure reason Oracle compiles JSSE with 
> no debug info (and it is not included in the srcs distributed with the 
> JRE) on it which makes

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Aurélien,

On 10/14/15 5:59 PM, Aurélien Terrestris wrote:
> Still no solutions, I suppose..
> 
> Did you enable the SSLv2 Hello as suggested by Chris, and what's
> the result ? I tested a small client with Java 8, by adding 
> -Djdk.tls.client.protocols="SSLv2Hello,TLSv1.2" at the command
> line, and I get my SSLv2 Hello.

It looks like if you add SSLv2Hello to the list of protocols you'll
accept, you'll get an SSLv2Hello in there (abridged output):

Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
...
main, WRITE: TLSv1.2 Handshake, length = 221
main, WRITE: SSLv2 client hello message, length = 140
main, READ: TLSv1.2 Handshake, length = 81
main, READ: TLSv1.2 Handshake, length = 2779
main, READ: TLSv1.2 Handshake, length = 589
main, READ: TLSv1.2 Handshake, length = 4
main, WRITE: TLSv1.2 Handshake, length = 70
main, WRITE: TLSv1.2 Change Cipher Spec, length = 1
main, WRITE: TLSv1.2 Handshake, length = 40
main, READ: TLSv1.2 Change Cipher Spec, length = 1
main, READ: TLSv1.2 Handshake, length = 40

You just have to use a custom SSLSocketFactory that sets the protocols
you want to enable on the (client) socket. If one of the protocols you
use is "SSLv2Hello".

Oddly enough, when *not* specifying SSLv2Hello, you'll get this
(abridged output):

Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
...
main, WRITE: TLSv1.2 Handshake, length = 221
main, READ: TLSv1.2 Handshake, length = 89
main, READ: TLSv1.2 Handshake, length = 2779
main, READ: TLSv1.2 Handshake, length = 589
main, READ: TLSv1.2 Handshake, length = 4
main, WRITE: TLSv1.2 Handshake, length = 70
main, WRITE: TLSv1.2 Change Cipher Spec, length = 1
main, WRITE: TLSv1.2 Handshake, length = 40
main, READ: TLSv1.2 Change Cipher Spec, length = 1
main, READ: TLSv1.2 Handshake, length = 40

When the SSLv2Hello "protocol" isn't enabled, you don't get the "main,
WRITE" and "main, READ"

Note that I'm not trying anything with a client certificate, here. I
hope that helps somewhat.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJWHxCIAAoJEBzwKT+lPKRYCNQQAMJx3cHj3Rl8ieX+2cANmXfW
fHr0MPkHNIcbzpX5WWJaEqfhnYqQTk9TiY7rKxwjo3OtJtEG1bkm9tqeq4pzHJcX
oQ03/wMOKrNqqGoILcpdWgRpc0jylsx1GouJ2qmmCNvZO1fBdBhtAE49dvg4Hd+c
uOzet5CizkTIfbu/i2Rb/szC9T/mopvicOsoS7oe1EE7sJZKL4BU3ayun5KvFXvr
0KbDRU0Btp3M0YcPP4R2MtExYROW9pwwb5UYJdmK8ZxHAsmhJsG8DzDQnywFEx3+
cm2e0W5v5FMAAh3PBNqfl5VN/8uIlHkeLtCjDU0JCMCfguwTQbitPpyhatnRlE7z
K8FfdZUC2zBprX1HnJl5aT02u3STzRsyL5DWlVAKPC/OAUEYFO26Ira1K86ACpww
O7t6phwHfXdGIkT/GdT9i2DgGippj6/mAhgq6XUsAkVr9usK33pNP8q/jf/ORwq/
Njf4d4vjRNw3W7UZ0w0NCgZ7dKdepC/x2sT6zugQugiLNQ+gHGQWfcOhrQsRsj8f
qHGU1E+94g5oQCqb14KWoZv8bAA2WYAqgUK3DK2icsiCEFqWd6Yb6gYcvIGsbV9t
g+Mtxfm5qjncCwHeyONd3uBWTjakZb7fIvk4di0pZcnZB7HFYx7/r0ndS+IRzUVS
LJxWiHhKQZ32QvVKtBxe
=zKZ4
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

So this time I spoke too soon. I was relying on Java to dump in its debug 
trace. Running rawcap/wireshark does reveal that Java indeed wraps the TLSv1.2 
hello payload in SSLv2Hello record envelope which that legacy protocol enables. 
The Java piece of code that I quoted was for the content of the ClientHello 
record, not how it is wrapped which happens later when the record is being 
serialized to the socket...

Anyways, thanks to all for the tip but it doesn't make a difference...still bad 
mac record...

George

-Original Message-
From: George Stanchev [mailto:gstanc...@serena.com] 
Sent: Tuesday, October 13, 2015 3:56 PM
To: Tomcat Users List
Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

May be I am mistaken. I will give jtouch a try, thanks for the pointers...at 
this point I am grasping at straws :)

Thanks Aurelien!

-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
Sent: Tuesday, October 13, 2015 3:52 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

"Another route to try the SSLv2Hello is to hack and recompile the JSSE.
It's on my agenda to learn to do that if possible without rebuilding the whole 
JRE since for some obscure reason Oracle compiles JSSE with no debug info (and 
it is not included in the srcs distributed with the JRE) on it which makes 
debugging and inspecting local vars really hard..."

No, it's not needed. I use jtouch.sourceforge.net and it's working well for 
trying SSLv2 hello's. From the network captures, I see that it's working.
Not making advertisement for my software here, but,.. ;)







2015-10-13 23:20 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> Just as a side note, https.protocols is read by HttpsUrlConnection 
> which feeds it down through setEnabledProtocols() on the SSL socket. "
> jdk.tls.client.protocols" is read directly by JSSE and does the same 
> thing...
>
> Another route to try the SSLv2Hello is to hack and recompile the JSSE.
> It's on my agenda to learn to do that if possible without rebuilding 
> the whole JRE since for some obscure reason Oracle compiles JSSE with 
> no debug info (and it is not included in the srcs distributed with the
> JRE) on it which makes debugging and inspecting local vars really hard...
>
> George
>
> -Original Message-
> From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
> Sent: Tuesday, October 13, 2015 3:13 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
> description = bad_record_mac
>
> Maybe writing too fast..
>
> "How do you force Java 8 to use SSLv2Hello?"
>
> As suggested before, by writing your own client OR by trying this hack.
> From the JSSE Ref Guide 5, we know how to the customize the protocol ( 
> https://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefG
> uide.html#InstallationAndCustomization
> ) by setting a system property (https.protocol). Although they are no 
> more talking about this in the Ref Guide 8 ( 
> https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunPro
> viders.html#SunJSSEProvider
> ) I would give it a try as I know that the documentation is poorly written.
> I suggested 10 years ago a change in the API doc about the enabled 
> protocols, and they didn't change anything although they said they would.
>
>
>
> "[1] states: " JDK 7-9 enables SSLv2Hello on the server side only.  
> (Will not send, but will accept SSLv2Hellos)""
>
> I believe they mean "by default" as for the client side. Poorly 
> written, probably.
>
>
>
> 2015-10-13 22:55 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
>
> > "How do you force Java 8 to use SSLv2Hello?"
> >
> > You can do this when writing your own Java client : calling the 
> > SSLSocketFactory to create an SSLSocket and configure with 
> > setEnabledProtocols ( 
> > https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.ht
> > ml
> > #setEnabledProtocols-java.lang.String:A-
> > )
> >
> > If you have some IIS server on internet which reproduces the 
> > problem, I'll try with JTouch ( jtouch.sourceforge.net ) or write a small 
> > client.
> >
> >
> >
> >
> > 2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
> >
> >> George,
> >>
> >> do you have any network capture that we can see ?
> >>
> >> 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
> >>
> >>> >> It might be doable with OpenSSL s_client or something. Tough t

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

George,

On 10/9/15 12:46 PM, George Stanchev wrote:
> One more clarification: on point [6] below I stated that Java is
> able to recover with a retry on a cached connection. Unfortunately
> that is only valid for higher level classes like HttpUrlConnection
> which makes 1 retry on IOException (and only on a GET and some
> other conditions). The lower-level sockets just throw and that’s
> it...

Can you confirm whether using SSLv2Hello also solves the problem? Or
do you have a requirement that nothing less than TLSv1 may even
handshake properly?

- -chris

> -Original Message- From: George Stanchev
> [mailto:gstanc...@serena.com] Sent: Friday, October 09, 2015 10:40
> AM To: Tomcat Users List Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67:
> SEND TLSv1 ALERT: fatal, description = bad_record_mac
> 
> Just for the record, https.protocols is a property used by the
> HttpsUrlConnection class. If your app is using a client that
> doesn't rely on the internal Oracle HTTP client, it's better to use
> " jdk.tls.client.protocols" which is read directly by the
> socket/SSL classes. Apache Http Client is one client that does use
> sockets directly instead of HttpsUrlConnection.
> 
> 
> Also, I want to mention that we have finally been able to isolate a
> really nasty problem with Java SSL implementation and Windows's
> SSPI based SChannel acting as a server. The issue we ran into is
> very similar as to what the OP reported, that’s why I asked him a
> while ago on a separate thread about keystores if the server is
> IIS-based. The issue that we ran into might sound esoteric but our
> product ran into it very often. We have a configuration where we
> lock our intracomponent communication via 2way SSL. We have Java
> Tomcat-based server components that talk to IIS based native
> components via web services over HTTP which we secure with 2way
> SSL. When we upgraded to JRE 8 we started getting those connection
> resets in Tomcat when we enabled the 2way SSL. Those were on the
> Java->IIS calls that work over regular SSL. On the java side, if
> you turn on the ssl debug on, you'd see connection reset
> IOException thrown, and on the Windows side, you'd see " The
> following fatal alert was generated: 20. The internal error state
> is 960." In the System event log. This error is "
> TLS1_ALERT_BAD_RECORD_MAC" from SSPI. A search in Oracle's bug
> database found a close resemblance to this issue:
> https://bugs.openjdk.java.net/browse/JDK-8068812. But the reporter
> couldn't reproduce it and also was against another Oracle component
> Java Web Download or something like this. So we started working
> with Oracle but the intermittent and hard-to-reproduce nature of
> this issue hampered the investigation. As the OP mentioned below,
> there are apocryphal reports for this all over the web but people
> cannot troubleshoot it very well. Anyway here are my findings. In
> order for the issue to arise all those conditions must be met:
> 
> [1] The client must start with ClientHello TLSv1.2 [2] The server
> must respond with ServerHello TLSv1 (so the server should not have
> TLS1.2 enabled) [3] The cipher key exchange must NOT be ECDHE. I
> have not tried DHE as I could not make Windows take it. According
> to this doc [1] it should have several DHE key exchange ciphers
> available but I could not make it work. RSA key exchange exhibits
> the issue. I have been using “TLS_RSA_WITH_AES_256_CBC_SHA” to
> reproduce. [4] The initial connection must not be 2way-SSL [5] The
> server must upgrade the connection after encrypted payload is read.
> In case of IIS, it has 2way SSL setting on a Virtual Directory. The
> initial handshake is not 2way SSL, once the HTTP request is read,
> it determines that the requested resource (/gsoap) is 2way
> SSL-protected and upgrades the connection to 2way SSL. [6] The
> server must kill the connection abruptly when error happens. This
> is important as if the server terminates the connection gracefully
> and keeps it around Java is able to recover since it reuses the
> cached connection. IIS does exactly this. [7] The server must be
> Windows 7, Win2k8, Win2k8-R2, Win8. Might be other versions
> affected but bug is not evident on Win2k12 or Win2k12-R2 or Win10.
> Perhaps it was fixed, perhaps those versions support TLSv1.2
> 
> It might sound complicated but it is not, it is very common. All
> you have to do is to have a Virtual Directory in IIS that requires
> 2way SSL and is set to "want". RSA-based key exchanges are one of
> the most commonly used ciphers. The Windows platforms we tested
> with are still widely used in the enterprises while the adoption
> rate of win2k12+ is still lagging 

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

Hi Chris,

How do you force Java 8 to use SSLv2Hello? So far I am unable to force it to go 
down to use SSLv2Hello. It would've been nice to be able to control the first 
handshake message version (TLSv1 or TLSv1.2 or SSLv2Hello), but I have not been 
able to do that...

George

-Original Message-
From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
Sent: Tuesday, October 13, 2015 7:42 AM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

George,

On 10/9/15 12:46 PM, George Stanchev wrote:
> One more clarification: on point [6] below I stated that Java is able 
> to recover with a retry on a cached connection. Unfortunately that is 
> only valid for higher level classes like HttpUrlConnection which makes 
> 1 retry on IOException (and only on a GET and some other conditions). 
> The lower-level sockets just throw and that’s it...

Can you confirm whether using SSLv2Hello also solves the problem? Or do you 
have a requirement that nothing less than TLSv1 may even handshake properly?

- -chris

> -Original Message- From: George Stanchev 
> [mailto:gstanc...@serena.com] Sent: Friday, October 09, 2015 10:40 AM 
> To: Tomcat Users List Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67:
> SEND TLSv1 ALERT: fatal, description = bad_record_mac
> 
> Just for the record, https.protocols is a property used by the 
> HttpsUrlConnection class. If your app is using a client that doesn't 
> rely on the internal Oracle HTTP client, it's better to use " 
> jdk.tls.client.protocols" which is read directly by the socket/SSL 
> classes. Apache Http Client is one client that does use sockets 
> directly instead of HttpsUrlConnection.
> 
> 
> Also, I want to mention that we have finally been able to isolate a 
> really nasty problem with Java SSL implementation and Windows's SSPI 
> based SChannel acting as a server. The issue we ran into is very 
> similar as to what the OP reported, that’s why I asked him a while ago 
> on a separate thread about keystores if the server is IIS-based. The 
> issue that we ran into might sound esoteric but our product ran into 
> it very often. We have a configuration where we lock our 
> intracomponent communication via 2way SSL. We have Java Tomcat-based 
> server components that talk to IIS based native components via web 
> services over HTTP which we secure with 2way SSL. When we upgraded to 
> JRE 8 we started getting those connection resets in Tomcat when we 
> enabled the 2way SSL. Those were on the
> Java->IIS calls that work over regular SSL. On the java side, if
> you turn on the ssl debug on, you'd see connection reset IOException 
> thrown, and on the Windows side, you'd see " The following fatal alert 
> was generated: 20. The internal error state is 960." In the System 
> event log. This error is "
> TLS1_ALERT_BAD_RECORD_MAC" from SSPI. A search in Oracle's bug 
> database found a close resemblance to this issue:
> https://bugs.openjdk.java.net/browse/JDK-8068812. But the reporter 
> couldn't reproduce it and also was against another Oracle component 
> Java Web Download or something like this. So we started working with 
> Oracle but the intermittent and hard-to-reproduce nature of this issue 
> hampered the investigation. As the OP mentioned below, there are 
> apocryphal reports for this all over the web but people cannot 
> troubleshoot it very well. Anyway here are my findings. In order for 
> the issue to arise all those conditions must be met:
> 
> [1] The client must start with ClientHello TLSv1.2 [2] The server must 
> respond with ServerHello TLSv1 (so the server should not have
> TLS1.2 enabled) [3] The cipher key exchange must NOT be ECDHE. I have 
> not tried DHE as I could not make Windows take it. According to this 
> doc [1] it should have several DHE key exchange ciphers available but 
> I could not make it work. RSA key exchange exhibits the issue. I have 
> been using “TLS_RSA_WITH_AES_256_CBC_SHA” to reproduce. [4] The 
> initial connection must not be 2way-SSL [5] The server must upgrade 
> the connection after encrypted payload is read.
> In case of IIS, it has 2way SSL setting on a Virtual Directory. The 
> initial handshake is not 2way SSL, once the HTTP request is read, it 
> determines that the requested resource (/gsoap) is 2way SSL-protected 
> and upgrades the connection to 2way SSL. [6] The server must kill the 
> connection abruptly when error happens. This is important as if the 
> server terminates the connection gracefully and keeps it around Java 
> is able to recover since it reuses the cached connection. IIS does 
> exactly this. [7] The server must be Windows 7, Win2k8, Win2k8-R2, 

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

Hi Chris,

[1] states: " JDK 7-9 enables SSLv2Hello on the server side only.  (Will not 
send, but will accept SSLv2Hellos)"

I've opened support case both MS and already there is a bug filed with Oracle 
on this and really, to be absolutely certain if the issue is in Java or 
SChannel, one would have to write a non-Java client that that mimics the 
handshake messages send from Java with something like OpenSSL or NSS or 
whatever and see if the bug replicates. Writing a Java/Java server client could 
still leave some doubts as one can argue the code reuse could mask the problem 
- it works but the bug on the client side is hidden by the server containing 
similar/same bug so the MACs check out...

Unfortunately I don't have the time to invest in this more than I already had. 
And if MS support engineers can pass it on to someone from the windows core 
team may be we can have some movement forward. 

George


[1] 
http://www.oracle.com/technetwork/java/javase/documentation/cve-2014-3566-2342133.html

-Original Message-
From: George Stanchev [mailto:gstanc...@serena.com] 
Sent: Tuesday, October 13, 2015 10:26 AM
To: Tomcat Users List
Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

Hi Chris,

How do you force Java 8 to use SSLv2Hello? So far I am unable to force it to go 
down to use SSLv2Hello. It would've been nice to be able to control the first 
handshake message version (TLSv1 or TLSv1.2 or SSLv2Hello), but I have not been 
able to do that...

George

-Original Message-
From: Christopher Schultz [mailto:ch...@christopherschultz.net]
Sent: Tuesday, October 13, 2015 7:42 AM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

George,

On 10/9/15 12:46 PM, George Stanchev wrote:
> One more clarification: on point [6] below I stated that Java is able 
> to recover with a retry on a cached connection. Unfortunately that is 
> only valid for higher level classes like HttpUrlConnection which makes
> 1 retry on IOException (and only on a GET and some other conditions). 
> The lower-level sockets just throw and that’s it...

Can you confirm whether using SSLv2Hello also solves the problem? Or do you 
have a requirement that nothing less than TLSv1 may even handshake properly?

- -chris

> -Original Message- From: George Stanchev 
> [mailto:gstanc...@serena.com] Sent: Friday, October 09, 2015 10:40 AM
> To: Tomcat Users List Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67:
> SEND TLSv1 ALERT: fatal, description = bad_record_mac
> 
> Just for the record, https.protocols is a property used by the 
> HttpsUrlConnection class. If your app is using a client that doesn't 
> rely on the internal Oracle HTTP client, it's better to use "
> jdk.tls.client.protocols" which is read directly by the socket/SSL 
> classes. Apache Http Client is one client that does use sockets 
> directly instead of HttpsUrlConnection.
> 
> 
> Also, I want to mention that we have finally been able to isolate a 
> really nasty problem with Java SSL implementation and Windows's SSPI 
> based SChannel acting as a server. The issue we ran into is very 
> similar as to what the OP reported, that’s why I asked him a while ago 
> on a separate thread about keystores if the server is IIS-based. The 
> issue that we ran into might sound esoteric but our product ran into 
> it very often. We have a configuration where we lock our 
> intracomponent communication via 2way SSL. We have Java Tomcat-based 
> server components that talk to IIS based native components via web 
> services over HTTP which we secure with 2way SSL. When we upgraded to 
> JRE 8 we started getting those connection resets in Tomcat when we 
> enabled the 2way SSL. Those were on the
> Java->IIS calls that work over regular SSL. On the java side, if
> you turn on the ssl debug on, you'd see connection reset IOException 
> thrown, and on the Windows side, you'd see " The following fatal alert 
> was generated: 20. The internal error state is 960." In the System 
> event log. This error is "
> TLS1_ALERT_BAD_RECORD_MAC" from SSPI. A search in Oracle's bug 
> database found a close resemblance to this issue:
> https://bugs.openjdk.java.net/browse/JDK-8068812. But the reporter 
> couldn't reproduce it and also was against another Oracle component 
> Java Web Download or something like this. So we started working with 
> Oracle but the intermittent and hard-to-reproduce nature of this issue 
> hampered the investigation. As the OP mentioned below, there are 
> apocryphal reports for this all over the web but people cannot 
> troubleshoot it very well. Anyway here are my findings. In order for 
> the issue to arise all those conditi

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

George,

On 10/13/15 12:35 PM, George Stanchev wrote:
> [1] states: " JDK 7-9 enables SSLv2Hello on the server side only. 
> (Will not send, but will accept SSLv2Hellos)"

Interesting. This absolutely makes sense, though, since SSL should
just die. :)

> I've opened support case both MS and already there is a bug filed 
> with Oracle on this and really, to be absolutely certain if the
> issue is in Java or SChannel, one would have to write a non-Java
> client that that mimics the handshake messages send from Java with
> something like OpenSSL or NSS or whatever and see if the bug
> replicates.

It might be doable with OpenSSL s_client or something. Tough to
replicate Java's behavior with a non-Java tool, though.

> Writing a Java/Java server client could still leave some doubts as 
> one can argue the code reuse could mask the problem - it works but 
> the bug on the client side is hidden by the server containing 
> similar/same bug so the MACs check out...
> 
> Unfortunately I don't have the time to invest in this more than I 
> already had. And if MS support engineers can pass it on to someone 
> from the windows core team may be we can have some movement
> forward.

Okay. Thanks for your work so far.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJWHWNuAAoJEBzwKT+lPKRYgWAP/A8fUJ5Dzu+O46GNWpdobqq0
7ugkb2cQ1VM92Q+22Wtl87GSRPhBS8jwNrBBCJmoyBjRZ/LKVcwtcWLzUIBllXm5
t8iorXpQxaps1G0AEEf5tAwHXyN75J2vC9qvRvD6dkekHXHwO3RRqvSCqQjEjeVJ
XsOdjuIhPwX0B0SN8Apdshvxe98sC9QPn73LNdSM9+j8Ob1vCDHDiMFj60K72Su1
E0UmPYEJdhb5D+PvSM/7CMcAlkJYmCl8VlNFWD320ymjObfIMymfOk+kqKLqVItQ
+r2e20At1qCyeyg2Gcxb4X1ajIhcxdgP7WJYtg57Pwrp4ZVZ6d7RM+CIqt28SfxT
EtTamDZ8aPYsCKqMWIYRPyLrWaouEuLJEmzweF8B+NxY0svK8vEOiro/vR4LycOZ
PG5zxuS/QMJR2oEgkeUz9+NhB8nP/qJxMhc40pKGmvZxC7ljM/tP7jTvE1MwmMHE
P8rX5b3yF8DfMdGZdIlrHJ8wXYSzJdTMJvA5ffXVKaObGMYtAFFDlfupud1iO9ML
Hh4exxX+/NU7fXt+ot6BLEFAfDD9+z6uOeq+vK6bxaITubFVGIavhowjAgQIBNt3
O//p9dJQVKan0db9kqyLpLMrrFYd/cmA8DZDxoY/iaVuoKhJ6blbDMQKi2DlsvgF
WGDHUsSBZIYTFq5mc7VO
=eyUN
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Aurélien Terrestris]

George,

do you have any network capture that we can see ?

2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> >> It might be doable with OpenSSL s_client or something. Tough to
> replicate Java's behavior with a non-Java tool, though.
>
> I tried hard with the s_client but it can limit the protocols to one or
> another but it canot mix-and-match (hello 1.2, ok we will do 1.0) like Java
> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which is
> also on top of openssl.
>
> Today, I spent 2.5 hours with a lemming from MS getting basically nowhere.
> I really need an engineer, but they give me those clueless support people
> that is wasting mine and their time. If someone knows how to escalate or a
> forum where MS developers hang out, I would appreciate it. The support
> person I got today was clueless, went over a script and any attempt to
> explain a little more technical details led to total confusion and
> rebooting the script to beginning. Totally frustrating. At least with
> Oracle I got to talk with an actual engineer...
>
> George
>
>
> -Original Message-
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Sent: Tuesday, October 13, 2015 2:03 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
> description = bad_record_mac
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> George,
>
> On 10/13/15 12:35 PM, George Stanchev wrote:
> > [1] states: " JDK 7-9 enables SSLv2Hello on the server side only.
> > (Will not send, but will accept SSLv2Hellos)"
>
> Interesting. This absolutely makes sense, though, since SSL should just
> die. :)
>
> > I've opened support case both MS and already there is a bug filed with
> > Oracle on this and really, to be absolutely certain if the issue is in
> > Java or SChannel, one would have to write a non-Java client that that
> > mimics the handshake messages send from Java with something like
> > OpenSSL or NSS or whatever and see if the bug replicates.
>
> It might be doable with OpenSSL s_client or something. Tough to replicate
> Java's behavior with a non-Java tool, though.
>
> > Writing a Java/Java server client could still leave some doubts as one
> > can argue the code reuse could mask the problem - it works but the bug
> > on the client side is hidden by the server containing similar/same bug
> > so the MACs check out...
> >
> > Unfortunately I don't have the time to invest in this more than I
> > already had. And if MS support engineers can pass it on to someone
> > from the windows core team may be we can have some movement forward.
>
> Okay. Thanks for your work so far.
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
>
> iQIcBAEBCAAGBQJWHWNuAAoJEBzwKT+lPKRYgWAP/A8fUJ5Dzu+O46GNWpdobqq0
> 7ugkb2cQ1VM92Q+22Wtl87GSRPhBS8jwNrBBCJmoyBjRZ/LKVcwtcWLzUIBllXm5
> t8iorXpQxaps1G0AEEf5tAwHXyN75J2vC9qvRvD6dkekHXHwO3RRqvSCqQjEjeVJ
> XsOdjuIhPwX0B0SN8Apdshvxe98sC9QPn73LNdSM9+j8Ob1vCDHDiMFj60K72Su1
> E0UmPYEJdhb5D+PvSM/7CMcAlkJYmCl8VlNFWD320ymjObfIMymfOk+kqKLqVItQ
> +r2e20At1qCyeyg2Gcxb4X1ajIhcxdgP7WJYtg57Pwrp4ZVZ6d7RM+CIqt28SfxT
> EtTamDZ8aPYsCKqMWIYRPyLrWaouEuLJEmzweF8B+NxY0svK8vEOiro/vR4LycOZ
> PG5zxuS/QMJR2oEgkeUz9+NhB8nP/qJxMhc40pKGmvZxC7ljM/tP7jTvE1MwmMHE
> P8rX5b3yF8DfMdGZdIlrHJ8wXYSzJdTMJvA5ffXVKaObGMYtAFFDlfupud1iO9ML
> Hh4exxX+/NU7fXt+ot6BLEFAfDD9+z6uOeq+vK6bxaITubFVGIavhowjAgQIBNt3
> O//p9dJQVKan0db9kqyLpLMrrFYd/cmA8DZDxoY/iaVuoKhJ6blbDMQKi2DlsvgF
> WGDHUsSBZIYTFq5mc7VO
> =eyUN
> -END PGP SIGNATURE-
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

>> It might be doable with OpenSSL s_client or something. Tough to replicate 
>> Java's behavior with a non-Java tool, though.

I tried hard with the s_client but it can limit the protocols to one or another 
but it canot mix-and-match (hello 1.2, ok we will do 1.0) like Java 8 does. 
Either TLSv1 or TLSv1.2 but not both. Neither can curl which is also on top of 
openssl.

Today, I spent 2.5 hours with a lemming from MS getting basically nowhere. I 
really need an engineer, but they give me those clueless support people that is 
wasting mine and their time. If someone knows how to escalate or a forum where 
MS developers hang out, I would appreciate it. The support person I got today 
was clueless, went over a script and any attempt to explain a little more 
technical details led to total confusion and rebooting the script to beginning. 
Totally frustrating. At least with Oracle I got to talk with an actual 
engineer...

George


-Original Message-
From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
Sent: Tuesday, October 13, 2015 2:03 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

George,

On 10/13/15 12:35 PM, George Stanchev wrote:
> [1] states: " JDK 7-9 enables SSLv2Hello on the server side only. 
> (Will not send, but will accept SSLv2Hellos)"

Interesting. This absolutely makes sense, though, since SSL should just die. :)

> I've opened support case both MS and already there is a bug filed with 
> Oracle on this and really, to be absolutely certain if the issue is in 
> Java or SChannel, one would have to write a non-Java client that that 
> mimics the handshake messages send from Java with something like 
> OpenSSL or NSS or whatever and see if the bug replicates.

It might be doable with OpenSSL s_client or something. Tough to replicate 
Java's behavior with a non-Java tool, though.

> Writing a Java/Java server client could still leave some doubts as one 
> can argue the code reuse could mask the problem - it works but the bug 
> on the client side is hidden by the server containing similar/same bug 
> so the MACs check out...
> 
> Unfortunately I don't have the time to invest in this more than I 
> already had. And if MS support engineers can pass it on to someone 
> from the windows core team may be we can have some movement forward.

Okay. Thanks for your work so far.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJWHWNuAAoJEBzwKT+lPKRYgWAP/A8fUJ5Dzu+O46GNWpdobqq0
7ugkb2cQ1VM92Q+22Wtl87GSRPhBS8jwNrBBCJmoyBjRZ/LKVcwtcWLzUIBllXm5
t8iorXpQxaps1G0AEEf5tAwHXyN75J2vC9qvRvD6dkekHXHwO3RRqvSCqQjEjeVJ
XsOdjuIhPwX0B0SN8Apdshvxe98sC9QPn73LNdSM9+j8Ob1vCDHDiMFj60K72Su1
E0UmPYEJdhb5D+PvSM/7CMcAlkJYmCl8VlNFWD320ymjObfIMymfOk+kqKLqVItQ
+r2e20At1qCyeyg2Gcxb4X1ajIhcxdgP7WJYtg57Pwrp4ZVZ6d7RM+CIqt28SfxT
EtTamDZ8aPYsCKqMWIYRPyLrWaouEuLJEmzweF8B+NxY0svK8vEOiro/vR4LycOZ
PG5zxuS/QMJR2oEgkeUz9+NhB8nP/qJxMhc40pKGmvZxC7ljM/tP7jTvE1MwmMHE
P8rX5b3yF8DfMdGZdIlrHJ8wXYSzJdTMJvA5ffXVKaObGMYtAFFDlfupud1iO9ML
Hh4exxX+/NU7fXt+ot6BLEFAfDD9+z6uOeq+vK6bxaITubFVGIavhowjAgQIBNt3
O//p9dJQVKan0db9kqyLpLMrrFYd/cmA8DZDxoY/iaVuoKhJ6blbDMQKi2DlsvgF
WGDHUsSBZIYTFq5mc7VO
=eyUN
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

Hi Aurelien,

https://www.dropbox.com/sh/az1r3agxx4w8r7e/AACRGedBG3G5oh4-qE9652WNa?dl=0

It's not a pcap but rather javax.debug logger capture which is similar. I can 
do pcap but since java dumps the stream along with handshake messages 
description, I haven't bothered to capture it. This folder also contains the 
test Java client and an SSPI echo server that emulates the problem and has the 
SSPI calls to demonstrate the issue. "cood capture.txt" is when you run with 
TLSv1-only. "bad capture.txt" is when you run with defaults...If you run the 
java client you need to update

final String targetURL = 
"https://rm9485:443/gsoap/gsoap_ssl.dll?sbminternalservices72;;

to point to the correct machine. Uncommenting 
//System.setProperty("jdk.tls.client.protocols", "TLSv1"); makes the call 
succeed. The SSLServer on the native side will return SEC_E_DECRYPT_FAILURE 
from AcceptSecurityContext() when the TLS connection is upgraded to 2way ssl 
after ClientCertVerify's changing of cipher specs. I spent time in the Java 
handshaker and while I don't understand it enough, I saw the failure right 
after it flushes the  running MAC down the stream onto the server. At this 
point SSPI fails the call with bad_mac_record which is behind the code 
SEC_E_DECRYPT_FAILURE...

George

-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com] 
Sent: Tuesday, October 13, 2015 2:23 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

George,

do you have any network capture that we can see ?

2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> >> It might be doable with OpenSSL s_client or something. Tough to
> replicate Java's behavior with a non-Java tool, though.
>
> I tried hard with the s_client but it can limit the protocols to one 
> or another but it canot mix-and-match (hello 1.2, ok we will do 1.0) 
> like Java
> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which 
> is also on top of openssl.
>
> Today, I spent 2.5 hours with a lemming from MS getting basically nowhere.
> I really need an engineer, but they give me those clueless support 
> people that is wasting mine and their time. If someone knows how to 
> escalate or a forum where MS developers hang out, I would appreciate 
> it. The support person I got today was clueless, went over a script 
> and any attempt to explain a little more technical details led to 
> total confusion and rebooting the script to beginning. Totally 
> frustrating. At least with Oracle I got to talk with an actual engineer...
>
> George
>
>
> -Original Message-
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Sent: Tuesday, October 13, 2015 2:03 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
> description = bad_record_mac
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> George,
>
> On 10/13/15 12:35 PM, George Stanchev wrote:
> > [1] states: " JDK 7-9 enables SSLv2Hello on the server side only.
> > (Will not send, but will accept SSLv2Hellos)"
>
> Interesting. This absolutely makes sense, though, since SSL should 
> just die. :)
>
> > I've opened support case both MS and already there is a bug filed 
> > with Oracle on this and really, to be absolutely certain if the 
> > issue is in Java or SChannel, one would have to write a non-Java 
> > client that that mimics the handshake messages send from Java with 
> > something like OpenSSL or NSS or whatever and see if the bug replicates.
>
> It might be doable with OpenSSL s_client or something. Tough to 
> replicate Java's behavior with a non-Java tool, though.
>
> > Writing a Java/Java server client could still leave some doubts as 
> > one can argue the code reuse could mask the problem - it works but 
> > the bug on the client side is hidden by the server containing 
> > similar/same bug so the MACs check out...
> >
> > Unfortunately I don't have the time to invest in this more than I 
> > already had. And if MS support engineers can pass it on to someone 
> > from the windows core team may be we can have some movement forward.
>
> Okay. Thanks for your work so far.
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
>
> iQIcBAEBCAAGBQJWHWNuAAoJEBzwKT+lPKRYgWAP/A8fUJ5Dzu+O46GNWpdobqq0
> 7ugkb2cQ1VM92Q+22Wtl87GSRPhBS8jwNrBBCJmoyBjRZ/LKVcwtcWLzUIBllXm5
> t8iorXpQxaps1G0AEEf5tAwHXyN75J2vC9qvRvD6dkekHXHwO3RRqvSCqQjEjeVJ
> XsOdjuIhPwX0B0SN8Apdshvxe98sC9QPn73LNdSM9+j8Ob1vCDHDiMFj60K72Su1
> E0UmPYEJdhb5D+PvSM/7CMcAlkJYmCl8VlNFWD320ymjObfIMymfOk+kqKLqVItQ
> +r2e20At1qCyey

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Aurélien Terrestris]

"Another route to try the SSLv2Hello is to hack and recompile the JSSE.
It's on my agenda to learn to do that if possible without rebuilding the
whole JRE since for some obscure reason Oracle compiles JSSE with no debug
info (and it is not included in the srcs distributed with the JRE) on it
which makes debugging and inspecting local vars really hard..."

No, it's not needed. I use jtouch.sourceforge.net and it's working well for
trying SSLv2 hello's. From the network captures, I see that it's working.
Not making advertisement for my software here, but,.. ;)







2015-10-13 23:20 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> Just as a side note, https.protocols is read by HttpsUrlConnection which
> feeds it down through setEnabledProtocols() on the SSL socket. "
> jdk.tls.client.protocols" is read directly by JSSE and does the same
> thing...
>
> Another route to try the SSLv2Hello is to hack and recompile the JSSE.
> It's on my agenda to learn to do that if possible without rebuilding the
> whole JRE since for some obscure reason Oracle compiles JSSE with no debug
> info (and it is not included in the srcs distributed with the JRE) on it
> which makes debugging and inspecting local vars really hard...
>
> George
>
> -Original Message-
> From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
> Sent: Tuesday, October 13, 2015 3:13 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
> description = bad_record_mac
>
> Maybe writing too fast..
>
> "How do you force Java 8 to use SSLv2Hello?"
>
> As suggested before, by writing your own client OR by trying this hack.
> From the JSSE Ref Guide 5, we know how to the customize the protocol (
> https://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefGuide.html#InstallationAndCustomization
> ) by setting a system property (https.protocol). Although they are no more
> talking about this in the Ref Guide 8 (
> https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#SunJSSEProvider
> ) I would give it a try as I know that the documentation is poorly written.
> I suggested 10 years ago a change in the API doc about the enabled
> protocols, and they didn't change anything although they said they would.
>
>
>
> "[1] states: " JDK 7-9 enables SSLv2Hello on the server side only.  (Will
> not send, but will accept SSLv2Hellos)""
>
> I believe they mean "by default" as for the client side. Poorly written,
> probably.
>
>
>
> 2015-10-13 22:55 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
>
> > "How do you force Java 8 to use SSLv2Hello?"
> >
> > You can do this when writing your own Java client : calling the
> > SSLSocketFactory to create an SSLSocket and configure with
> > setEnabledProtocols (
> > https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html
> > #setEnabledProtocols-java.lang.String:A-
> > )
> >
> > If you have some IIS server on internet which reproduces the problem,
> > I'll try with JTouch ( jtouch.sourceforge.net ) or write a small client.
> >
> >
> >
> >
> > 2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
> >
> >> George,
> >>
> >> do you have any network capture that we can see ?
> >>
> >> 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
> >>
> >>> >> It might be doable with OpenSSL s_client or something. Tough to
> >>> replicate Java's behavior with a non-Java tool, though.
> >>>
> >>> I tried hard with the s_client but it can limit the protocols to one
> >>> or another but it canot mix-and-match (hello 1.2, ok we will do 1.0)
> >>> like Java
> >>> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which
> >>> is also on top of openssl.
> >>>
> >>> Today, I spent 2.5 hours with a lemming from MS getting basically
> >>> nowhere. I really need an engineer, but they give me those clueless
> >>> support people that is wasting mine and their time. If someone knows
> >>> how to escalate or a forum where MS developers hang out, I would
> appreciate it.
> >>> The support person I got today was clueless, went over a script and
> >>> any attempt to explain a little more technical details led to total
> >>> confusion and rebooting the script to beginning. Totally
> >>> frustrating. At least with Oracle I got to talk with an actual
> engineer...
> >>>
> >>> George
> >&g

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

George and Aurélien,

On 10/13/15 5:06 PM, George Stanchev wrote:
> Try the dropbox location in my prev messages. It contains a sample 
> echo server you can use to test. It is a Visual Studio 2013
> project. In case you don’t have that, I've uploaded x64 executables
> under SSLServer_executbale.zip. It is pretty much self contained,
> it has the redist DLLs, the server cert, all...
> 
> It eliminates the need of IIS as it does the same thing - accept 
> connection, read payload, upgrade to 2 way ssl...
> 
> It generates output like this [1]
> 
> The line " Decrypt error from SCHANNEL, Client ID: 
> a1cefeb8-bad3-4903-8dbe-fdea347f666e" is emitted when bad record
> mac is caught on the server side...

On 10/13/15 4:55 PM, Aurélien Terrestris wrote:
> "How do you force Java 8 to use SSLv2Hello?"
> 
> You can do this when writing your own Java client : calling the 
> SSLSocketFactory to create an SSLSocket and configure with 
> setEnabledProtocols (
> 
https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html#s
etEnabledProtocols-java.lang.String:A-
> )
> 
> If you have some IIS server on internet which reproduces the
> problem, I'll try with JTouch ( jtouch.sourceforge.net ) or write a
> small client.

I've got a client already written. I'll post the latest code somewhere.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJWHXLNAAoJEBzwKT+lPKRYCp4P+wZX2tiJ41FlRIYSK1sf7kAl
0yBJiFGDChNa8+zYvs5WYGcOFCQmBHXHepJTi7Zff3G0NrRcgEAS/Yo9yUjSFPBJ
R0kRoIUAwdeU882lhCkRulC8SyJZv6jq13NqUjYO9J4GACf58rweFfZcF5o/5RL8
rZ7ZSgXNj/VyXIilOKuZ3Ak6262X7qLnlhnSnXImSfZJND2PKf0l05cL4jrObtaA
mqWv/zuGdDfRKJgBWMo9oHxjGfe+c9NDIm18uFUf23hMbuX9bJImQ+LLAXgibUTk
eRuqr1igjAqqqnVPowDOLX/CKH7ikQ+ZygPvzFvztjeEXC4IhBj8hC/LjPG7tkfl
XNZVD0K1ryni0f/qWvyTL5MHGiJX1+zS6VFm3cYTif+FREvg1P4eGxAFIFwAHQgm
2Yz3N1+r8qJHnVSjQoDlbjPcB9ba2xDg+gMyryDTx0Zxc83qzVOfOdcMuHWgwqTD
5b9KlzdqLzhh7mXj9L5/+L2EomsehWayzM4PLv31QVrv1UzR3vA/6L64Pwe0n512
9VdDMgk3qX54RHetxpE63JYl9EtmWVaE1GxOb/ar1kPi4h08TTTvs2Vtxk3i+F8Y
sv1KLziEW1MsQg1l8Nvn++5GoLhTNJapra3FYBkpitK3AMObip3G7hOPIqdu/C8P
HtBxh5fmOEqYmDC2UB5c
=3OPW
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

Ok, may be you are ahead of me on this one, but setEnabledProtocols(), to my 
understanding does the same as the system property "jdk.tls.client.protocols". 
It gives the JSSE a pool of protocols to choose from but it will be up to JSSE 
to select the ClientHello version. I might have missed something but I spent 
quite a bit of time in the Handshaker and related classes in the JDK and 
couldn't see anything that can change that...

George

-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com] 
Sent: Tuesday, October 13, 2015 2:55 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

"How do you force Java 8 to use SSLv2Hello?"

You can do this when writing your own Java client : calling the 
SSLSocketFactory to create an SSLSocket and configure with setEnabledProtocols (
https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html#setEnabledProtocols-java.lang.String:A-
)

If you have some IIS server on internet which reproduces the problem, I'll try 
with JTouch ( jtouch.sourceforge.net ) or write a small client.




2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:

> George,
>
> do you have any network capture that we can see ?
>
> 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
>
>> >> It might be doable with OpenSSL s_client or something. Tough to
>> replicate Java's behavior with a non-Java tool, though.
>>
>> I tried hard with the s_client but it can limit the protocols to one 
>> or another but it canot mix-and-match (hello 1.2, ok we will do 1.0) 
>> like Java
>> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which 
>> is also on top of openssl.
>>
>> Today, I spent 2.5 hours with a lemming from MS getting basically 
>> nowhere. I really need an engineer, but they give me those clueless 
>> support people that is wasting mine and their time. If someone knows 
>> how to escalate or a forum where MS developers hang out, I would appreciate 
>> it.
>> The support person I got today was clueless, went over a script and 
>> any attempt to explain a little more technical details led to total 
>> confusion and rebooting the script to beginning. Totally frustrating. 
>> At least with Oracle I got to talk with an actual engineer...
>>
>> George
>>
>>
>> -----Original Message-----
>> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
>> Sent: Tuesday, October 13, 2015 2:03 PM
>> To: Tomcat Users List
>> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
>> description = bad_record_mac
>>
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA256
>>
>> George,
>>
>> On 10/13/15 12:35 PM, George Stanchev wrote:
>> > [1] states: " JDK 7-9 enables SSLv2Hello on the server side only.
>> > (Will not send, but will accept SSLv2Hellos)"
>>
>> Interesting. This absolutely makes sense, though, since SSL should 
>> just die. :)
>>
>> > I've opened support case both MS and already there is a bug filed 
>> > with Oracle on this and really, to be absolutely certain if the 
>> > issue is in Java or SChannel, one would have to write a non-Java 
>> > client that that mimics the handshake messages send from Java with 
>> > something like OpenSSL or NSS or whatever and see if the bug replicates.
>>
>> It might be doable with OpenSSL s_client or something. Tough to 
>> replicate Java's behavior with a non-Java tool, though.
>>
>> > Writing a Java/Java server client could still leave some doubts as 
>> > one can argue the code reuse could mask the problem - it works but 
>> > the bug on the client side is hidden by the server containing 
>> > similar/same bug so the MACs check out...
>> >
>> > Unfortunately I don't have the time to invest in this more than I 
>> > already had. And if MS support engineers can pass it on to someone 
>> > from the windows core team may be we can have some movement forward.
>>
>> Okay. Thanks for your work so far.
>>
>> - -chris
>> -BEGIN PGP SIGNATURE-
>> Comment: GPGTools - http://gpgtools.org
>>
>> iQIcBAEBCAAGBQJWHWNuAAoJEBzwKT+lPKRYgWAP/A8fUJ5Dzu+O46GNWpdobqq0
>> 7ugkb2cQ1VM92Q+22Wtl87GSRPhBS8jwNrBBCJmoyBjRZ/LKVcwtcWLzUIBllXm5
>> t8iorXpQxaps1G0AEEf5tAwHXyN75J2vC9qvRvD6dkekHXHwO3RRqvSCqQjEjeVJ
>> XsOdjuIhPwX0B0SN8Apdshvxe98sC9QPn73LNdSM9+j8Ob1vCDHDiMFj60K72Su1
>> E0UmPYEJdhb5D+PvSM/7CMcAlkJYmCl8VlNFWD320ymjObfIMymfOk+kqKLqVItQ
>> +r2e20At1qCyeyg2Gcxb4X1ajIhcxdgP7WJYtg57

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Aurélien Terrestris]

"How do you force Java 8 to use SSLv2Hello?"

You can do this when writing your own Java client : calling the
SSLSocketFactory to create an SSLSocket and configure with
setEnabledProtocols (
https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html#setEnabledProtocols-java.lang.String:A-
)

If you have some IIS server on internet which reproduces the problem, I'll
try with JTouch ( jtouch.sourceforge.net ) or write a small client.




2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:

> George,
>
> do you have any network capture that we can see ?
>
> 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
>
>> >> It might be doable with OpenSSL s_client or something. Tough to
>> replicate Java's behavior with a non-Java tool, though.
>>
>> I tried hard with the s_client but it can limit the protocols to one or
>> another but it canot mix-and-match (hello 1.2, ok we will do 1.0) like Java
>> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which is
>> also on top of openssl.
>>
>> Today, I spent 2.5 hours with a lemming from MS getting basically
>> nowhere. I really need an engineer, but they give me those clueless support
>> people that is wasting mine and their time. If someone knows how to
>> escalate or a forum where MS developers hang out, I would appreciate it.
>> The support person I got today was clueless, went over a script and any
>> attempt to explain a little more technical details led to total confusion
>> and rebooting the script to beginning. Totally frustrating. At least with
>> Oracle I got to talk with an actual engineer...
>>
>> George
>>
>>
>> -----Original Message-----
>> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
>> Sent: Tuesday, October 13, 2015 2:03 PM
>> To: Tomcat Users List
>> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
>> description = bad_record_mac
>>
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA256
>>
>> George,
>>
>> On 10/13/15 12:35 PM, George Stanchev wrote:
>> > [1] states: " JDK 7-9 enables SSLv2Hello on the server side only.
>> > (Will not send, but will accept SSLv2Hellos)"
>>
>> Interesting. This absolutely makes sense, though, since SSL should just
>> die. :)
>>
>> > I've opened support case both MS and already there is a bug filed with
>> > Oracle on this and really, to be absolutely certain if the issue is in
>> > Java or SChannel, one would have to write a non-Java client that that
>> > mimics the handshake messages send from Java with something like
>> > OpenSSL or NSS or whatever and see if the bug replicates.
>>
>> It might be doable with OpenSSL s_client or something. Tough to replicate
>> Java's behavior with a non-Java tool, though.
>>
>> > Writing a Java/Java server client could still leave some doubts as one
>> > can argue the code reuse could mask the problem - it works but the bug
>> > on the client side is hidden by the server containing similar/same bug
>> > so the MACs check out...
>> >
>> > Unfortunately I don't have the time to invest in this more than I
>> > already had. And if MS support engineers can pass it on to someone
>> > from the windows core team may be we can have some movement forward.
>>
>> Okay. Thanks for your work so far.
>>
>> - -chris
>> -BEGIN PGP SIGNATURE-
>> Comment: GPGTools - http://gpgtools.org
>>
>> iQIcBAEBCAAGBQJWHWNuAAoJEBzwKT+lPKRYgWAP/A8fUJ5Dzu+O46GNWpdobqq0
>> 7ugkb2cQ1VM92Q+22Wtl87GSRPhBS8jwNrBBCJmoyBjRZ/LKVcwtcWLzUIBllXm5
>> t8iorXpQxaps1G0AEEf5tAwHXyN75J2vC9qvRvD6dkekHXHwO3RRqvSCqQjEjeVJ
>> XsOdjuIhPwX0B0SN8Apdshvxe98sC9QPn73LNdSM9+j8Ob1vCDHDiMFj60K72Su1
>> E0UmPYEJdhb5D+PvSM/7CMcAlkJYmCl8VlNFWD320ymjObfIMymfOk+kqKLqVItQ
>> +r2e20At1qCyeyg2Gcxb4X1ajIhcxdgP7WJYtg57Pwrp4ZVZ6d7RM+CIqt28SfxT
>> EtTamDZ8aPYsCKqMWIYRPyLrWaouEuLJEmzweF8B+NxY0svK8vEOiro/vR4LycOZ
>> PG5zxuS/QMJR2oEgkeUz9+NhB8nP/qJxMhc40pKGmvZxC7ljM/tP7jTvE1MwmMHE
>> P8rX5b3yF8DfMdGZdIlrHJ8wXYSzJdTMJvA5ffXVKaObGMYtAFFDlfupud1iO9ML
>> Hh4exxX+/NU7fXt+ot6BLEFAfDD9+z6uOeq+vK6bxaITubFVGIavhowjAgQIBNt3
>> O//p9dJQVKan0db9kqyLpLMrrFYd/cmA8DZDxoY/iaVuoKhJ6blbDMQKi2DlsvgF
>> WGDHUsSBZIYTFq5mc7VO
>> =eyUN
>> -END PGP SIGNATURE-
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>
>

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

Aurlien,

Try the dropbox location in my prev messages. It contains a sample echo server 
you can use to test. It is a Visual Studio 2013 project. In case you don’t have 
that, I've uploaded x64 executables under SSLServer_executbale.zip. It is 
pretty much self contained, it has the redist DLLs, the server cert, all...

It eliminates the need of IIS as it does the same thing - accept connection, 
read payload, upgrade to 2 way ssl...

It generates output like this [1]

The line " Decrypt error from SCHANNEL, Client ID: 
a1cefeb8-bad3-4903-8dbe-fdea347f666e" is emitted when bad record mac is caught 
on the server side...

George



[1]


Read: 1093 bytes from 192.168.2.164:41525
Read: 165 bytes from 192.168.2.164:41525
Read: 90 bytes from 192.168.2.164:41525
Decrypt error from SCHANNEL, Client ID: a1cefeb8-bad3-4903-8dbe-fdea347f666e
Removing Dead Connection, Client ID: a1cefeb8-bad3-4903-8dbe-fdea347f666e
Active Connections: 0
New Connection from 192.168.2.164:41526 with Client ID: a94811d3-0872-4b63-a429-
97f980fdc42d
Read: 89 bytes from 192.168.2.164:41526
Sent: 145 bytes to 192.168.2.164:41526
Read: 6 bytes from 192.168.2.164:41526
Read: 53 bytes from 192.168.2.164:41526
Handshake Success, Client ID: a94811d3-0872-4b63-a429-97f980fdc42d
Read: 213 bytes from 192.168.2.164:41526
Message Client ID: SSL renegotiation: request client cert
Handshake Success, Client ID: a94811d3-0872-4b63-a429-97f980fdc42d
Sent: 37 bytes to 192.168.2.164:41526
Read: 133 bytes from 192.168.2.164:41526
Sent: 805 bytes to 192.168.2.164:41526
Read: 1093 bytes from 192.168.2.164:41526
Read: 165 bytes from 192.168.2.164:41526
Read: 90 bytes from 192.168.2.164:41526
Sent: 90 bytes to 192.168.2.164:41526
Handshake Success, Client ID: a94811d3-0872-4b63-a429-97f980fdc42d

-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com] 
Sent: Tuesday, October 13, 2015 2:55 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

"How do you force Java 8 to use SSLv2Hello?"

You can do this when writing your own Java client : calling the 
SSLSocketFactory to create an SSLSocket and configure with setEnabledProtocols (
https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html#setEnabledProtocols-java.lang.String:A-
)

If you have some IIS server on internet which reproduces the problem, I'll try 
with JTouch ( jtouch.sourceforge.net ) or write a small client.




2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:

> George,
>
> do you have any network capture that we can see ?
>
> 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
>
>> >> It might be doable with OpenSSL s_client or something. Tough to
>> replicate Java's behavior with a non-Java tool, though.
>>
>> I tried hard with the s_client but it can limit the protocols to one 
>> or another but it canot mix-and-match (hello 1.2, ok we will do 1.0) 
>> like Java
>> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which 
>> is also on top of openssl.
>>
>> Today, I spent 2.5 hours with a lemming from MS getting basically 
>> nowhere. I really need an engineer, but they give me those clueless 
>> support people that is wasting mine and their time. If someone knows 
>> how to escalate or a forum where MS developers hang out, I would appreciate 
>> it.
>> The support person I got today was clueless, went over a script and 
>> any attempt to explain a little more technical details led to total 
>> confusion and rebooting the script to beginning. Totally frustrating. 
>> At least with Oracle I got to talk with an actual engineer...
>>
>> George
>>
>>
>> -----Original Message-----
>> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
>> Sent: Tuesday, October 13, 2015 2:03 PM
>> To: Tomcat Users List
>> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
>> description = bad_record_mac
>>
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA256
>>
>> George,
>>
>> On 10/13/15 12:35 PM, George Stanchev wrote:
>> > [1] states: " JDK 7-9 enables SSLv2Hello on the server side only.
>> > (Will not send, but will accept SSLv2Hellos)"
>>
>> Interesting. This absolutely makes sense, though, since SSL should 
>> just die. :)
>>
>> > I've opened support case both MS and already there is a bug filed 
>> > with Oracle on this and really, to be absolutely certain if the 
>> > issue is in Java or SChannel, one would have to write a non-Java 
>> > client that that mimics the handshake messages send from Java with 
>> > something like OpenSSL or NSS or whatever and see if th

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Aurélien Terrestris]

Maybe writing too fast..

"How do you force Java 8 to use SSLv2Hello?"

As suggested before, by writing your own client OR by trying this hack.
>From the JSSE Ref Guide 5, we know how to the customize the protocol (
https://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefGuide.html#InstallationAndCustomization
) by setting a system property (https.protocol). Although they are no more
talking about this in the Ref Guide 8 (
https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#SunJSSEProvider
) I would give it a try as I know that the documentation is poorly written.
I suggested 10 years ago a change in the API doc about the enabled
protocols, and they didn't change anything although they said they would.



"[1] states: " JDK 7-9 enables SSLv2Hello on the server side only.  (Will
not send, but will accept SSLv2Hellos)""

I believe they mean "by default" as for the client side. Poorly written,
probably.



2015-10-13 22:55 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:

> "How do you force Java 8 to use SSLv2Hello?"
>
> You can do this when writing your own Java client : calling the
> SSLSocketFactory to create an SSLSocket and configure with
> setEnabledProtocols (
> https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html#setEnabledProtocols-java.lang.String:A-
> )
>
> If you have some IIS server on internet which reproduces the problem, I'll
> try with JTouch ( jtouch.sourceforge.net ) or write a small client.
>
>
>
>
> 2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
>
>> George,
>>
>> do you have any network capture that we can see ?
>>
>> 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
>>
>>> >> It might be doable with OpenSSL s_client or something. Tough to
>>> replicate Java's behavior with a non-Java tool, though.
>>>
>>> I tried hard with the s_client but it can limit the protocols to one or
>>> another but it canot mix-and-match (hello 1.2, ok we will do 1.0) like Java
>>> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which is
>>> also on top of openssl.
>>>
>>> Today, I spent 2.5 hours with a lemming from MS getting basically
>>> nowhere. I really need an engineer, but they give me those clueless support
>>> people that is wasting mine and their time. If someone knows how to
>>> escalate or a forum where MS developers hang out, I would appreciate it.
>>> The support person I got today was clueless, went over a script and any
>>> attempt to explain a little more technical details led to total confusion
>>> and rebooting the script to beginning. Totally frustrating. At least with
>>> Oracle I got to talk with an actual engineer...
>>>
>>> George
>>>
>>>
>>> -Original Message-
>>> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
>>> Sent: Tuesday, October 13, 2015 2:03 PM
>>> To: Tomcat Users List
>>> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
>>> description = bad_record_mac
>>>
>>> -BEGIN PGP SIGNED MESSAGE-
>>> Hash: SHA256
>>>
>>> George,
>>>
>>> On 10/13/15 12:35 PM, George Stanchev wrote:
>>> > [1] states: " JDK 7-9 enables SSLv2Hello on the server side only.
>>> > (Will not send, but will accept SSLv2Hellos)"
>>>
>>> Interesting. This absolutely makes sense, though, since SSL should just
>>> die. :)
>>>
>>> > I've opened support case both MS and already there is a bug filed with
>>> > Oracle on this and really, to be absolutely certain if the issue is in
>>> > Java or SChannel, one would have to write a non-Java client that that
>>> > mimics the handshake messages send from Java with something like
>>> > OpenSSL or NSS or whatever and see if the bug replicates.
>>>
>>> It might be doable with OpenSSL s_client or something. Tough to
>>> replicate Java's behavior with a non-Java tool, though.
>>>
>>> > Writing a Java/Java server client could still leave some doubts as one
>>> > can argue the code reuse could mask the problem - it works but the bug
>>> > on the client side is hidden by the server containing similar/same bug
>>> > so the MACs check out...
>>> >
>>> > Unfortunately I don't have the time to invest in this more than I
>>> > already had. And if MS support engineers can pass it on to someone
>>> >

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

On 10/13/15 5:08 PM, Christopher Schultz wrote:
> George and Aurélien,
> 
> On 10/13/15 5:06 PM, George Stanchev wrote:
>> Try the dropbox location in my prev messages. It contains a
>> sample echo server you can use to test. It is a Visual Studio
>> 2013 project. In case you don’t have that, I've uploaded x64
>> executables under SSLServer_executbale.zip. It is pretty much
>> self contained, it has the redist DLLs, the server cert, all...
> 
>> It eliminates the need of IIS as it does the same thing - accept
>>  connection, read payload, upgrade to 2 way ssl...
> 
>> It generates output like this [1]
> 
>> The line " Decrypt error from SCHANNEL, Client ID: 
>> a1cefeb8-bad3-4903-8dbe-fdea347f666e" is emitted when bad record 
>> mac is caught on the server side...
> 
> On 10/13/15 4:55 PM, Aurélien Terrestris wrote:
>> "How do you force Java 8 to use SSLv2Hello?"
> 
>> You can do this when writing your own Java client : calling the 
>> SSLSocketFactory to create an SSLSocket and configure with 
>> setEnabledProtocols (
> 
> https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html
#s
>
> 
etEnabledProtocols-java.lang.String:A-
>> )
> 
>> If you have some IIS server on internet which reproduces the 
>> problem, I'll try with JTouch ( jtouch.sourceforge.net ) or write
>> a small client.
> 
> I've got a client already written. I'll post the latest code
> somewhere.

https://wiki.apache.org/tomcat/tools/SSLTest.java
and
https://wiki.apache.org/tomcat/tools/SSLUtils.java

This tool only performs the SSL handshake, so it doesn't do anything
at the HTTP level. It doesn't do client certificates; I'd be happy to
see someone add that capability and update the code on the Wiki.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJWHXaUAAoJEBzwKT+lPKRYTgcQAKdzEBtjLyCbkBBlJJObUxm0
dxAhLIFfBkcmo0IquMUJBnCHJ9hfuuhnXDvzLhvvAhfoiG8CkGfcjw79LwcL49iO
+eAuz1bycFvfSoVW+ZEPIXl13n4v+t2hBYfGJJgcYefOBJBfIGLHTazmi53F9vfn
zrnno6wRZkZ+JrFrzSlfMMvWnOE7Uv5ZDKw/8tv85ZTPuIIdIud8Uy0Z5J3HHgYx
t5WBL3JaqLcUihIUUuibSfmw27iyfKPv6OusoVPKc9pKb+uWoLftyE4d4btVTFEV
qDAdmW3XaA/1GHZ3n5szHND+EkdKkiiic7mHRgsN+4F/AH3KVoWs9+Qw9vOhO+da
XgeehuKB1nfnNrhI1n4eEoYC+P162GxVqNKIsKFat38BGEZlCRaAt8UtsfhE0AvO
2gaF1k1vyYcCoXhNYF9WQLatnx1MXeAZMYw3oAZPOusDoqtGElCJfYc1K8iJayEl
bBzYojkzLjFqPfJJ8tvCqBwkngNCw6MgyqVbYSMxDREQtZBa2xYGLHv9bWL0Kp+6
Q5Y3u4uks7OCGH+Usfvq82ZGnp6mnneA2jgAd0KWSJDjKcaNUWQVOirPVTFnpNmh
+dUZUM/gR1CIQHoiu8i8NH9qn2jtBtDvnYRWswzJlt9tOpaxPQWwdBL86gj4cWHp
aGwTUveYwOgLokoEUXHO
=Iha7
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

Just as a side note, https.protocols is read by HttpsUrlConnection which feeds 
it down through setEnabledProtocols() on the SSL socket. " 
jdk.tls.client.protocols" is read directly by JSSE and does the same thing...

Another route to try the SSLv2Hello is to hack and recompile the JSSE. It's on 
my agenda to learn to do that if possible without rebuilding the whole JRE 
since for some obscure reason Oracle compiles JSSE with no debug info (and it 
is not included in the srcs distributed with the JRE) on it which makes 
debugging and inspecting local vars really hard...

George

-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com] 
Sent: Tuesday, October 13, 2015 3:13 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

Maybe writing too fast..

"How do you force Java 8 to use SSLv2Hello?"

As suggested before, by writing your own client OR by trying this hack.
From the JSSE Ref Guide 5, we know how to the customize the protocol ( 
https://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefGuide.html#InstallationAndCustomization
) by setting a system property (https.protocol). Although they are no more 
talking about this in the Ref Guide 8 ( 
https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#SunJSSEProvider
) I would give it a try as I know that the documentation is poorly written.
I suggested 10 years ago a change in the API doc about the enabled protocols, 
and they didn't change anything although they said they would.



"[1] states: " JDK 7-9 enables SSLv2Hello on the server side only.  (Will not 
send, but will accept SSLv2Hellos)""

I believe they mean "by default" as for the client side. Poorly written, 
probably.



2015-10-13 22:55 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:

> "How do you force Java 8 to use SSLv2Hello?"
>
> You can do this when writing your own Java client : calling the 
> SSLSocketFactory to create an SSLSocket and configure with 
> setEnabledProtocols (
> https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html
> #setEnabledProtocols-java.lang.String:A-
> )
>
> If you have some IIS server on internet which reproduces the problem, 
> I'll try with JTouch ( jtouch.sourceforge.net ) or write a small client.
>
>
>
>
> 2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
>
>> George,
>>
>> do you have any network capture that we can see ?
>>
>> 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
>>
>>> >> It might be doable with OpenSSL s_client or something. Tough to
>>> replicate Java's behavior with a non-Java tool, though.
>>>
>>> I tried hard with the s_client but it can limit the protocols to one 
>>> or another but it canot mix-and-match (hello 1.2, ok we will do 1.0) 
>>> like Java
>>> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which 
>>> is also on top of openssl.
>>>
>>> Today, I spent 2.5 hours with a lemming from MS getting basically 
>>> nowhere. I really need an engineer, but they give me those clueless 
>>> support people that is wasting mine and their time. If someone knows 
>>> how to escalate or a forum where MS developers hang out, I would appreciate 
>>> it.
>>> The support person I got today was clueless, went over a script and 
>>> any attempt to explain a little more technical details led to total 
>>> confusion and rebooting the script to beginning. Totally 
>>> frustrating. At least with Oracle I got to talk with an actual engineer...
>>>
>>> George
>>>
>>>
>>> -Original Message-
>>> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
>>> Sent: Tuesday, October 13, 2015 2:03 PM
>>> To: Tomcat Users List
>>> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
>>> description = bad_record_mac
>>>
>>> -BEGIN PGP SIGNED MESSAGE-
>>> Hash: SHA256
>>>
>>> George,
>>>
>>> On 10/13/15 12:35 PM, George Stanchev wrote:
>>> > [1] states: " JDK 7-9 enables SSLv2Hello on the server side only.
>>> > (Will not send, but will accept SSLv2Hellos)"
>>>
>>> Interesting. This absolutely makes sense, though, since SSL should 
>>> just die. :)
>>>
>>> > I've opened support case both MS and already there is a bug filed 
>>> > with Oracle on this and really, to be absolutely certain if the 
>>> > issue is in Java or SChan

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Aurélien Terrestris]

"Ok, may be you are ahead of me on this one, but setEnabledProtocols(), to
my understanding does the same as the system property
"jdk.tls.client.protocols". It gives the JSSE a pool of protocols to choose
from but it will be up to JSSE to select the ClientHello version. I might
have missed something but I spent quite a bit of time in the Handshaker and
related classes in the JDK and couldn't see anything that can change
that..."

Sorry, but I believe that no. To understand, you need to look to the old
Ref Guide (
https://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefGuide.html#InstallationAndCustomization
) which in "Annex A" says that you context.getInstance() takes one argument
called "protocol" while sslsocket.setEnabledProtocols() takes a list of
protocols including the SSLv2Hello.
If you are about to write a TLS client using a SSLv2Hello, you will call
getInstance("TLS") and setEnabledProtocols("SSLv2").

I hope things are more understandable :)




2015-10-13 23:12 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> Ok, may be you are ahead of me on this one, but setEnabledProtocols(), to
> my understanding does the same as the system property
> "jdk.tls.client.protocols". It gives the JSSE a pool of protocols to choose
> from but it will be up to JSSE to select the ClientHello version. I might
> have missed something but I spent quite a bit of time in the Handshaker and
> related classes in the JDK and couldn't see anything that can change that...
>
> George
>
> -Original Message-
> From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
> Sent: Tuesday, October 13, 2015 2:55 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
> description = bad_record_mac
>
> "How do you force Java 8 to use SSLv2Hello?"
>
> You can do this when writing your own Java client : calling the
> SSLSocketFactory to create an SSLSocket and configure with
> setEnabledProtocols (
>
> https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html#setEnabledProtocols-java.lang.String:A-
> )
>
> If you have some IIS server on internet which reproduces the problem, I'll
> try with JTouch ( jtouch.sourceforge.net ) or write a small client.
>
>
>
>
> 2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
>
> > George,
> >
> > do you have any network capture that we can see ?
> >
> > 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
> >
> >> >> It might be doable with OpenSSL s_client or something. Tough to
> >> replicate Java's behavior with a non-Java tool, though.
> >>
> >> I tried hard with the s_client but it can limit the protocols to one
> >> or another but it canot mix-and-match (hello 1.2, ok we will do 1.0)
> >> like Java
> >> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl which
> >> is also on top of openssl.
> >>
> >> Today, I spent 2.5 hours with a lemming from MS getting basically
> >> nowhere. I really need an engineer, but they give me those clueless
> >> support people that is wasting mine and their time. If someone knows
> >> how to escalate or a forum where MS developers hang out, I would
> appreciate it.
> >> The support person I got today was clueless, went over a script and
> >> any attempt to explain a little more technical details led to total
> >> confusion and rebooting the script to beginning. Totally frustrating.
> >> At least with Oracle I got to talk with an actual engineer...
> >>
> >> George
> >>
> >>
> >> -Original Message-
> >> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> >> Sent: Tuesday, October 13, 2015 2:03 PM
> >> To: Tomcat Users List
> >> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal,
> >> description = bad_record_mac
> >>
> >> -BEGIN PGP SIGNED MESSAGE-
> >> Hash: SHA256
> >>
> >> George,
> >>
> >> On 10/13/15 12:35 PM, George Stanchev wrote:
> >> > [1] states: " JDK 7-9 enables SSLv2Hello on the server side only.
> >> > (Will not send, but will accept SSLv2Hellos)"
> >>
> >> Interesting. This absolutely makes sense, though, since SSL should
> >> just die. :)
> >>
> >> > I've opened support case both MS and already there is a bug filed
> >> > with Oracle on this and really, to be absolutely certain if the
> >> > issue is in Java or SChannel, one would have to writ

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

From the source of truth: sun.security.ssl.ClientHandshaker.java of the latest 
JDK 8 update 60

this.enabledProtocols = [SSLv2Hello, TLSv1, TLSv1.1, TLSv1.2]
this. maxProtocolVersion = TLSv1.2


// Not a TLS1.2+ handshake
// For SSLv2Hello, HandshakeHash.reset() will be called, so we
// cannot call HandshakeHash.protocolDetermined() here. As it does
// not follow the spec that HandshakeHash.reset() can be only be
// called before protocolDetermined.
// if (maxProtocolVersion.v < ProtocolVersion.TLS12.v) {
// handshakeHash.protocolDetermined(maxProtocolVersion);
// }

// create the ClientHello message
ClientHello clientHelloMessage = new ClientHello(
sslContext.getSecureRandom(), maxProtocolVersion,
sessionId, cipherSuites);


I just stepped through the code, there is absolutely no way to override this 
without hacking JSSE...

/sigh

George

-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com] 
Sent: Tuesday, October 13, 2015 3:41 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

"Ok, may be you are ahead of me on this one, but setEnabledProtocols(), to my 
understanding does the same as the system property "jdk.tls.client.protocols". 
It gives the JSSE a pool of protocols to choose from but it will be up to JSSE 
to select the ClientHello version. I might have missed something but I spent 
quite a bit of time in the Handshaker and related classes in the JDK and 
couldn't see anything that can change that..."

Sorry, but I believe that no. To understand, you need to look to the old Ref 
Guide ( 
https://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefGuide.html#InstallationAndCustomization
) which in "Annex A" says that you context.getInstance() takes one argument 
called "protocol" while sslsocket.setEnabledProtocols() takes a list of 
protocols including the SSLv2Hello.
If you are about to write a TLS client using a SSLv2Hello, you will call
getInstance("TLS") and setEnabledProtocols("SSLv2").

I hope things are more understandable :)




2015-10-13 23:12 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> Ok, may be you are ahead of me on this one, but setEnabledProtocols(), 
> to my understanding does the same as the system property 
> "jdk.tls.client.protocols". It gives the JSSE a pool of protocols to 
> choose from but it will be up to JSSE to select the ClientHello 
> version. I might have missed something but I spent quite a bit of time 
> in the Handshaker and related classes in the JDK and couldn't see anything 
> that can change that...
>
> George
>
> -Original Message-
> From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
> Sent: Tuesday, October 13, 2015 2:55 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
> description = bad_record_mac
>
> "How do you force Java 8 to use SSLv2Hello?"
>
> You can do this when writing your own Java client : calling the 
> SSLSocketFactory to create an SSLSocket and configure with 
> setEnabledProtocols (
>
> https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html
> #setEnabledProtocols-java.lang.String:A-
> )
>
> If you have some IIS server on internet which reproduces the problem, 
> I'll try with JTouch ( jtouch.sourceforge.net ) or write a small client.
>
>
>
>
> 2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
>
> > George,
> >
> > do you have any network capture that we can see ?
> >
> > 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
> >
> >> >> It might be doable with OpenSSL s_client or something. Tough to
> >> replicate Java's behavior with a non-Java tool, though.
> >>
> >> I tried hard with the s_client but it can limit the protocols to 
> >> one or another but it canot mix-and-match (hello 1.2, ok we will do 
> >> 1.0) like Java
> >> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl 
> >> which is also on top of openssl.
> >>
> >> Today, I spent 2.5 hours with a lemming from MS getting basically 
> >> nowhere. I really need an engineer, but they give me those clueless 
> >> support people that is wasting mine and their time. If someone 
> >> knows how to escalate or a forum where MS developers hang out, I 
> >> would
> appreciate it.
> >> The support person I got today was clueless, went over a script and 
> >> any attempt to explain a little more technical details led to total 
> >> con

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

May be I am mistaken. I will give jtouch a try, thanks for the pointers...at 
this point I am grasping at straws :)

Thanks Aurelien!

-Original Message-
From: Aurélien Terrestris [mailto:aterrest...@gmail.com] 
Sent: Tuesday, October 13, 2015 3:52 PM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

"Another route to try the SSLv2Hello is to hack and recompile the JSSE.
It's on my agenda to learn to do that if possible without rebuilding the whole 
JRE since for some obscure reason Oracle compiles JSSE with no debug info (and 
it is not included in the srcs distributed with the JRE) on it which makes 
debugging and inspecting local vars really hard..."

No, it's not needed. I use jtouch.sourceforge.net and it's working well for 
trying SSLv2 hello's. From the network captures, I see that it's working.
Not making advertisement for my software here, but,.. ;)







2015-10-13 23:20 GMT+02:00 George Stanchev <gstanc...@serena.com>:

> Just as a side note, https.protocols is read by HttpsUrlConnection 
> which feeds it down through setEnabledProtocols() on the SSL socket. "
> jdk.tls.client.protocols" is read directly by JSSE and does the same 
> thing...
>
> Another route to try the SSLv2Hello is to hack and recompile the JSSE.
> It's on my agenda to learn to do that if possible without rebuilding 
> the whole JRE since for some obscure reason Oracle compiles JSSE with 
> no debug info (and it is not included in the srcs distributed with the 
> JRE) on it which makes debugging and inspecting local vars really hard...
>
> George
>
> -Original Message-
> From: Aurélien Terrestris [mailto:aterrest...@gmail.com]
> Sent: Tuesday, October 13, 2015 3:13 PM
> To: Tomcat Users List
> Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, 
> description = bad_record_mac
>
> Maybe writing too fast..
>
> "How do you force Java 8 to use SSLv2Hello?"
>
> As suggested before, by writing your own client OR by trying this hack.
> From the JSSE Ref Guide 5, we know how to the customize the protocol ( 
> https://docs.oracle.com/javase/1.5.0/docs/guide/security/jsse/JSSERefG
> uide.html#InstallationAndCustomization
> ) by setting a system property (https.protocol). Although they are no 
> more talking about this in the Ref Guide 8 ( 
> https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunPro
> viders.html#SunJSSEProvider
> ) I would give it a try as I know that the documentation is poorly written.
> I suggested 10 years ago a change in the API doc about the enabled 
> protocols, and they didn't change anything although they said they would.
>
>
>
> "[1] states: " JDK 7-9 enables SSLv2Hello on the server side only.  
> (Will not send, but will accept SSLv2Hellos)""
>
> I believe they mean "by default" as for the client side. Poorly 
> written, probably.
>
>
>
> 2015-10-13 22:55 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
>
> > "How do you force Java 8 to use SSLv2Hello?"
> >
> > You can do this when writing your own Java client : calling the 
> > SSLSocketFactory to create an SSLSocket and configure with 
> > setEnabledProtocols ( 
> > https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.ht
> > ml
> > #setEnabledProtocols-java.lang.String:A-
> > )
> >
> > If you have some IIS server on internet which reproduces the 
> > problem, I'll try with JTouch ( jtouch.sourceforge.net ) or write a small 
> > client.
> >
> >
> >
> >
> > 2015-10-13 22:22 GMT+02:00 Aurélien Terrestris <aterrest...@gmail.com>:
> >
> >> George,
> >>
> >> do you have any network capture that we can see ?
> >>
> >> 2015-10-13 22:10 GMT+02:00 George Stanchev <gstanc...@serena.com>:
> >>
> >>> >> It might be doable with OpenSSL s_client or something. Tough to
> >>> replicate Java's behavior with a non-Java tool, though.
> >>>
> >>> I tried hard with the s_client but it can limit the protocols to 
> >>> one or another but it canot mix-and-match (hello 1.2, ok we will 
> >>> do 1.0) like Java
> >>> 8 does. Either TLSv1 or TLSv1.2 but not both. Neither can curl 
> >>> which is also on top of openssl.
> >>>
> >>> Today, I spent 2.5 hours with a lemming from MS getting basically 
> >>> nowhere. I really need an engineer, but they give me those 
> >>> clueless support people that is wasting mine and their time. If 
> >>> someone knows how to escalate or a forum where MS develop

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

2way SSL that you can work around by turning off TLSv1.2 or RSA key exchanges.

-Original Message-
From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
Sent: Monday, August 31, 2015 11:15 AM
To: Tomcat Users List
Subject: Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Diarmuid,

On 8/31/15 11:36 AM, dmccrthy wrote:
> To cut a long story short openssl helped. Using openssl -connect 
> showed the error below. When I added the -tls1 flag the error went 
> away.
> 
> 2104:error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or 
> bad record mac:.\ssl\s3_pkt.c:532:
> 
> Configuring Tomcat with the JVM parameter below resolved the issue.
> 
> -Dhttps.protocols="TLSv1"

Yep, you have to use a TLS handshake; the older SSL3 handshake won't wor k.

Now... you *can* enable SSLv2Hello and then only support TLSv1+, but let's face 
it: SSL is dead and clients should get with the program.

> This may limit our client Web application but I have very little 
> influence over the server-side application, so it'll have to do as a 
> solution for now.

Your client should not try to use SSL handshake unless you know it's required.

> So the issue seems to be a Java handshake error whereby it has a 
> problem downgrading from TLSv1.2 to TLSv1 during the handshake.

I don't know enough about TLS to know if there is an appreciable difference 
between the handshake protocol between the two. I know that
SSLv2 was the lowest common denominator for a while, and most servers would 
enable it *just for the handshake* but would refuse to actually negotiate an 
SSLv2 connection (SSLv2 was basically DOA). These days, nobody should be using 
SSL at all, but I might understand wanting to use SSL-compatible handshakes, so 
using "SSLv2Hello" but not supporting the SSLv3 protocol would be the way to do 
that.

Again, we should just let SSL die. The more we push clients and servers to only 
negotiate TLS from here on out, the better off Internet security will be 
overall.

> Why it works in our non-prod environment with the same versions of 
> everything is a mystery, but I can speculate at some difference in 
> underlying O/S settings, or perhaps a Java bug.

Or more likely a configuration option you didn't detect when you checked 
everything out.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJV5IudAAoJEBzwKT+lPKRY+xsP/RBgD3VErGPgLZIOu6GDdUn9
4e/yH5kVAdiuOcnn26En+OJKGONK5c9Hf9gW23utPw2ruG1KYwSkr9dJ4yFIiKxh
1wYhq4oTN3j8DwNb8C2Y522eyH9hRYo+W8S7er0pvklcGiImgx+vp/LYitM2dHeG
6MEPJ03+/5CLbrmYfLbcZvnPydqKYVJtIqozHGhrC5+fAbcFFsWBcwrfk0Menwji
wE4w7o5yi38/MtXX1b3FtMnvyjfiKHdPoKJ4PJv8KA3hKjWzEVLiehFGEQGb9VS6
uqgMLB5YR3QrI0yYN53soi4apgqktcfwGimK+B5Ar/2uvsarLvZgIF8uYuEHXTYY
bn+V7ngmRxuwX71VUJDP47MtwQsgxko4H1qa2jIh7Pp8kRHeHwKytJxOLPUepmKp
K1rmii+OtVPMQusXGVJedohXMwOCi3kUf9exmadnN02EgAPmIh/rfy7uBARfYJdC
CqjiQxmTCm5yKu1uB9PXFbZMiCFjq+Hw3g6/cVnn1X5nxlfkPBCAODk5UouRrO+w
oIErBEAbLSb7twWo8S6B9pupb/8C4RNvdhPk9pCfPf2cC+Ouybg4Sz/S4KSQ1N8P
0zfXVoDL4AJtl3k1ylXbfPVRE5SY+ZMqT4fC3odI1frWGI3oSnnXNQsqxr4ehmpo
PUDzJVZbKxLMruMEbnSh
=OrTI
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[George Stanchev]

One more clarification: on point [6] below I stated that Java is able to 
recover with a retry on a cached connection. Unfortunately that is only valid 
for higher level classes like HttpUrlConnection which makes 1 retry on 
IOException (and only on a GET and some other conditions). The lower-level 
sockets just throw and that’s it...

-Original Message-
From: George Stanchev [mailto:gstanc...@serena.com] 
Sent: Friday, October 09, 2015 10:40 AM
To: Tomcat Users List
Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description 
= bad_record_mac

Just for the record, https.protocols is a property used by the 
HttpsUrlConnection class. If your app is using a client that doesn't rely on 
the internal Oracle HTTP client, it's better to use " jdk.tls.client.protocols" 
which is read directly by the socket/SSL classes. Apache Http Client is one 
client that does use sockets directly instead of HttpsUrlConnection.


Also, I want to mention that we have finally been able to isolate a really 
nasty problem with Java SSL implementation and Windows's SSPI based SChannel 
acting as a server. The issue we ran into is very similar as to what the OP 
reported, that’s why I asked him a while ago on a separate thread about 
keystores if the server is IIS-based. The issue that we ran into might sound 
esoteric but our product ran into it very often. We have a configuration where 
we lock our intracomponent communication via 2way SSL. We have Java 
Tomcat-based server components that talk to IIS based native components via web 
services over HTTP which we secure with 2way SSL. When we upgraded to JRE 8 we 
started getting those connection resets in Tomcat when we enabled the 2way SSL. 
Those were on the Java->IIS calls that work over regular SSL. On the java side, 
if you turn on the ssl debug on, you'd see connection reset IOException thrown, 
and on the Windows side, you'd see " The following fatal alert was generated: 
20. The internal error state is 960." In the System event log. This error is " 
TLS1_ALERT_BAD_RECORD_MAC" from SSPI. A search in Oracle's bug database found a 
close resemblance to this issue: 
https://bugs.openjdk.java.net/browse/JDK-8068812. But the reporter couldn't 
reproduce it and also was against another Oracle component Java Web Download or 
something like this. So we started working with Oracle but the intermittent and 
hard-to-reproduce nature of this issue hampered the investigation. As the OP 
mentioned below, there are apocryphal reports for this all over the web but 
people cannot troubleshoot it very well. Anyway here are my findings. In order 
for the issue to arise all those conditions must be met:

[1] The client must start with ClientHello TLSv1.2 [2] The server must respond 
with ServerHello TLSv1 (so the server should not have TLS1.2 enabled) [3] The 
cipher key exchange must NOT be ECDHE. I have not tried DHE as I could not make 
Windows take it. According to this doc [1] it should have several DHE key 
exchange ciphers available but I could not make it work. RSA key exchange 
exhibits the issue. I have been using “TLS_RSA_WITH_AES_256_CBC_SHA” to 
reproduce.
[4] The initial connection must not be 2way-SSL [5] The server must upgrade the 
connection after encrypted payload is read. In case of IIS, it has 2way SSL 
setting on a Virtual Directory. The initial handshake is not 2way SSL, once the 
HTTP request is read, it determines that the requested resource (/gsoap) is 
2way SSL-protected and upgrades the connection to 2way SSL.
[6] The server must kill the connection abruptly when error happens. This is 
important as if the server terminates the connection gracefully and keeps it 
around Java is able to recover since it reuses the cached connection. IIS does 
exactly this.
[7] The server must be Windows 7, Win2k8, Win2k8-R2, Win8. Might be other 
versions affected but bug is not evident on Win2k12 or Win2k12-R2 or Win10. 
Perhaps it was fixed, perhaps those versions support TLSv1.2

It might sound complicated but it is not, it is very common. All you have to do 
is to have a Virtual Directory in IIS that requires 2way SSL and is set to 
"want". RSA-based key exchanges are one of the most commonly used ciphers. The 
Windows platforms we tested with are still widely used in the enterprises while 
the adoption rate of win2k12+ is still lagging (from our experience with our 
customers). And you have equal "chance" of cipher support between Windows 
servers and Java 8 clients to land on RSA based cipher (which exhibits the 
issue) or ECDHE key exchange that is ok. Again, it might sound too complicated 
but it was blocker for us to ship our product.

For us, to work around this issue is for the 2way SSL to turn off TLSv1.2 (via 
"-Djdk.tls.client.protocols=TLSv1" or RSA key exchange algorithm (via the java 
security properties) on affected systems which fixes it.

I have a lot more technical det

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[dmccrthy]

Hi Chris,

To cut a long story short openssl helped. Using openssl -connect showed the
error below. When I added the -tls1 flag the error went away.

2104:error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad
record mac:.\ssl\s3_pkt.c:532:

Configuring Tomcat with the JVM parameter below resolved the issue.

-Dhttps.protocols="TLSv1"

This may limit our client Web application but I have very little influence
over the server-side application, so it'll have to do as a solution for
now.

So the issue seems to be a Java handshake error whereby it has a problem
downgrading from TLSv1.2 to TLSv1 during the handshake. Why it works in our
non-prod environment with the same versions of everything is a mystery, but
I can speculate at some difference in underlying O/S settings, or perhaps a
Java bug.

Many thanks for your patience and advice.

Diarmuid
On 28 Aug 2015 20:46, "Christopher Schultz" 
wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Diarmuid,
>
> On 8/28/15 3:25 PM, dmccrthy wrote:
> > Thanks again for replying. I agree it doesn't look like a Tomcat
> > issue and would be happy to close the issue. I've included
> > responses to your points below prefixed with [DMC] for my initials,
> > but think I need to focus on the Java side of things and perhaps
> > open an Oracle ticket.
>
> Feel free to continue the conversation here. I just marked it as "off
> topic" because it's not directly-related to Tomcat. This could have
> happened in any other application server, for instance.
>
> There's nothing wrong with having an off-topic discussion, here. I
> just wanted everyone to know that this wasn't directly-related to Tomcat
> .
>
> See below.
>
> > On 8/27/15 4:31 PM, dmccrthy wrote:
>  Thanks for responding so quickly. My apologies, I should have
>  been clearer on the topology. We have a Tomcat instance with
>  a 3rd party web app deployed on it (the "Tomcat client")
>  running on Windows 2008 . This connects via HTTPS to a 3rd
>  party service running behind IBM Http Server on Solaris 10.
> >
> > Okay, so, just to confirm:
> >
> > Tomcat (app) -> HTTPS -> 3rd party service
> >
> > Configurations and results:
> >
> > 0. prod Tomcat -> prod 3rd-party service FAILS with
> > TLS_RSA_WITH_AES_128_CBC_SHA
> >
> >> [DMC]  Correct
>
> Ok.
>
> > (do other ciphers work? or are you completely dead in the water?)
> >
> >> [DMC] No, this is the next step. Our production environment is
> >> completely locked down and we have to exhaust all other avenues
> >> before making speculative changes.
>
> I suppose what I meant was: do 100% of your connections to this
> 3rd-party service fail because this one cipher suite seems to be
> broken, or do you have some connections that work and others that do
> not? If some work, do they have anything in common with each other?
>
> > 2. prod Tomcat -> prod 3rd-party service with lower version of
> > Tomcat and/or Java WORKS with TLS_RSA_WITH_AES_128_CBC_SHA
> >
> >> [DMC] Correct
>
> So, downgrading does in fact improve the situation. Can you try
> upgrading Java in one of your working production environments but
> leave Tomcat alone? That would definitely rule-out Tomcat as a problem.
>
> > 1. test Tocmat -> test 3rd-party service WORKS with
> > TLS_RSA_WITH_AES_128_CBC_SHA
> >
> >> [DMC] Correct
>
> Good. What version was this running again? Same Java/Tomcat versions
> as the production server which seems to fail?
>
> > 2. test client (command-line?) on production host -> prod 3rd
> > parts service WORKS with TLS_RSA_WITH_AES_128_CBC_SHA
> >
> >> [DMC] Correct. The command line test client is a simple Java
> >> class that just opens an InputStream to the 3rd party service
>
> So: same Java, same host, same client library: works. But within
> Tomcat on that host with that Java with that same client library:
> fails? :(
>
> > No changes are known/expected, other than Tomcat and Java version.
> >
> >> [DMC] It's a greenfield app and no changes are planned bar
> >> functional patches. The same version of operating system, java,
> >> tomcat, and the application is in our non-prod environment and
> >> works ok
>
> There is obviously a difference. We just have to figure out what it is.
>
> What are your JVM parameters to launch Tomcat? You can find those out
> using "jinfo" from the command-line if you have the process id (at
> least I *think* that works on Windows). Check the running production
> application as well as the CLI client running on the same machine.
> Quadruple-check the Java versions of those processes.
>
>  Two additional pieces of analysis that came out of our
>  investigations today are:
> 
>  1) we are using NextGen Cisco devices. These are application
>  and protocol aware. This means that the firewall might allow
>  allow our standalone Java client to connect using HTTPS, but
>  interfere with the HTTPS SOAP requests from our Tomcat/3rd
>  party app "Tomcat 

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Diarmuid,

On 8/31/15 11:36 AM, dmccrthy wrote:
> To cut a long story short openssl helped. Using openssl -connect
> showed the error below. When I added the -tls1 flag the error went
> away.
> 
> 2104:error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed
> or bad record mac:.\ssl\s3_pkt.c:532:
> 
> Configuring Tomcat with the JVM parameter below resolved the
> issue.
> 
> -Dhttps.protocols="TLSv1"

Yep, you have to use a TLS handshake; the older SSL3 handshake won't wor
k.

Now... you *can* enable SSLv2Hello and then only support TLSv1+, but
let's face it: SSL is dead and clients should get with the program.

> This may limit our client Web application but I have very little
> influence over the server-side application, so it'll have to do as
> a solution for now.

Your client should not try to use SSL handshake unless you know it's
required.

> So the issue seems to be a Java handshake error whereby it has a
> problem downgrading from TLSv1.2 to TLSv1 during the handshake.

I don't know enough about TLS to know if there is an appreciable
difference between the handshake protocol between the two. I know that
SSLv2 was the lowest common denominator for a while, and most servers
would enable it *just for the handshake* but would refuse to actually
negotiate an SSLv2 connection (SSLv2 was basically DOA). These days,
nobody should be using SSL at all, but I might understand wanting to
use SSL-compatible handshakes, so using "SSLv2Hello" but not
supporting the SSLv3 protocol would be the way to do that.

Again, we should just let SSL die. The more we push clients and
servers to only negotiate TLS from here on out, the better off
Internet security will be overall.

> Why it works in our non-prod environment with the same versions of 
> everything is a mystery, but I can speculate at some difference in 
> underlying O/S settings, or perhaps a Java bug.

Or more likely a configuration option you didn't detect when you
checked everything out.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJV5IudAAoJEBzwKT+lPKRY+xsP/RBgD3VErGPgLZIOu6GDdUn9
4e/yH5kVAdiuOcnn26En+OJKGONK5c9Hf9gW23utPw2ruG1KYwSkr9dJ4yFIiKxh
1wYhq4oTN3j8DwNb8C2Y522eyH9hRYo+W8S7er0pvklcGiImgx+vp/LYitM2dHeG
6MEPJ03+/5CLbrmYfLbcZvnPydqKYVJtIqozHGhrC5+fAbcFFsWBcwrfk0Menwji
wE4w7o5yi38/MtXX1b3FtMnvyjfiKHdPoKJ4PJv8KA3hKjWzEVLiehFGEQGb9VS6
uqgMLB5YR3QrI0yYN53soi4apgqktcfwGimK+B5Ar/2uvsarLvZgIF8uYuEHXTYY
bn+V7ngmRxuwX71VUJDP47MtwQsgxko4H1qa2jIh7Pp8kRHeHwKytJxOLPUepmKp
K1rmii+OtVPMQusXGVJedohXMwOCi3kUf9exmadnN02EgAPmIh/rfy7uBARfYJdC
CqjiQxmTCm5yKu1uB9PXFbZMiCFjq+Hw3g6/cVnn1X5nxlfkPBCAODk5UouRrO+w
oIErBEAbLSb7twWo8S6B9pupb/8C4RNvdhPk9pCfPf2cC+Ouybg4Sz/S4KSQ1N8P
0zfXVoDL4AJtl3k1ylXbfPVRE5SY+ZMqT4fC3odI1frWGI3oSnnXNQsqxr4ehmpo
PUDzJVZbKxLMruMEbnSh
=OrTI
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Diarmuid,

I'm marking this as off-topic because it seems like Tomcat really has
nothing to do with the situation.

On 8/27/15 4:31 PM, dmccrthy wrote:
 Thanks for responding so quickly. My apologies, I should have been
 clearer on the topology. We have a Tomcat instance with a 3rd party
 web app deployed on it (the Tomcat client) running on Windows
 2008 . This connects via HTTPS to a 3rd party service running
 behind IBM Http Server on Solaris 10.

Okay, so, just to confirm:

Tomcat (app) - HTTPS - 3rd party service

Configurations and results:

0. prod Tomcat - prod 3rd-party service
   FAILS with TLS_RSA_WITH_AES_128_CBC_SHA

   (do other ciphers work? or are you completely dead in the water?)

2. prod Tomcat - prod 3rd-party service
   with lower version of Tomcat and/or Java
   WORKS with TLS_RSA_WITH_AES_128_CBC_SHA

1. test Tocmat - test 3rd-party service
   WORKS with TLS_RSA_WITH_AES_128_CBC_SHA

2. test client (command-line?) on production host - prod 3rd parts
   service
   WORKS with TLS_RSA_WITH_AES_128_CBC_SHA

No changes are known/expected, other than Tomcat and Java version.

 So the tool we're using is our 3rd party client Web app (the
 Tomcat client). It shows the same negotiation from TLSv1.2 down
 to TLSv1. My reading of your comments is that the
 truststore/keystore has no bearing here because it's an outgoing
 client connection and there are no inbound connections at play.

Exactly. Neither the keystore nor the truststore have any bearing,
here. If the client library itself uses (or supports) a
truststore/keystore, then you'll likely know about it because you'd
have to make a call like client.setTruststore(3rdparty.jks) or whateve
r.

 * I don't know what the highest supported cipher suite is on either
 client or server hosts. I do know that TLSv1 is the highest
 supported by the 3rd party server.

Good to know, but not terribly relevant. The client and server should
negotiate a common cipher suite based upon those available on either
end. If they can't agree, the handshake fails.

 * we're not using JCE Unlimited Security because it hasn't been an
 issue and frankly I didn't think of it when we were doing the
 installations :)

Okay.

 Two additional pieces of analysis that came out of our
 investigations today are:
 
 1) we are using NextGen Cisco devices. These are application and
 protocol aware. This means that the firewall might allow allow our
 standalone Java client to connect using HTTPS, but interfere with
 the HTTPS SOAP requests from our Tomcat/3rd party app Tomcat
 client.

How can the device be application aware? How does the applicaiton
identify itself to the device? How is that authenticated? What is
stopping EvilApp.exe from claiming to be InnocentApp.exe?

 2) there are some anecdotes of Java 7 clients failing to downgrade
 from TLSv1.2 to TLSv1. For example 
 http://stackoverflow.com/questions/31579587/java-tls-1-2-downgrade-to-
tls-1-0-sometimes-error.

This
 
is not a piece of analysis. This is something you found online
that might be related. But you haven't collected any information to
support the idea that it's related. I note that the experience is
different in that SO article: the user is getting a different error at
a different part of the handshake.

 Some takeaway actions for me to explore are below. I'll welcome any
 other thoughts that you may have.
 
 1) compare the firewall rules on our working non-production client
 with the rules governing the failing production client

Firewalls are usually pretty dumb. I'd be surprised if the firewall is
breaking anything, here.

 2) use openssl from our client server and see what it shows

Yes, please do that. Why wasn't that the first thing you tried?

 3) consider forcing the Tomcat/3rd party Tomcat client to use
 TLSv1

This is a possibility, but you shouldn't have to hamper your own
application because there are some servers out there that are so old
they don't support TLSv1.2: you should always offer the best
encryption you can, and only fall-back when necessary.

 4) check other logs on the server side to see if other clients are
 failing TLS handshakes

You didn't do this already?

 5) investigate the supported ciphers on client and server

There is code available in the archives of this mailing list to do
that. Search for my name and ssl and poke-around.

 6) consider upgrading Tomcat (as a last resort)

I thought the initial problem was that an upgrade to Tomcat broke
something. I think it's more likely to be the Java upgrade that did
something, and not Tomcat itself.

Are you sure that the standalone test client is using the exact same
version of Java as the Tomcat server is?

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJV4KNUAAoJEBzwKT+lPKRYPVMQAILQm/n33Mxh7cbYF3u0VfNg
eodcwvrXxWynhDX61ZjUGNTAPbEAsReLIuCzZjJpmB+LIaBsg8qsxpNL4zu+W4YA
ZYB2wiORHpgqXVXG3JuX2tlzKGmAaF8YnlWnsjaw+taZNWOwVvFvJN4esdp8eYYw

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Diarmuid,

On 8/28/15 3:25 PM, dmccrthy wrote:
 Thanks again for replying. I agree it doesn't look like a Tomcat
 issue and would be happy to close the issue. I've included
 responses to your points below prefixed with [DMC] for my initials,
 but think I need to focus on the Java side of things and perhaps
 open an Oracle ticket.

Feel free to continue the conversation here. I just marked it as off
topic because it's not directly-related to Tomcat. This could have
happened in any other application server, for instance.

There's nothing wrong with having an off-topic discussion, here. I
just wanted everyone to know that this wasn't directly-related to Tomcat
.

See below.

 On 8/27/15 4:31 PM, dmccrthy wrote:
 Thanks for responding so quickly. My apologies, I should have
 been clearer on the topology. We have a Tomcat instance with
 a 3rd party web app deployed on it (the Tomcat client)
 running on Windows 2008 . This connects via HTTPS to a 3rd
 party service running behind IBM Http Server on Solaris 10.
 
 Okay, so, just to confirm:
 
 Tomcat (app) - HTTPS - 3rd party service
 
 Configurations and results:
 
 0. prod Tomcat - prod 3rd-party service FAILS with
 TLS_RSA_WITH_AES_128_CBC_SHA
 
 [DMC]  Correct

Ok.

 (do other ciphers work? or are you completely dead in the water?)
 
 [DMC] No, this is the next step. Our production environment is
 completely locked down and we have to exhaust all other avenues
 before making speculative changes.

I suppose what I meant was: do 100% of your connections to this
3rd-party service fail because this one cipher suite seems to be
broken, or do you have some connections that work and others that do
not? If some work, do they have anything in common with each other?

 2. prod Tomcat - prod 3rd-party service with lower version of
 Tomcat and/or Java WORKS with TLS_RSA_WITH_AES_128_CBC_SHA
 
 [DMC] Correct

So, downgrading does in fact improve the situation. Can you try
upgrading Java in one of your working production environments but
leave Tomcat alone? That would definitely rule-out Tomcat as a problem.

 1. test Tocmat - test 3rd-party service WORKS with
 TLS_RSA_WITH_AES_128_CBC_SHA
 
 [DMC] Correct

Good. What version was this running again? Same Java/Tomcat versions
as the production server which seems to fail?

 2. test client (command-line?) on production host - prod 3rd
 parts service WORKS with TLS_RSA_WITH_AES_128_CBC_SHA
 
 [DMC] Correct. The command line test client is a simple Java
 class that just opens an InputStream to the 3rd party service

So: same Java, same host, same client library: works. But within
Tomcat on that host with that Java with that same client library:
fails? :(

 No changes are known/expected, other than Tomcat and Java version.
 
 [DMC] It's a greenfield app and no changes are planned bar
 functional patches. The same version of operating system, java,
 tomcat, and the application is in our non-prod environment and
 works ok

There is obviously a difference. We just have to figure out what it is.

What are your JVM parameters to launch Tomcat? You can find those out
using jinfo from the command-line if you have the process id (at
least I *think* that works on Windows). Check the running production
application as well as the CLI client running on the same machine.
Quadruple-check the Java versions of those processes.

 Two additional pieces of analysis that came out of our 
 investigations today are:
 
 1) we are using NextGen Cisco devices. These are application
 and protocol aware. This means that the firewall might allow
 allow our standalone Java client to connect using HTTPS, but
 interfere with the HTTPS SOAP requests from our Tomcat/3rd
 party app Tomcat client.
 
 How can the device be application aware? How does the
 applicaiton identify itself to the device? How is that
 authenticated? What is stopping EvilApp.exe from claiming to be
 InnocentApp.exe?
 
 [DMC] the firewall devices check the tcp connections. I don't
 know how. I do know we had endless trouble from overzealous
 firewall guys putting restrictive rules in place. The impact of
 that was that, for example, we could telnet through the firewall
 but not establish a Sybase connection. In any case, we compared
 all the non-prod and prod firewall rules and they are configured
 the same way. We also had several debug sessions with the 
 firewall guys and no traffic is being blocked. So at least we've
 ruled that out.

Good.

 2) there are some anecdotes of Java 7 clients failing to
 downgrade from TLSv1.2 to TLSv1. For example 
 http://stackoverflow.com/questions/31579587/java-tls-1-2-downgrade-
to-

 
tls-1-0-sometimes-error.
 
 This
 
 is not a piece of analysis. This is something you found online 
 that might be related. But you haven't collected any information
 to support the idea that it's related. I note that the experience
 is different in that SO article: the user is getting a different
 error at a different 

Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac

[dmccrthy]

Hi Chris,

Thanks again for replying. I agree it doesn't look like a Tomcat issue and
would be happy to close the issue. I've included responses to your points
below prefixed with [DMC] for my initials, but think I need to focus on the
Java side of things and perhaps open an Oracle ticket.

Best regards,
Diarmuid
On 28 Aug 2015 19:08, Christopher Schultz ch...@christopherschultz.net
wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256

 Diarmuid,

 I'm marking this as off-topic because it seems like Tomcat really has
 nothing to do with the situation.

 On 8/27/15 4:31 PM, dmccrthy wrote:
  Thanks for responding so quickly. My apologies, I should have been
  clearer on the topology. We have a Tomcat instance with a 3rd party
  web app deployed on it (the Tomcat client) running on Windows
  2008 . This connects via HTTPS to a 3rd party service running
  behind IBM Http Server on Solaris 10.

 Okay, so, just to confirm:

 Tomcat (app) - HTTPS - 3rd party service

 Configurations and results:

 0. prod Tomcat - prod 3rd-party service
FAILS with TLS_RSA_WITH_AES_128_CBC_SHA

[DMC]  Correct

(do other ciphers work? or are you completely dead in the water?)

[DMC] No, this is the next step. Our production environment is completely
locked down and we have to exhaust all other avenues before making
speculative changes.

 2. prod Tomcat - prod 3rd-party service
with lower version of Tomcat and/or Java
WORKS with TLS_RSA_WITH_AES_128_CBC_SHA

[DMC] Correct

 1. test Tocmat - test 3rd-party service
WORKS with TLS_RSA_WITH_AES_128_CBC_SHA

[DMC] Correct

 2. test client (command-line?) on production host - prod 3rd parts
service
WORKS with TLS_RSA_WITH_AES_128_CBC_SHA

[DMC] Correct. The command line test client is a simple Java class that
just opens an InputStream to the 3rd party service

 No changes are known/expected, other than Tomcat and Java version.

[DMC] It's a greenfield app and no changes are planned bar functional
patches. The same version of operating system, java, tomcat, and the
application is in our non-prod environment and works ok

  So the tool we're using is our 3rd party client Web app (the
  Tomcat client). It shows the same negotiation from TLSv1.2 down
  to TLSv1. My reading of your comments is that the
  truststore/keystore has no bearing here because it's an outgoing
  client connection and there are no inbound connections at play.

 Exactly. Neither the keystore nor the truststore have any bearing,
 here. If the client library itself uses (or supports) a
 truststore/keystore, then you'll likely know about it because you'd
 have to make a call like client.setTruststore(3rdparty.jks) or whateve
 r.

  * I don't know what the highest supported cipher suite is on either
  client or server hosts. I do know that TLSv1 is the highest
  supported by the 3rd party server.

 Good to know, but not terribly relevant. The client and server should
 negotiate a common cipher suite based upon those available on either
 end. If they can't agree, the handshake fails.

  * we're not using JCE Unlimited Security because it hasn't been an
  issue and frankly I didn't think of it when we were doing the
  installations :)

 Okay.

  Two additional pieces of analysis that came out of our
  investigations today are:
 
  1) we are using NextGen Cisco devices. These are application and
  protocol aware. This means that the firewall might allow allow our
  standalone Java client to connect using HTTPS, but interfere with
  the HTTPS SOAP requests from our Tomcat/3rd party app Tomcat
  client.

 How can the device be application aware? How does the applicaiton
 identify itself to the device? How is that authenticated? What is
 stopping EvilApp.exe from claiming to be InnocentApp.exe?

[DMC] the firewall devices check the tcp connections. I don't know how. I
do know we had endless trouble from overzealous firewall guys putting
restrictive rules in place. The impact of that was that, for example, we
could telnet through the firewall but not establish a Sybase connection. In
any case, we compared all the non-prod and prod firewall rules and they are
configured the same way. We also had several debug sessions with the
firewall guys and no traffic is being blocked. So at least we've ruled that
out

  2) there are some anecdotes of Java 7 clients failing to downgrade
  from TLSv1.2 to TLSv1. For example
  http://stackoverflow.com/questions/31579587/java-tls-1-2-downgrade-to-
 tls-1-0-sometimes-error.

 This
 
 is not a piece of analysis. This is something you found online
 that might be related. But you haven't collected any information to
 support the idea that it's related. I note that the experience is
 different in that SO article: the user is getting a different error at
 a different part of the handshake.

[DMC] I agree. But in the spirit of trying different cipher suites, this is
something that may (or may not) be relevant

  Some takeaway actions for me to explore are below. I'll