> From: owner-openssl-us...@openssl.org On Behalf Of Jmail Clist > Sent: Wednesday, 08 June, 2011 14:34
> Basically, I have an IBM Datatpower appliance that cannot complete > a successful handshake with a F5 LTM (load balancer). After the client > and server hellos, i get a "Fatal Alert" Bad Record Mac". Can someone > explain this error more clearly and what are the possible causes along > with some tips on how to debug/troubleshoot this issue? I have also traces > available if anyone wants them. Please refer to frame 7 below for the error. (trimmed) > Frame 5 (192 bytes on wire, 192 bytes captured) from server contains > TLSv1 Record Layer: Handshake Protocol: Server Hello len 74 > TLSv1 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec > TLSv1 Record Layer: Handshake Protocol: Encrypted msg len 36 > Frame 6 (66 bytes on wire, 66 bytes captured) TCP-level ACK > Frame 7 (73 bytes on wire, 73 bytes captured) from client contains > TLSv1 Record Layer: Alert (Level: Fatal, Description: Bad Record MAC) The only case server should send ServerHello and immediately ChangeCipher like that is "resuming" a session, i.e., using the same handshake result (masterkey) for this connection as a previous one between same client and server. If this is wireshark (as it looks) click the box-plus to left of "Handshake Protocol: Server Hello" and see if you have SessionIDLength=32 and SessiondID=bunchofhex. (I *think* the only ciphersuite that makes encrypted Finish 36 is RSA_WITH_RC4_128_SHA, abbreviated just RC4-SHA to openssl; you might confirm that's what the trace shows selected.) Resume should be accepted by the server only if there was a previous connection and the client (correctly) remembers at least the (a?) sessionID. I think if there was a previous successful handshake, it is less likely (though not impossible) that something is damaging the data between the endpoints, as described by Eduardo Navarro. Maybe either the client or server is misremembering or misapplying the session so that encryption/MAC on the new connection fails. Note that openssl uses the "bad MAC" alert code for most(!) decryption failures as well, because an attacker could benefit from knowing which error was detected. But neither of these should occur if both ends are correct and the data isn't damaged between. Was there in fact a previous full-handshake connection from same client to same server? Can you configure or direct client to do only initial connection(s) not resume, or similarly on server (perhaps only for this client)? Do other clients work okay to same server, and can you verify if they are using resume? Do you know this client works to other server(s)? With resume? Can you try openssl commandline s_client to that server, with appropriate certs etc and -reconnect, or with -sess_out and then -sess_in using a scratch file? (But not sending any data over the connection if you get one, just Q+return or EOF.) ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
