It turned out that OpenSSL was encrypting wrong, but only because I have custom EVP_PKEY_METHOD and RSA_METHOD hooks and my implementation of evp_pkey_method_st::sign was not complete. In the end I discovered that I did not need to hook that function at all and can use the default implementation which led to my favourite coding activity - deleting code!
In fact I am beginning to think I do not need a custom EVP_PKEY_METHOD at all and RSA_METHOD is sufficient to route the actual private key operation through my code. Thank you for your help getting me to the answer. From: owner-openssl-us...@openssl.org [mailto:owner-openssl-us...@openssl.org] On Behalf Of Dave Thompson Sent: 19 December 2013 08:36 To: openssl-users@openssl.org Subject: RE: HTTPS TLSv1.2 Client-Auth negotiation Yes, that Wireshark decode of (encrypted) renegotiation is clearly wrong. Sending two ClientKX would be wrong, sending (Client)Cert and ClientKX is right, and the first size would match Cert and not ClientKX. You might try is s_client -connect 23.66.176.239 -msg -debug with redirect from a file that contains a suitable GET request (caveat: if on Windows native such as ShiningLight, to redirect from a file or pipe you must also type something on the keyboard, even though that keyboard input isn't read or used.) That will show both the protocol messages sent and received, before encryption and after decryption, as well as the wire data. Actually checking whether OpenSSL is 'encrypting' wrong or the server is 'decrypting' wrong is quite a bit of work. In your place I would first see if changing some parameters avoids or affects the problem: - you say the server 'requires' 1.2 - how forcefully? Won't even negotiate lower? Disconnects afterward? Treats as unauthorized? I - different cipher: maybe AES128 better 3DES or even RC4 if allowed* - if you were using SHA2 HMAC I would definitely recommend trying SHA1. You might try MD5 if allowed*. These might at least narrow down where to look for the problem. Note that 'decrypt error' may actually mean 'HMAC error'. Originally these were separate alert codes but they were combined (officially in TLSv1.1 IIRC, but often by implementations even earlier). (* USgov non-national-security systems are supposed to obey NIST rules which prohibit SSLv3, RC4, and MD5. Nat-sec systems are subject to NSA rules which I as a civilian don't know, but I would be astonished and disappointed if they are weaker than NIST.) If you are using an openssl build with asm, you might try one without. The asm code is platform dependent, and depending on your platform might not get as thoroughly exercised as the generic C. Also if you are using a FIPS build (that is enabled) try non-FIPS. The FIPS module code is necessarily a good bit older than the non-FIPS. (Even if you can't use non-FIPS for production, just knowing if it affects the problem would help.) Good luck. From: owner-openssl-us...@openssl.org<mailto:owner-openssl-us...@openssl.org> [mailto:owner-openssl-us...@openssl.org] On Behalf Of Ben Arnold Sent: Tuesday, December 17, 2013 06:05 To: openssl-users@openssl.org<mailto:openssl-users@openssl.org> Subject: *** Spam *** HTTPS TLSv1.2 Client-Auth negotiation Hi, I am using libcurl and OpenSSL to communicate with various webservers, most of which require client authentication. I am having trouble connecting to one server that requires TLSv1.2. After the server has sent a Certificate Request, OpenSSL sends up the client cert (I think) and the server replies with a Decrypt Error alert. The messages that are sent to the server are (as decoded by Wireshark): Client Key Exchange Client Key Exchange Certificate Verify Change Cipher Spec Encrypted Handshake Message The first CKE is 1456 bytes long, which I suspect means it includes the certificate as the PreMaster is only 258 bytes. I am wondering if this is something to do with TLSv1.2, all of the other servers I connect to are happy with TLSv1. If I use the cURL command line tool then it works: ... * SSLv3, TLS handshake, Request CERT (13): * SSLv3, TLS handshake, Server finished (14): * SSLv3, TLS handshake, CERT (11): * SSLv3, TLS handshake, Client key exchange (16): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): I have attached ClientCertFail.pcapng which shows the trace of a failure, along with ClientCertFail.keys which contains the keys for that session. (btw, are the strange CKE messages from client -> server simply an artefact of Wireshark's decoding, or do they point to the problem? They don't seem to match cURL's diagnostic output, but I can't see the network capture from cURL as it won't output the session keys) Many thanks, Ben
