Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
I'm afraid to ask where Boost is storing them. But I do have a morbid curiosity: would you happen to know? Is this a little sense of irony ? :) From what I know Boost does not store the keys... Boost has an implementation of SSL based on OpenSSL in boost::asio::ssl. This mimics the functionality and capabilities of OpenSSL and only accepts specifying certificates and private keys stored in files (in PEM format). For security, the private keys in a PEM file can be encrypted with a password. If OpenSSL, and thus Boost, keeps the private key in some protected way (like using DPAPI on windows) after reading it from a PEM file, I really don't know and I would be curious to know too. My question was about using the OpenSSL CAPI engine because I tried to mix the security offered by windows certificate store (and read the certificate and PKEY from there with CAPI engine) with using Boost::asio::ssl and OpenSSL (which I'm forced to do because of the framework I use). Unfortunately OpenSSL CAPI engine does not offer all the functionality I need even in the latest 1.0.1c version - like for example loading a server certificate. It can only load a private key, but that is questionable too, because I tried to save the private key to a PEM file and read it back and it didn't work. Saving/reading to/from the PEM file was done with the API offered by OpenSSL in pem.h so I doubt that the API was not capable to save it correctly (even if instead of the prime numbers, the EVP_PKEY structure returned from ENGINE_load_privatekey contained some indexes, as someone mentioned in a post to this thread). Using CryptoAPI directly to export an exportable private key from windows certificate store and to encode into PEM format, worked. I even fed the obtained PEM file into OpenSSL with success. I have exported the key as a PRIVATEKEYBLOB whereas the OpenSSL CAPI engine exports it as a PUBLICKEYBLOB and then converts it to the OpenSSL EVP_PKEY structure (but only the public exponent and modulus are put in the EVP_PKEY). From: Jeffrey Walton noloa...@gmail.com To: openssl-users@openssl.org Sent: Friday, December 7, 2012 4:52 PM Subject: Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine On Fri, Dec 7, 2012 at 5:05 AM, LN lnicu...@yahoo.com wrote: ... MS CAPI has an option to mark a private key as exportable when you create or install it, which means that the private key can then be read anyway, but I don't know if that feature is used by the OpenSSL CAPI Engine. It is almost always a good idea NOT to mark private keys as exportable. Note that whatever is decided when the private key is first stored by CAPI will be permanent (There is a 3 step workaround for making an exportable key non-exportable, but any ability to go the other way would compromise security just by being possible). Indeed, private keys are not exportable as long as they are not marked as such when the certificate is imported in the windows store. Unfortunately, I am forced to use boost::asio::ssl which (AFAIK) does not integrate with CAPI engine so I cannot ask it to sign or decrypt communication. Anyway, seems more secure, then, to have the private key in a file encrypted with a password, then keeping it in the windows store, if I want to pass it to OpenSSL (through boost::asio::ssl) :) Keys should be stored in DPAPI. See Howard and LeBlanc's Writing Secure Code, Chapter 9 (http://www.amazon.com/Writing-Secure-Second-Michael-Howard/dp/0735617228). I'm afraid to ask where Boost is storing them. But I do have a morbid curiosity: would you happen to know? Jeff __ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
On Wednesday, December 5, 2012, Ashok C ash@gmail.com wrote: Hi, Our current SSL server loads plain-text private keys using the SSL_CTX_use_PrivateKey_file() method. We are moving from this strategy to use custom encrypted private keys using the TPM concept. For this, we have an engine implemented. Now the question is this. Does my SSL server need to change from loading private keys using SSL_CTX_use_PrivateKey_file() method to use this method -- ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data) Or can this be achieved by using the SSL_CTX_use_PrivateKey_file() method itself and the openssl takes care of loading my encrypted private key with the help of my implemented engine. Basically wanted to understand if the SSL_CTX method for loading private keys has the support for loading engine specific keys. -- Ashok
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
Thanks for reply, Jakob. Your are probably right. Microsoft CAPI essentially treats all its key storages like physical smart cards, which means that by default, you cannot extract the private key using any documented method (if at all), ... It's confusing... OpenSSL provides an API that it is called ENGINE_load_private_key but which returns the PUBLIC key ?! Passing the EVP_PKEY structure returned from ENGINE_load_private_key to OpenSSL using SSL_CTX_use_PrivateKey seems to work... The server-client communication works as expected though, I guess, OpenSSL expects a private key in SSL_CTX_use_PrivateKey. Is this possible because I use a RSA key ? ...but you (and thus the OpenSSL CAPI Engine) can ask CAPI to use the key to decrypt or sign something. MS CAPI has an option to mark a private key as exportable when you create or install it, which means that the private key can then be read anyway, but I don't know if that feature is used by the OpenSSL CAPI Engine. It is almost always a good idea NOT to mark private keys as exportable. Note that whatever is decided when the private key is first stored by CAPI will be permanent (There is a 3 step workaround for making an exportable key non-exportable, but any ability to go the other way would compromise security just by being possible). Indeed, private keys are not exportable as long as they are not marked as such when the certificate is imported in the windows store. Unfortunately, I am forced to use boost::asio::ssl which (AFAIK) does not integrate with CAPI engine so I cannot ask it to sign or decrypt communication. Anyway, seems more secure, then, to have the private key in a file encrypted with a password, then keeping it in the windows store, if I want to pass it to OpenSSL (through boost::asio::ssl) :) From: Jakob Bohm jb-open...@wisemo.com To: openssl-users@openssl.org Sent: Thursday, December 6, 2012 12:23 PM Subject: Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine On 12/6/2012 9:13 AM, LN wrote: Hi, Somehow related to private keys but about loading them with CAPI engine... Does someone know if the ENGINE_load_private_key() for CAPI engine returns the PUBLIC KEY ? I have a feeling it does so because I tried to save that returned EVP_PKEY to a PEM file with PEM_write_bio_PrivateKey and then to load it back from the same file with PEM_read_bio_PrivateKey. Saving worked, but loading failed (with some decoding error inside openss). The length of the base64 string in the PEM file for the key was very small in comparision with what is usually the length for private keys. PEM_write_bio_PUBKEY and PEM_read_bio_PUBKEY worked with the returned EVP_PKEY. Your are probably right. Microsoft CAPI essentially treats all its key storages like physical smart cards, which means that by default, you cannot extract the private key using any documented method (if at all), but you (and thus the OpenSSL CAPI Engine) can ask CAPI to use the key to decrypt or sign something. MS CAPI has an option to mark a private key as exportable when you create or install it, which means that the private key can then be read anyway, but I don't know if that feature is used by the OpenSSL CAPI Engine. It is almost always a good idea NOT to mark private keys as exportable. Note that whatever is decided when the private key is first stored by CAPI will be permanent (There is a 3 step workaround for making an exportable key non-exportable, but any ability to go the other way would compromise security just by being possible). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded __ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
On 12/07/2012 11:05 AM, LN wrote: I have a feeling it does so because I tried to save that returned EVP_PKEY to a PEM file with PEM_write_bio_PrivateKey and then to load it back from the same file with PEM_read_bio_PrivateKey. Saving worked, but loading failed (with some decoding error inside openss). The length of the base64 string in the PEM file for the key was very small in comparision with what is usually the length for private keys. Often, at least for other HSM, a private key returned does not contain the real p and q values but just some index to enumerate a key in the hsm. These are normally some small numbers and not big primes. you might see this if you do openssl rsa -text yourfile
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
On Fri, Dec 7, 2012 at 5:05 AM, LN lnicu...@yahoo.com wrote: ... MS CAPI has an option to mark a private key as exportable when you create or install it, which means that the private key can then be read anyway, but I don't know if that feature is used by the OpenSSL CAPI Engine. It is almost always a good idea NOT to mark private keys as exportable. Note that whatever is decided when the private key is first stored by CAPI will be permanent (There is a 3 step workaround for making an exportable key non-exportable, but any ability to go the other way would compromise security just by being possible). Indeed, private keys are not exportable as long as they are not marked as such when the certificate is imported in the windows store. Unfortunately, I am forced to use boost::asio::ssl which (AFAIK) does not integrate with CAPI engine so I cannot ask it to sign or decrypt communication. Anyway, seems more secure, then, to have the private key in a file encrypted with a password, then keeping it in the windows store, if I want to pass it to OpenSSL (through boost::asio::ssl) :) Keys should be stored in DPAPI. See Howard and LeBlanc's Writing Secure Code, Chapter 9 (http://www.amazon.com/Writing-Secure-Second-Michael-Howard/dp/0735617228). I'm afraid to ask where Boost is storing them. But I do have a morbid curiosity: would you happen to know? Jeff __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
Hi, Somehow related to private keys but about loading them with CAPI engine... Does someone know if the ENGINE_load_private_key() for CAPI engine returns the PUBLIC KEY ? I have a feeling it does so because I tried to save that returned EVP_PKEY to a PEM file with PEM_write_bio_PrivateKey and then to load it back from the same file with PEM_read_bio_PrivateKey. Saving worked, but loading failed (with some decoding error inside openss). The length of the base64 string in the PEM file for the key was very small in comparision with what is usually the length for private keys. PEM_write_bio_PUBKEY and PEM_read_bio_PUBKEY worked with the returned EVP_PKEY. Thanks! From: Ashok C ash@gmail.com To: openssl-users@openssl.org Sent: Thursday, December 6, 2012 7:03 AM Subject: Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine Thanks Steve and Kent for the pointers. Makes things clear for now. On Thu, Dec 6, 2012 at 4:22 AM, Dr. Stephen Henson st...@openssl.org wrote: On Wed, Dec 05, 2012, Ashok C wrote: Hi, Our current SSL server loads plain-text private keys using the SSL_CTX_use_PrivateKey_file() method. We are moving from this strategy to use custom encrypted private keys using the TPM concept. For this, we have an engine implemented. Now the question is this. Does my SSL server need to change from loading private keys using SSL_CTX_use_PrivateKey_file() method to use this method -- ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data) You can't use SSL_CTX_use_PrivateKey_file to load an ENGINE specific key. Instead you use ENGINE_load_private_key() to load the key into an EVP_PKEY structure and then pass that to SSL_CTX_use_PrivateKey(). Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
On 12/6/2012 9:13 AM, LN wrote: Hi, Somehow related to private keys but about loading them with CAPI engine... Does someone know if the ENGINE_load_private_key() for CAPI engine returns the PUBLIC KEY ? I have a feeling it does so because I tried to save that returned EVP_PKEY to a PEM file with PEM_write_bio_PrivateKey and then to load it back from the same file with PEM_read_bio_PrivateKey. Saving worked, but loading failed (with some decoding error inside openss). The length of the base64 string in the PEM file for the key was very small in comparision with what is usually the length for private keys. PEM_write_bio_PUBKEY and PEM_read_bio_PUBKEY worked with the returned EVP_PKEY. Your are probably right. Microsoft CAPI essentially treats all its key storages like physical smart cards, which means that by default, you cannot extract the private key using any documented method (if at all), but you (and thus the OpenSSL CAPI Engine) can ask CAPI to use the key to decrypt or sign something. MS CAPI has an option to mark a private key as exportable when you create or install it, which means that the private key can then be read anyway, but I don't know if that feature is used by the OpenSSL CAPI Engine. It is almost always a good idea NOT to mark private keys as exportable. Note that whatever is decided when the private key is first stored by CAPI will be permanent (There is a 3 step workaround for making an exportable key non-exportable, but any ability to go the other way would compromise security just by being possible). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
Hi Ashok, On Wed, Dec 5, 2012 at 12:29 AM, Ashok C ash@gmail.com wrote: Hi, Our current SSL server loads plain-text private keys using the SSL_CTX_use_PrivateKey_file() method. We are moving from this strategy to use custom encrypted private keys using the TPM concept. For this, we have an engine implemented. Now the question is this. Does my SSL server need to change from loading private keys using SSL_CTX_use_PrivateKey_file() method to use this method -- ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data) Or can this be achieved by using the SSL_CTX_use_PrivateKey_file() method itself and the openssl takes care of loading my encrypted private key with the help of my implemented engine. Basically wanted to understand if the SSL_CTX method for loading private keys has the support for loading engine specific keys. Last I knew there were two ways to get your TPM key loaded through an engine. Directly in your app, using code like: ENGINE_by_id(); ENGINE_init(); ENGINE_set_default_RSA(); ENGINE_load_private_key(); ... (use key ... and indirectly by using an openssl-config enabled app, and the RSA_* APIs to load the key. Basically in this case you have a system openssl config that you've added something like: --[cut]-- openssl_conf = openssl_def [openssl_def] engines = engine_section [engine_section] foo = tpm_section [tpm_section] dynamic_path = /usr/local/ssl/lib/engines/libtpm.so engine_id = tpm default_algorithms = RAND,RSA init = 1 --[cut]-- then your app needs to be compiled with -DOPENSSL_LOAD_CONF=1, which triggers hooks in RSA_* apis which will read the config file and redirect key loading to your engine. Kent -- Ashok __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
On Wed, Dec 05, 2012, Ashok C wrote: Hi, Our current SSL server loads plain-text private keys using the SSL_CTX_use_PrivateKey_file() method. We are moving from this strategy to use custom encrypted private keys using the TPM concept. For this, we have an engine implemented. Now the question is this. Does my SSL server need to change from loading private keys using SSL_CTX_use_PrivateKey_file() method to use this method -- ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data) You can't use SSL_CTX_use_PrivateKey_file to load an ENGINE specific key. Instead you use ENGINE_load_private_key() to load the key into an EVP_PKEY structure and then pass that to SSL_CTX_use_PrivateKey(). Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: How to over-ride SSL_CTX_use_PrivateKey_file() behavior with custom engine
Thanks Steve and Kent for the pointers. Makes things clear for now. On Thu, Dec 6, 2012 at 4:22 AM, Dr. Stephen Henson st...@openssl.orgwrote: On Wed, Dec 05, 2012, Ashok C wrote: Hi, Our current SSL server loads plain-text private keys using the SSL_CTX_use_PrivateKey_file() method. We are moving from this strategy to use custom encrypted private keys using the TPM concept. For this, we have an engine implemented. Now the question is this. Does my SSL server need to change from loading private keys using SSL_CTX_use_PrivateKey_file() method to use this method -- ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data) You can't use SSL_CTX_use_PrivateKey_file to load an ENGINE specific key. Instead you use ENGINE_load_private_key() to load the key into an EVP_PKEY structure and then pass that to SSL_CTX_use_PrivateKey(). Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
