Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
In message <584d7f4e.8090...@roumenpetrov.info> on Sun, 11 Dec 2016 18:31:10 +0200, Roumen Petrov said: openssl> One remark for store load function api - in most cases (load from openssl> file) it is password callback but is other cases it could be PIN or openssl> something different. openssl> Please use more generic description. openssl> For instance engine callback is defined in generic way - ui_method and openssl> its callback_data. I see what you mean. Just did the improvement. -- Richard Levitte levi...@openssl.org OpenSSL Project http://www.openssl.org/~levitte/ -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
Roumen Petrov skrev: (11 december 2016 17:31:10 CET) >Hi Richard, > >Richard Levitte wrote: >> In message<20161206.223057.237264374331072901.levi...@openssl.org> >on Tue, 06 Dec 2016 22:30:57 +0100 (CET), Richard >Levitte said: >> >> levitte> [SNIP] >> >> The easiest was actually to rewrite PEM_read_bio_PrivateKey() >> entirely, so it solely uses the internal store_file functions I've >> provided. >> I wonder what kind of impact this would have on the community at >> large. > >PEM_read_bio_PrivateKey use custom password callback . You propose >"Store-API" with UI_METHOD as password callback. >Rewrite of pem_read... method obsoletes pem_password_cb. > >What about to ensure a transition period? >For instance in openssl 1.1 to provide new functions based on UI_METHOD >and to mark existing as deprecated. > > >One remark for store load function api - in most cases (load from file) >it is password callback but is other cases it could be PIN or something >different. >Please use more generic description. >For instance engine callback is defined in generic way - ui_method and >its callback_data. Earlier, I mentioned an experimental branch, https://github.com/levitte/openssl/tree/tpm_engine-support?files=1 If you have a look, you'll find an added UI utility function to wrap a pem password callback in a UI_METHOD. > > > >> Cheers, >> Richard > >Regards, >Roumen -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
Hi Richard, Richard Levitte wrote: In message<20161206.223057.237264374331072901.levi...@openssl.org> on Tue, 06 Dec 2016 22:30:57 +0100 (CET), Richard Levitte said: levitte> [SNIP] The easiest was actually to rewrite PEM_read_bio_PrivateKey() entirely, so it solely uses the internal store_file functions I've provided. I wonder what kind of impact this would have on the community at large. PEM_read_bio_PrivateKey use custom password callback . You propose "Store-API" with UI_METHOD as password callback. Rewrite of pem_read... method obsoletes pem_password_cb. What about to ensure a transition period? For instance in openssl 1.1 to provide new functions based on UI_METHOD and to mark existing as deprecated. One remark for store load function api - in most cases (load from file) it is password callback but is other cases it could be PIN or something different. Please use more generic description. For instance engine callback is defined in generic way - ui_method and its callback_data. Cheers, Richard Regards, Roumen -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
HI Richard, Richard Levitte wrote: In message<58472e4f.3010...@roumenpetrov.info> on Tue, 06 Dec 2016 23:31:59 +0200, Roumen Petrov said: openssl> Hi Richard, openssl> [SNIP] openssl> > Check. My STORE branch is made to support that. openssl> One URI could represent more then one item. openssl> STORE_INFO_types is enumerate but URI could be associated to custom openssl> data (handle) and this data could be used to get other data(handles). openssl> openssl> See capi engine CAPI_KEY *capi_find_key(CAPI_CTX * ctx, const char openssl> *id) [SNIP] openssl> Is above case PKEY is loaded only if CERT is located(found). I'm trying to understand but am failing. Looking at your example, it's quite clear that what you want to retrieve is a key, even though you have to go through the corresponding certificate to get to it. After first review of API delared in openssl/store.h I misunderstand goal of load method. I think that code of capi engine could be considered as sample what is need for an loadable module (engine) to use "OpenSSL Store API". I post above code just to get idea where currently is used an "external store api". Just imagine how existing capi code could be changed to use store-API and to implement loader(scheme?). I'm asking as currently there is no interface (API) that could associate key (private) and X.509 certificate. Currently engines implement custom command as work-around. For instance LOAD_CERT_CTRL (pkcs11 and e_nss) and LOAD_CERT_EVP(e_nss). This one of areas where applications could benefit from "Store API". I post a sniped from CAPI code because it is part of OpenSSL, but king of "external store api" is used by other engines. However,*nothing* stops anyone from making a loader for the "capi" scheme (if there is such a thing) that has a load method that will return the certificate (STORE_INFO_CERT) on the first call and the associated key (STORE_INFO_PKEY) on the second for the same URI. It's all about caching information, and there is a context variable (type STORE_LOADER_CTX, which is just a template type for loader defined 'struct store_loader_ctx_st') to be used exactly for that kind of purpose. I guess that "load" method is supposed to return all data at once. Actually it is an iterator! Please update comments before method and if possible to change name of method. [SNIP] In your example above, I fail to see where the custom data would be needed... And frankly, STORE is first of all meant to handle types that can be used with the rest of OpenSSL. That being said, adding a "whatever" STORE_INFO type isn't very hard either. I'm just not terribly convinced yet, but let's keep talking, I'll probably understand sooner or later what you're actually after. I also fail to see why a store scheme has to return "custom data". Note that thread start from request for load TPM keys and some one mention that TMP key has custom data. In addition to load of key from file there is one another interfaces where could be used store api - see RT4681. The goal is expired from fact that X.509 load method is hidden in OpenSSL 1.1. I guess that "by dir" and "by file" could be updated to use store api. Also applications has to able to register that a "store scheme" could by used by X.509 lookups. Cheers, Richard ( oh, and if example code is needed, I can provide ) +4 for OpenSSL store api ;) Regards, Roumen -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Fri, 2016-12-09 at 00:25 +, David Woodhouse wrote: > On Thu, 2016-12-08 at 16:22 -0800, James Bottomley wrote: > > > > I'm guessing you mean this: > > > > https://www.trustedcomputinggroup.org/wp-content/uploads/TSS_Versio > > n_1.2_Level_1_FINAL.pdf > > > > ? It still doesn't tell you who the expected parent of the key > > would be, which is the problem I'm currently trying to solve. > > Right, in S3.23 ("Portable Data") or thereabouts. And yes, it doesn't > indicate the parent key. As discussed, the TssBlob structure there is > fairly pointless which is presumably why the TPM ENGINE didn't use > it, and just dumped the actual blob (which I later put into an > OCTET_STRING in the PEM files, IIRC). > > I was hoping that the TPM 2.0 spec would fix that, maybe? Not far enough through it yet to be definitive, but I don't think so ... there's even more parent like seeds in it. James smime.p7s Description: S/MIME cryptographic signature -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Thu, 2016-12-08 at 16:22 -0800, James Bottomley wrote: > > I'm guessing you mean this: > > https://www.trustedcomputinggroup.org/wp-content/uploads/TSS_Version_1.2_Level_1_FINAL.pdf > > ? It still doesn't tell you who the expected parent of the key would > be, which is the problem I'm currently trying to solve. Right, in S3.23 ("Portable Data") or thereabouts. And yes, it doesn't indicate the parent key. As discussed, the TssBlob structure there is fairly pointless which is presumably why the TPM ENGINE didn't use it, and just dumped the actual blob (which I later put into an OCTET_STRING in the PEM files, IIRC). I was hoping that the TPM 2.0 spec would fix that, maybe? -- dwmw2 smime.p7s Description: S/MIME cryptographic signature -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Thu, 2016-12-08 at 15:56 -0800, James Bottomley wrote: > On Thu, 2016-12-08 at 23:44 +, David Woodhouse wrote: > > On Tue, 2016-12-06 at 22:30 +0100, Richard Levitte wrote: > > > Oh > > > > > > I think I aired some thoughts on using PEM headers a very long > > > while > > > ago, but that never came into fruition, among others because I > > > ended > > > up doubting that it would be the best way in the long run. > > > > > > These days, the use of PEM headers is considered old and kinda > > > sorta > > > deprecated, even though OpenSSL still produces encrypted private > > > key > > > PEM files that uses headers for the encryption metadata. It > > > seems > > > that PKCS#8 is prefered "out there". > > > > > > So I have to wonder, is PEM really the right way to go for this? > > > Would it be just as possible to wrap a TSS key with a PKCS#8 > > > container, and use the associated attributes for the external > > > data? > > > Just a thought, though... I can't do more than throw around > > > ideas, > > > considering how little I know about TPM. > > > > I would definitely suggest that we *don't* want to do it with PEM > > headers. Just put the additional information into the binary ASN.1 > > structure. > > Which evil is lesser? If we put it in ASN.1 we'll be defining our > own > instead of using the TSS defined one. If we use headers, we can put > the extra data in them and use the TSS defined ASN.1 for the key > blob. > > > The 2.0 version of the TssBlob (from §3.23 of the 1.2 spec) should > > hopefully contain all the auxiliary information we need, without > > having to stick it in PEM headers. > > Which of the many specs is this? I'm guessing you mean this: https://www.trustedcomputinggroup.org/wp-content/uploads/TSS_Version_1.2_Level_1_FINAL.pdf ? It still doesn't tell you who the expected parent of the key would be, which is the problem I'm currently trying to solve. James smime.p7s Description: S/MIME cryptographic signature -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Thu, 2016-12-08 at 23:44 +, David Woodhouse wrote: > On Tue, 2016-12-06 at 22:30 +0100, Richard Levitte wrote: > > Oh > > > > I think I aired some thoughts on using PEM headers a very long > > while > > ago, but that never came into fruition, among others because I > > ended > > up doubting that it would be the best way in the long run. > > > > These days, the use of PEM headers is considered old and kinda > > sorta > > deprecated, even though OpenSSL still produces encrypted private > > key > > PEM files that uses headers for the encryption metadata. It seems > > that PKCS#8 is prefered "out there". > > > > So I have to wonder, is PEM really the right way to go for this? > > Would it be just as possible to wrap a TSS key with a PKCS#8 > > container, and use the associated attributes for the external data? > > Just a thought, though... I can't do more than throw around ideas, > > considering how little I know about TPM. > > I would definitely suggest that we *don't* want to do it with PEM > headers. Just put the additional information into the binary ASN.1 > structure. Which evil is lesser? If we put it in ASN.1 we'll be defining our own instead of using the TSS defined one. If we use headers, we can put the extra data in them and use the TSS defined ASN.1 for the key blob. > The 2.0 version of the TssBlob (from §3.23 of the 1.2 spec) should > hopefully contain all the auxiliary information we need, without > having to stick it in PEM headers. Which of the many specs is this? James smime.p7s Description: S/MIME cryptographic signature -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Tue, 2016-12-06 at 22:30 +0100, Richard Levitte wrote: > Oh > > I think I aired some thoughts on using PEM headers a very long while > ago, but that never came into fruition, among others because I ended > up doubting that it would be the best way in the long run. > > These days, the use of PEM headers is considered old and kinda sorta > deprecated, even though OpenSSL still produces encrypted private key > PEM files that uses headers for the encryption metadata. It seems > that PKCS#8 is prefered "out there". > > So I have to wonder, is PEM really the right way to go for this? > Would it be just as possible to wrap a TSS key with a PKCS#8 > container, and use the associated attributes for the external data? > Just a thought, though... I can't do more than throw around ideas, > considering how little I know about TPM. I would definitely suggest that we *don't* want to do it with PEM headers. Just put the additional information into the binary ASN.1 structure. The 2.0 version of the TssBlob (from §3.23 of the 1.2 spec) should hopefully contain all the auxiliary information we need, without having to stick it in PEM headers. -- dwmw2 smime.p7s Description: S/MIME cryptographic signature -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On 12/06/2016 10:42 PM, Richard Levitte wrote: > The easiest was actually to rewrite PEM_read_bio_PrivateKey() > entirely, so it solely uses the internal store_file functions I've > provided. > I wonder what kind of impact this would have on the community at > large. > If you do that, please ensure that secure memory is used for the temporary buffers involved in reading private key material (assuming that some form of https://github.com/openssl/openssl/pull/1700 gets merged). -Ben -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
In message <20161206.223057.237264374331072901.levi...@openssl.org> on Tue, 06 Dec 2016 22:30:57 +0100 (CET), Richard Levitte said: levitte> That being said, it should certainly be easy enough to change the levitte> appropriate places to make sure headers are available as well, and I levitte> have zero issues adding a header parameter to the try_decode levitte> prototype and associated functions. Done. levitte> One thing I didn't think of earlier is that PEM_bytes_read_bio() levitte> checks the pem name against a known set, *or* in the private key case, levitte> that the pem name ends with " PRIVATE KEY" (which "TSS KEY BLOB" does levitte> not), so some kind of refactoring is needed to accomodate the levitte> store_file_load() call either way. levitte> (quite frankly, I'm slowly realising that the STORE_FILE_HANDLER code levitte> can replace quite a lot of the discovery code in the PEM module, so levitte> refactoring could be in order either way) The easiest was actually to rewrite PEM_read_bio_PrivateKey() entirely, so it solely uses the internal store_file functions I've provided. I wonder what kind of impact this would have on the community at large. Cheers, Richard -- Richard Levitte levi...@openssl.org OpenSSL Project http://www.openssl.org/~levitte/ -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
In message <58472e4f.3010...@roumenpetrov.info> on Tue, 06 Dec 2016 23:31:59 +0200, Roumen Petrov said: openssl> Hi Richard, openssl> openssl> Richard Levitte wrote: openssl> > [SNIP] openssl> > James.Bottomley> 1. We agreed that usability is greatly enhanced if openssl> > openssl simply loads openssl> > James.Bottomley> a key when presented with the file/uri etc. without openssl> > the user having openssl> > James.Bottomley> to specify what the format of a key is openssl> > openssl> > Check. My STORE branch is made to support that. openssl> One URI could represent more then one item. openssl> STORE_INFO_types is enumerate but URI could be associated to custom openssl> data (handle) and this data could be used to get other data(handles). openssl> openssl> See capi engine CAPI_KEY *capi_find_key(CAPI_CTX * ctx, const char openssl> *id) openssl> .. openssl> hstore = capi_open_store(ctx, NULL); openssl> if (!hstore) openssl> return NULL; openssl> cert = capi_find_cert(ctx, id, hstore); openssl> if (cert) { openssl> key = capi_get_cert_key(ctx, cert); openssl> CertFreeCertificateContext(cert); openssl> } openssl> CertCloseStore(hstore, 0); openssl> .. openssl> Is above case PKEY is loaded only if CERT is located(found). I'm trying to understand but am failing. Looking at your example, it's quite clear that what you want to retrieve is a key, even though you have to go through the corresponding certificate to get to it. However, *nothing* stops anyone from making a loader for the "capi" scheme (if there is such a thing) that has a load method that will return the certificate (STORE_INFO_CERT) on the first call and the associated key (STORE_INFO_PKEY) on the second for the same URI. It's all about caching information, and there is a context variable (type STORE_LOADER_CTX, which is just a template type for loader defined 'struct store_loader_ctx_st') to be used exactly for that kind of purpose. In your example above, I fail to see where the custom data would be needed... And frankly, STORE is first of all meant to handle types that can be used with the rest of OpenSSL. That being said, adding a "whatever" STORE_INFO type isn't very hard either. I'm just not terribly convinced yet, but let's keep talking, I'll probably understand sooner or later what you're actually after. Cheers, Richard ( oh, and if example code is needed, I can provide ) -- Richard Levitte levi...@openssl.org OpenSSL Project http://www.openssl.org/~levitte/ -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
Hi Richard, Richard Levitte wrote: [SNIP] James.Bottomley>1. We agreed that usability is greatly enhanced if openssl simply loads James.Bottomley> a key when presented with the file/uri etc. without the user having James.Bottomley> to specify what the format of a key is Check. My STORE branch is made to support that. One URI could represent more then one item. STORE_INFO_types is enumerate but URI could be associated to custom data (handle) and this data could be used to get other data(handles). See capi engine CAPI_KEY *capi_find_key(CAPI_CTX * ctx, const char *id) .. hstore = capi_open_store(ctx, NULL); if (!hstore) return NULL; cert = capi_find_cert(ctx, id, hstore); if (cert) { key = capi_get_cert_key(ctx, cert); CertFreeCertificateContext(cert); } CertCloseStore(hstore, 0); .. Is above case PKEY is loaded only if CERT is located(found). I guess that STORE_INFO_types could be bit-mask and to support application specific data in addition. [SNIP] Roumen -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
In message <1481043672.4406.22.ca...@hansenpartnership.com> on Tue, 06 Dec 2016 09:01:12 -0800, James Bottomley said: James.Bottomley> On Tue, 2016-12-06 at 17:47 +0100, Richard Levitte wrote: James.Bottomley> > In message <1481042048.4406.14.ca...@hansenpartnership.com> on Tue, James.Bottomley> > 06 Dec 2016 08:34:08 -0800, James Bottomley < James.Bottomley> > james.bottom...@hansenpartnership.com> said: James.Bottomley> > James.Bottomley> > James.Bottomley> On Tue, 2016-12-06 at 15:12 +0100, Richard Levitte James.Bottomley> > wrote: James.Bottomley> > James.Bottomley> > In message < James.Bottomley> > 1480697558.2410.33.ca...@hansenpartnership.com> on Fri, James.Bottomley> > James.Bottomley> > 02 Dec 2016 08:52:38 -0800, James Bottomley < James.Bottomley> > James.Bottomley> > james.bottom...@hansenpartnership.com> said: James.Bottomley> > James.Bottomley> > James.Bottomley> > James.Bottomley> > When I made that argument, I hadn't thought and James.Bottomley> > felt things through James.Bottomley> > James.Bottomley> > entirely. Truth be told, I'm feeling very uneasy James.Bottomley> > handing over the James.Bottomley> > James.Bottomley> > reading and parsing of the PEM file to an engine. James.Bottomley> > However, handing James.Bottomley> > James.Bottomley> > over the decoded data and leaving it to the engine James.Bottomley> > to do something James.Bottomley> > James.Bottomley> > sensible with it, no issues at all. James.Bottomley> > James.Bottomley> James.Bottomley> > James.Bottomley> OK, can I pick on this a bit (I'll look at the store James.Bottomley> > stuff later and James.Bottomley> > James.Bottomley> reply after I've played with it)? What is it that James.Bottomley> > you imagine handing James.Bottomley> > James.Bottomley> to the engine? Some type of buffer that contains James.Bottomley> > the full PEM James.Bottomley> > James.Bottomley> representation? James.Bottomley> > James.Bottomley> > In my STORE branch, you will find this in James.Bottomley> > include/openssl/store_file.h: James.Bottomley> > James.Bottomley> > /* James.Bottomley> > * The try_decode function is called to check if the blob of data James.Bottomley> > can James.Bottomley> > * be used by this handler, and if it can, decodes it into a James.Bottomley> > supported James.Bottomley> > * OpenSSL and returns a STORE_INFO with the recorded data. James.Bottomley> > * Input: James.Bottomley> > *pem_name: If this blob comes from a PEM file, this James.Bottomley> > holds James.Bottomley> > * the PEM name. If it comes from another type James.Bottomley> > of James.Bottomley> > * file, this is NULL. James.Bottomley> > *blob: The blob of data to match with what this James.Bottomley> > handler James.Bottomley> > * can use. James.Bottomley> > *len: The length of the blob. James.Bottomley> > *handler_ctx: For a handler marked repeatable, this pointer James.Bottomley> > can James.Bottomley> > * be used to create a context for the handler. James.Bottomley> > IT IS James.Bottomley> > * THE HANDLER'S RESPONSIBILITY TO CREATE AND James.Bottomley> > DESTROY James.Bottomley> > * THIS CONTEXT APPROPRIATELY, i.e. create on James.Bottomley> > first call James.Bottomley> > * and destroy when about to return NULL. James.Bottomley> > *password_ui: Application UI method for getting a password. James.Bottomley> > *password_ui_data: James.Bottomley> > * Application data to be passed to password_ui James.Bottomley> > when James.Bottomley> > * it's called. James.Bottomley> > * Output: James.Bottomley> > *a STORE_INFO James.Bottomley> > */ James.Bottomley> > typedef STORE_INFO *(*STORE_FILE_try_decode_fn)(const char James.Bottomley> > *pem_name, James.Bottomley> > const unsigned James.Bottomley> > char *blob, James.Bottomley> > size_t len, James.Bottomley> > void James.Bottomley> > **handler_ctx, James.Bottomley> > const UI_METHOD James.Bottomley> > *password_ui, James.Bottomley> > void James.Bottomley> > *password_ui_data); James.Bottomley> > James.Bottomley> > This is the central function that tries to see if it can decode James.Bottomley> > whatever's thrown at it. As you can see, it gets handed the PEM name James.Bottomley> > if it's called as part of a PEM read. The |blob| is the decoded *and James.Bottomley> > decrypted* data. James.Bottomley> > James.Bottomley> > James.Bottomley> The reason I chose a BIO is that it's the basic James.Bott
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On 12/06/2016 11:01 AM, James Bottomley wrote: > The next problem is that this is slightly harder simply to insert into > the PEM code. The BIO parsing is done in PEM_bytes_read_bio() not > PEM_read_bio_PrivateKey(). The easy way to cope with this would be to > move PEM parsing into the ENGINE_find_engine_load_key() function and > then hand the name, header, blob to the engine. The disadvantage of > this is that we'll end up pulling the PEM apart twice: once in > ENGINE_find_engine_load_key() and then again in PEM_bytes_read_bio(). > Is this OK? If it is I can code up a v3. If not, I can think about > how we might integrate this into PEM_bytes_read_bio(). FWIW, you may find the refactored version at https://github.com/openssl/openssl/pull/1700 easier to reason about. I expect to have some time to update the pull request this week. -Ben -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Tue, 2016-12-06 at 17:47 +0100, Richard Levitte wrote: > In message <1481042048.4406.14.ca...@hansenpartnership.com> on Tue, > 06 Dec 2016 08:34:08 -0800, James Bottomley < > james.bottom...@hansenpartnership.com> said: > > James.Bottomley> On Tue, 2016-12-06 at 15:12 +0100, Richard Levitte > wrote: > James.Bottomley> > In message < > 1480697558.2410.33.ca...@hansenpartnership.com> on Fri, > James.Bottomley> > 02 Dec 2016 08:52:38 -0800, James Bottomley < > James.Bottomley> > james.bottom...@hansenpartnership.com> said: > James.Bottomley> > > James.Bottomley> > When I made that argument, I hadn't thought and > felt things through > James.Bottomley> > entirely. Truth be told, I'm feeling very uneasy > handing over the > James.Bottomley> > reading and parsing of the PEM file to an engine. > However, handing > James.Bottomley> > over the decoded data and leaving it to the engine > to do something > James.Bottomley> > sensible with it, no issues at all. > James.Bottomley> > James.Bottomley> OK, can I pick on this a bit (I'll look at the store > stuff later and > James.Bottomley> reply after I've played with it)? What is it that > you imagine handing > James.Bottomley> to the engine? Some type of buffer that contains > the full PEM > James.Bottomley> representation? > > In my STORE branch, you will find this in > include/openssl/store_file.h: > > /* > * The try_decode function is called to check if the blob of data > can > * be used by this handler, and if it can, decodes it into a > supported > * OpenSSL and returns a STORE_INFO with the recorded data. > * Input: > *pem_name: If this blob comes from a PEM file, this > holds > * the PEM name. If it comes from another type > of > * file, this is NULL. > *blob: The blob of data to match with what this > handler > * can use. > *len: The length of the blob. > *handler_ctx: For a handler marked repeatable, this pointer > can > * be used to create a context for the handler. > IT IS > * THE HANDLER'S RESPONSIBILITY TO CREATE AND > DESTROY > * THIS CONTEXT APPROPRIATELY, i.e. create on > first call > * and destroy when about to return NULL. > *password_ui: Application UI method for getting a password. > *password_ui_data: > * Application data to be passed to password_ui > when > * it's called. > * Output: > *a STORE_INFO > */ > typedef STORE_INFO *(*STORE_FILE_try_decode_fn)(const char > *pem_name, > const unsigned > char *blob, > size_t len, > void > **handler_ctx, > const UI_METHOD > *password_ui, > void > *password_ui_data); > > This is the central function that tries to see if it can decode > whatever's thrown at it. As you can see, it gets handed the PEM name > if it's called as part of a PEM read. The |blob| is the decoded *and > decrypted* data. > > James.Bottomley> The reason I chose a BIO is that it's the basic > abstract data handler > James.Bottomley> for openssl. I can hand a buffer to the engine, > sure, but I'd need to > James.Bottomley> transform it to a BIO again (because it would need > PEM parsing and all > James.Bottomley> the PEM parsers speak BIOs), so it feels suboptimal. > > With a try_decode function, I really do not see why there's a need to > deal with the BIO... OK, I get what you want. There are two problems with this. The first is easy: I'm eventually going to need the BIO header as well because TPM2 keys are going to need to describe external things that aren't in the key representation (and even TPM1 keys might need to say who their parent key is). However, this is fixable by simply expanding your prototype above to have const char *pem_name and const char *pem_header ... is that OK? The next problem is that this is slightly harder simply to insert into the PEM code. The BIO parsing is done in PEM_bytes_read_bio() not PEM_read_bio_PrivateKey(). The easy way to cope with this would be to move PEM parsing into the ENGINE_find_engine_load_key() function and then hand the name, header, blob to the engine. The disadvantage of this is that we'll end up pulling the PEM apart twice: once in ENGINE_find_engine_load_key() and then again in PEM_bytes_read_bio(). Is this OK? If it is I can code up a v3. If not, I can think about how we might integrate this into PEM_bytes_read_bio(). James -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
In message <1481042048.4406.14.ca...@hansenpartnership.com> on Tue, 06 Dec 2016 08:34:08 -0800, James Bottomley said: James.Bottomley> On Tue, 2016-12-06 at 15:12 +0100, Richard Levitte wrote: James.Bottomley> > In message <1480697558.2410.33.ca...@hansenpartnership.com> on Fri, James.Bottomley> > 02 Dec 2016 08:52:38 -0800, James Bottomley < James.Bottomley> > james.bottom...@hansenpartnership.com> said: James.Bottomley> > James.Bottomley> > When I made that argument, I hadn't thought and felt things through James.Bottomley> > entirely. Truth be told, I'm feeling very uneasy handing over the James.Bottomley> > reading and parsing of the PEM file to an engine. However, handing James.Bottomley> > over the decoded data and leaving it to the engine to do something James.Bottomley> > sensible with it, no issues at all. James.Bottomley> James.Bottomley> OK, can I pick on this a bit (I'll look at the store stuff later and James.Bottomley> reply after I've played with it)? What is it that you imagine handing James.Bottomley> to the engine? Some type of buffer that contains the full PEM James.Bottomley> representation? In my STORE branch, you will find this in include/openssl/store_file.h: /* * The try_decode function is called to check if the blob of data can * be used by this handler, and if it can, decodes it into a supported * OpenSSL and returns a STORE_INFO with the recorded data. * Input: *pem_name: If this blob comes from a PEM file, this holds * the PEM name. If it comes from another type of * file, this is NULL. *blob: The blob of data to match with what this handler * can use. *len: The length of the blob. *handler_ctx: For a handler marked repeatable, this pointer can * be used to create a context for the handler. IT IS * THE HANDLER'S RESPONSIBILITY TO CREATE AND DESTROY * THIS CONTEXT APPROPRIATELY, i.e. create on first call * and destroy when about to return NULL. *password_ui: Application UI method for getting a password. *password_ui_data: * Application data to be passed to password_ui when * it's called. * Output: *a STORE_INFO */ typedef STORE_INFO *(*STORE_FILE_try_decode_fn)(const char *pem_name, const unsigned char *blob, size_t len, void **handler_ctx, const UI_METHOD *password_ui, void *password_ui_data); This is the central function that tries to see if it can decode whatever's thrown at it. As you can see, it gets handed the PEM name if it's called as part of a PEM read. The |blob| is the decoded *and decrypted* data. James.Bottomley> The reason I chose a BIO is that it's the basic abstract data handler James.Bottomley> for openssl. I can hand a buffer to the engine, sure, but I'd need to James.Bottomley> transform it to a BIO again (because it would need PEM parsing and all James.Bottomley> the PEM parsers speak BIOs), so it feels suboptimal. With a try_decode function, I really do not see why there's a need to deal with the BIO... -- Richard Levitte levi...@openssl.org OpenSSL Project http://www.openssl.org/~levitte/ -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Tue, 2016-12-06 at 15:12 +0100, Richard Levitte wrote: > In message <1480697558.2410.33.ca...@hansenpartnership.com> on Fri, > 02 Dec 2016 08:52:38 -0800, James Bottomley < > james.bottom...@hansenpartnership.com> said: > > James.Bottomley> On Thu, 2016-12-01 at 09:30 +0100, Richard Levitte > wrote: > James.Bottomley> > > James.Bottomley> > James Bottomley < > james.bottom...@hansenpartnership.com> skrev: (1 > James.Bottomley> > december 2016 07:36:26 CET) > James.Bottomley> > > On Thu, 2016-12-01 at 01:38 +0100, Richard > Levitte wrote: > James.Bottomley> > > > > James.Bottomley> > > > James Bottomley < > james.bottom...@hansenpartnership.com> skrev: (1 > James.Bottomley> > > > december 2016 00:42:09 CET) > James.Bottomley> [...] > James.Bottomley> > > > > On Thu, 2016-12-01 at 00:22 +0100, Richard > Levitte wrote: > James.Bottomley> > > > > > Generally speaking, I am unsure about your > solution. It seems > James.Bottomley> > > > > > like hack to fit a specific case where > something more general > James.Bottomley> > > > > > could be of greater service to others as > well. > James.Bottomley> > > > > > James.Bottomley> > > > > Well, the more adaptable patch set was the > previous one that > James.Bottomley> > > > > overloaded the meaning of key_id. This one > has a specific bio > James.Bottomley> > > > > mechanism for loading PEM files, so it only > really works for > James.Bottomley> > > > > engines that have PEM representable unloaded > keys (which, to be > James.Bottomley> > > > > fair, is almost all of them, since even the > USB crypto keys > James.Bottomley> > > > > have a wrapped format). > James.Bottomley> > > > > > James.Bottomley> > > > > I've tried to make it as generic as > possible, but I am > James.Bottomley> > > > > conditioned to think to my use case: TPM > keys. If you give an > James.Bottomley> > > > > example of another use case, it will help me > see where it > James.Bottomley> > > > > should be more generic. > James.Bottomley> > > > > James.Bottomley> > > > Among other things, I'd rather not encourage > an API that > James.Bottomley> > > > inherently makes the engine<->libcrypto tie > even tighter. Also, > James.Bottomley> > > > it puts a requirement on the engine to parse > PEM, which is > James.Bottomley> > > > unnecessary. > James.Bottomley> > James.Bottomley> Actually, I missed this initially. This is > definitely not a > James.Bottomley> requirement: engines that wish to parse PEM keys may > do so, but an > James.Bottomley> engine that doesn't simply doesn't implement the > callback. Only > James.Bottomley> engines that are loaded from the configuration file > actually get asked > James.Bottomley> if they understand the key, and then only if they > supply the callback, > James.Bottomley> so this is a decision on which engines to is made by > the admin or > James.Bottomley> packager (and the engine builder, who decides > whether to implement or > James.Bottomley> not). > > When I made that argument, I hadn't thought and felt things through > entirely. Truth be told, I'm feeling very uneasy handing over the > reading and parsing of the PEM file to an engine. However, handing > over the decoded data and leaving it to the engine to do something > sensible with it, no issues at all. OK, can I pick on this a bit (I'll look at the store stuff later and reply after I've played with it)? What is it that you imagine handing to the engine? Some type of buffer that contains the full PEM representation? The reason I chose a BIO is that it's the basic abstract data handler for openssl. I can hand a buffer to the engine, sure, but I'd need to transform it to a BIO again (because it would need PEM parsing and all the PEM parsers speak BIOs), so it feels suboptimal. James -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
In message <1480697558.2410.33.ca...@hansenpartnership.com> on Fri, 02 Dec 2016 08:52:38 -0800, James Bottomley said: James.Bottomley> On Thu, 2016-12-01 at 09:30 +0100, Richard Levitte wrote: James.Bottomley> > James.Bottomley> > James Bottomley skrev: (1 James.Bottomley> > december 2016 07:36:26 CET) James.Bottomley> > > On Thu, 2016-12-01 at 01:38 +0100, Richard Levitte wrote: James.Bottomley> > > > James.Bottomley> > > > James Bottomley skrev: (1 James.Bottomley> > > > december 2016 00:42:09 CET) James.Bottomley> [...] James.Bottomley> > > > > On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte wrote: James.Bottomley> > > > > > Generally speaking, I am unsure about your solution. It seems James.Bottomley> > > > > > like hack to fit a specific case where something more general James.Bottomley> > > > > > could be of greater service to others as well. James.Bottomley> > > > > James.Bottomley> > > > > Well, the more adaptable patch set was the previous one that James.Bottomley> > > > > overloaded the meaning of key_id. This one has a specific bio James.Bottomley> > > > > mechanism for loading PEM files, so it only really works for James.Bottomley> > > > > engines that have PEM representable unloaded keys (which, to be James.Bottomley> > > > > fair, is almost all of them, since even the USB crypto keys James.Bottomley> > > > > have a wrapped format). James.Bottomley> > > > > James.Bottomley> > > > > I've tried to make it as generic as possible, but I am James.Bottomley> > > > > conditioned to think to my use case: TPM keys. If you give an James.Bottomley> > > > > example of another use case, it will help me see where it James.Bottomley> > > > > should be more generic. James.Bottomley> > > > James.Bottomley> > > > Among other things, I'd rather not encourage an API that James.Bottomley> > > > inherently makes the engine<->libcrypto tie even tighter. Also, James.Bottomley> > > > it puts a requirement on the engine to parse PEM, which is James.Bottomley> > > > unnecessary. James.Bottomley> James.Bottomley> Actually, I missed this initially. This is definitely not a James.Bottomley> requirement: engines that wish to parse PEM keys may do so, but an James.Bottomley> engine that doesn't simply doesn't implement the callback. Only James.Bottomley> engines that are loaded from the configuration file actually get asked James.Bottomley> if they understand the key, and then only if they supply the callback, James.Bottomley> so this is a decision on which engines to is made by the admin or James.Bottomley> packager (and the engine builder, who decides whether to implement or James.Bottomley> not). When I made that argument, I hadn't thought and felt things through entirely. Truth be told, I'm feeling very uneasy handing over the reading and parsing of the PEM file to an engine. However, handing over the decoded data and leaving it to the engine to do something sensible with it, no issues at all. James.Bottomley> If I summarise the arguments about self identifying files from the v1 James.Bottomley> thread: James.Bottomley> James.Bottomley>1. We agreed that usability is greatly enhanced if openssl simply loads James.Bottomley> a key when presented with the file/uri etc. without the user having James.Bottomley> to specify what the format of a key is Check. My STORE branch is made to support that. James.Bottomley>2. For PEM files, we believe this is easy because the guards uniquely James.Bottomley> identify the file format, so whatever key the application needs, we James.Bottomley> can verify is contained in the thing presented Check. James.Bottomley>3. THere was more debate on the files that aren't fully self James.Bottomley> describing, like DER. The argument was that the DER structure is James.Bottomley> usually unique enough, so we could do this, but there were James.Bottomley> reservations. Yup... The current TSS KEY BLOB is among those reservations ;-) James.Bottomley>4. I thought there was agreement that we could move forwards with the James.Bottomley> PEM bit because it was uncontroversial We're not quite on the same page regarding the exact implementation. However, I think I have a solution... James.Bottomley> The current PEM key loading code is already hooked for PKCS8, to make James.Bottomley> the above a reality, we should hook it for pkcs12 as well. Once you James.Bottomley> have the store code, I believe it should be hooked for this as well. James.Bottomley> In this scenario, I don't quite see why it's not a hack to hook for James.Bottomley> pkcs8 (and presumably 12) but it is a hack to hook for engines and James.Bottomley> store. James.Bottomley> James.Bottomley> If we agree on DER, the ideal world becomes all apps simply use James.Bottomley> OPENSSL_load_privkey() and no-one needs to worry about format for key James.Bottomley> loading because any reasonable format just
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Thu, 2016-12-01 at 09:30 +0100, Richard Levitte wrote: > > James Bottomley skrev: (1 > december 2016 07:36:26 CET) > > On Thu, 2016-12-01 at 01:38 +0100, Richard Levitte wrote: > > > > > > James Bottomley skrev: (1 > > > december 2016 00:42:09 CET) [...] > > > > On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte wrote: > > > > > Generally speaking, I am unsure about your solution. It seems > > > > > like hack to fit a specific case where something more general > > > > > could be of greater service to others as well. > > > > > > > > Well, the more adaptable patch set was the previous one that > > > > overloaded the meaning of key_id. This one has a specific bio > > > > mechanism for loading PEM files, so it only really works for > > > > engines that have PEM representable unloaded keys (which, to be > > > > fair, is almost all of them, since even the USB crypto keys > > > > have a wrapped format). > > > > > > > > I've tried to make it as generic as possible, but I am > > > > conditioned to think to my use case: TPM keys. If you give an > > > > example of another use case, it will help me see where it > > > > should be more generic. > > > > > > Among other things, I'd rather not encourage an API that > > > inherently makes the engine<->libcrypto tie even tighter. Also, > > > it puts a requirement on the engine to parse PEM, which is > > > unnecessary. Actually, I missed this initially. This is definitely not a requirement: engines that wish to parse PEM keys may do so, but an engine that doesn't simply doesn't implement the callback. Only engines that are loaded from the configuration file actually get asked if they understand the key, and then only if they supply the callback, so this is a decision on which engines to is made by the admin or packager (and the engine builder, who decides whether to implement or not). > > > Also, we already have thoughts on loading keys by uri references, > > > and there may be new ideas and formats coming in the future. All > > > this is tied together and solving it one small hack at a time is > > > sub-optimal in the long run. > > > > > > I'll repeat myself again, please have a look at the STORE stuff > > > I'm working on. TPM files could very well serve as a first use > > > case. > > > > That's this new pull request: > > > > https://github.com/openssl/openssl/pull/2011/files > > Yes. > > > Just so I understand you, you mean register a store handler from > > the engine for the TPM BLOB PEM files so that a reference to the > > file via the store can use the PEM file as an EVP_PKEY? > > Essentially, yes. > > > That will work, I'm just not clear from the current pull request > > how the store would be integrated with the existing PEM file > > handler > > ... > > Will every application have to be modified to use the new store, or > > will you hook it like I did for the engine keys? > > Generally speaking, I was thinking that applications would move to > use the STORE API. > > > The think I'm looking for is to have a TPM PEM key file just work > > in place of a standard RSA private key file. > > I get your point. That could be med as a call after the > PEM_bytes_read_bio call. At that point, the necessary data for an > internal store loading are available. It's still a hack, but if you > must... If I summarise the arguments about self identifying files from the v1 thread: 1. We agreed that usability is greatly enhanced if openssl simply loads a key when presented with the file/uri etc. without the user having to specify what the format of a key is 2. For PEM files, we believe this is easy because the guards uniquely identify the file format, so whatever key the application needs, we can verify is contained in the thing presented 3. THere was more debate on the files that aren't fully self describing, like DER. The argument was that the DER structure is usually unique enough, so we could do this, but there were reservations. 4. I thought there was agreement that we could move forwards with the PEM bit because it was uncontroversial The current PEM key loading code is already hooked for PKCS8, to make the above a reality, we should hook it for pkcs12 as well. Once you have the store code, I believe it should be hooked for this as well. In this scenario, I don't quite see why it's not a hack to hook for pkcs8 (and presumably 12) but it is a hack to hook for engines and store. If we agree on DER, the ideal world becomes all apps simply use OPENSSL_load_privkey() and no-one needs to worry about format for key loading because any reasonable format just works. James -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
James Bottomley skrev: (1 december 2016 07:36:26 CET) >On Thu, 2016-12-01 at 01:38 +0100, Richard Levitte wrote: >> >> James Bottomley skrev: (1 >> december 2016 00:42:09 CET) >> > On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte wrote: >> > > This patch doesn't fit the rest... >> > >> > I'm not quite sure I follow why. >> >> It casts bp to const char *. That was for your earlier >> implementation, wasn't it? It doesn't fit the latest incarnation of >> your API. > >Good catch: a leftover I missed the warning for in the spray of other >compile information. Apparently the debug-linux-x86_64 target doesn't >have a -Werror ... I've added it so I should pick a problem like this >up easily. The debug config targets create binaries that can easily be run through a debugger, nothing more. What you're looking for is the configuration option --strict-warnings. >> > > Generally speaking, I am unsure about your solution. It seems >> > > like hack to fit a specific case where something more general >> > > could be of greater service to others as well. >> > >> > Well, the more adaptable patch set was the previous one that >> > overloaded the meaning of key_id. This one has a specific bio >> > mechanism for loading PEM files, so it only really works for >> > engines that have PEM representable unloaded keys (which, to be >> > fair, is almost all of them, since even the USB crypto keys have a >> > wrapped format). >> > >> > I've tried to make it as generic as possible, but I am conditioned >> > to think to my use case: TPM keys. If you give an example of >> > another use case, it will help me see where it should be more >> > generic. >> >> Among other things, I'd rather not encourage an API that inherently >> makes the engine<->libcrypto tie even tighter. Also, it puts a >> requirement on the engine to parse PEM, which is unnecessary. >> >> Also, we already have thoughts on loading keys by uri references, and >> there may be new ideas and formats coming in the future. All this is >> tied together and solving it one small hack at a time is sub-optimal >> in the long run. >> >> I'll repeat myself again, please have a look at the STORE stuff I'm >> working on. TPM files could very well serve as a first use case. > >That's this new pull request: > >https://github.com/openssl/openssl/pull/2011/files Yes. >Just so I understand you, you mean register a store handler from the >engine for the TPM BLOB PEM files so that a reference to the file via >the store can use the PEM file as an EVP_PKEY? Essentially, yes. >That will work, I'm just not clear from the current pull request how >the store would be integrated with the existing PEM file handler ... >Will every application have to be modified to use the new store, or >will you hook it like I did for the engine keys? Generally speaking, I was thinking that applications would move to use the STORE API. >The think I'm looking for is to have a TPM PEM key file just work in >place of a standard RSA private key file. I get your point. That could be med as a call after the PEM_bytes_read_bio call. At that point, the necessary data for an internal store loading are available. It's still a hack, but if you must... > >James > > >> > James >> > >> > >> > > Cheers >> > > Richard >> > > >> > > On November 30, 2016 4:27:49 PM GMT+01:00, James Bottomley < >> > > james.bottom...@hansenpartnership.com> wrote: >> > > > Before trying to process the PEM file, hand it to each of the >> > > > loaded >> > > > engines to see if they recognise the PEM guards. This uses the >> > > > new >> > > > bio based load key callback, so the engine must be loaded and >> > > > implement this callback to be considered. >> > > > >> > > > Signed-off-by: James Bottomley >> > > > --- >> > > > crypto/pem/pem_pkey.c | 4 >> > > > 1 file changed, 4 insertions(+) >> > > > >> > > > diff --git a/crypto/pem/pem_pkey.c b/crypto/pem/pem_pkey.c >> > > > index 04d6319..e3737f0 100644 >> > > > --- a/crypto/pem/pem_pkey.c >> > > > +++ b/crypto/pem/pem_pkey.c >> > > > @@ -85,6 +85,10 @@ EVP_PKEY *PEM_read_bio_PrivateKey(BIO *bp, >> > > > EVP_PKEY >> > > > **x, pem_password_cb *cb, >> > > > int slen; >> > > > EVP_PKEY *ret = NULL; >> > > > >> > > > +/* first check to see if an engine can load the PEM */ >> > > > +if (ENGINE_find_engine_load_key(NULL, &ret, (const char >> > > > *)bp, >> > > > cb, >> > > > u) == 1) >> > > > +return ret; >> > > > + >> > > > if (!PEM_bytes_read_bio(&data, &len, &nm, PEM_STRING_EVP_PKEY, >> > > > bp, >> > > > cb, >> > > > u)) >> > > > return NULL; >> > > > p = data; >> > > >> > > -- >> > > levi...@openssl.org >> >> -- >> Sent from my Android device with K-9 Mail. Please excuse my brevity. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Thu, 2016-12-01 at 01:38 +0100, Richard Levitte wrote: > > James Bottomley skrev: (1 > december 2016 00:42:09 CET) > > On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte wrote: > > > This patch doesn't fit the rest... > > > > I'm not quite sure I follow why. > > It casts bp to const char *. That was for your earlier > implementation, wasn't it? It doesn't fit the latest incarnation of > your API. Good catch: a leftover I missed the warning for in the spray of other compile information. Apparently the debug-linux-x86_64 target doesn't have a -Werror ... I've added it so I should pick a problem like this up easily. > > > Generally speaking, I am unsure about your solution. It seems > > > like hack to fit a specific case where something more general > > > could be of greater service to others as well. > > > > Well, the more adaptable patch set was the previous one that > > overloaded the meaning of key_id. This one has a specific bio > > mechanism for loading PEM files, so it only really works for > > engines that have PEM representable unloaded keys (which, to be > > fair, is almost all of them, since even the USB crypto keys have a > > wrapped format). > > > > I've tried to make it as generic as possible, but I am conditioned > > to think to my use case: TPM keys. If you give an example of > > another use case, it will help me see where it should be more > > generic. > > Among other things, I'd rather not encourage an API that inherently > makes the engine<->libcrypto tie even tighter. Also, it puts a > requirement on the engine to parse PEM, which is unnecessary. > > Also, we already have thoughts on loading keys by uri references, and > there may be new ideas and formats coming in the future. All this is > tied together and solving it one small hack at a time is sub-optimal > in the long run. > > I'll repeat myself again, please have a look at the STORE stuff I'm > working on. TPM files could very well serve as a first use case. That's this new pull request: https://github.com/openssl/openssl/pull/2011/files ? Just so I understand you, you mean register a store handler from the engine for the TPM BLOB PEM files so that a reference to the file via the store can use the PEM file as an EVP_PKEY? That will work, I'm just not clear from the current pull request how the store would be integrated with the existing PEM file handler ... Will every application have to be modified to use the new store, or will you hook it like I did for the engine keys? The think I'm looking for is to have a TPM PEM key file just work in place of a standard RSA private key file. James > > James > > > > > > > Cheers > > > Richard > > > > > > On November 30, 2016 4:27:49 PM GMT+01:00, James Bottomley < > > > james.bottom...@hansenpartnership.com> wrote: > > > > Before trying to process the PEM file, hand it to each of the > > > > loaded > > > > engines to see if they recognise the PEM guards. This uses the > > > > new > > > > bio based load key callback, so the engine must be loaded and > > > > implement this callback to be considered. > > > > > > > > Signed-off-by: James Bottomley > > > > --- > > > > crypto/pem/pem_pkey.c | 4 > > > > 1 file changed, 4 insertions(+) > > > > > > > > diff --git a/crypto/pem/pem_pkey.c b/crypto/pem/pem_pkey.c > > > > index 04d6319..e3737f0 100644 > > > > --- a/crypto/pem/pem_pkey.c > > > > +++ b/crypto/pem/pem_pkey.c > > > > @@ -85,6 +85,10 @@ EVP_PKEY *PEM_read_bio_PrivateKey(BIO *bp, > > > > EVP_PKEY > > > > **x, pem_password_cb *cb, > > > > int slen; > > > > EVP_PKEY *ret = NULL; > > > > > > > > +/* first check to see if an engine can load the PEM */ > > > > +if (ENGINE_find_engine_load_key(NULL, &ret, (const char > > > > *)bp, > > > > cb, > > > > u) == 1) > > > > +return ret; > > > > + > > > > if (!PEM_bytes_read_bio(&data, &len, &nm, PEM_STRING_EVP_PKEY, > > > > bp, > > > > cb, > > > > u)) > > > > return NULL; > > > > p = data; > > > > > > -- > > > levi...@openssl.org > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
James Bottomley skrev: (1 december 2016 00:42:09 CET) >On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte wrote: >> This patch doesn't fit the rest... > >I'm not quite sure I follow why. It casts bp to const char *. That was for your earlier implementation, wasn't it? It doesn't fit the latest incarnation of your API. >> Generally speaking, I am unsure about your solution. It seems like >> hack to fit a specific case where something more general could be of >> greater service to others as well. > >Well, the more adaptable patch set was the previous one that overloaded >the meaning of key_id. This one has a specific bio mechanism for >loading PEM files, so it only really works for engines that have PEM >representable unloaded keys (which, to be fair, is almost all of them, >since even the USB crypto keys have a wrapped format). > >I've tried to make it as generic as possible, but I am conditioned to >think to my use case: TPM keys. If you give an example of another use >case, it will help me see where it should be more generic. Among other things, I'd rather not encourage an API that inherently makes the engine<->libcrypto tie even tighter. Also, it puts a requirement on the engine to parse PEM, which is unnecessary. Also, we already have thoughts on loading keys by uri references, and there may be new ideas and formats coming in the future. All this is tied together and solving it one small hack at a time is sub-optimal in the long run. I'll repeat myself again, please have a look at the STORE stuff I'm working on. TPM files could very well serve as a first use case. >James > > >> Cheers >> Richard >> >> On November 30, 2016 4:27:49 PM GMT+01:00, James Bottomley < >> james.bottom...@hansenpartnership.com> wrote: >> > Before trying to process the PEM file, hand it to each of the >> > loaded >> > engines to see if they recognise the PEM guards. This uses the new >> > bio based load key callback, so the engine must be loaded and >> > implement this callback to be considered. >> > >> > Signed-off-by: James Bottomley >> > --- >> > crypto/pem/pem_pkey.c | 4 >> > 1 file changed, 4 insertions(+) >> > >> > diff --git a/crypto/pem/pem_pkey.c b/crypto/pem/pem_pkey.c >> > index 04d6319..e3737f0 100644 >> > --- a/crypto/pem/pem_pkey.c >> > +++ b/crypto/pem/pem_pkey.c >> > @@ -85,6 +85,10 @@ EVP_PKEY *PEM_read_bio_PrivateKey(BIO *bp, >> > EVP_PKEY >> > **x, pem_password_cb *cb, >> > int slen; >> > EVP_PKEY *ret = NULL; >> > >> > +/* first check to see if an engine can load the PEM */ >> > +if (ENGINE_find_engine_load_key(NULL, &ret, (const char *)bp, >> > cb, >> > u) == 1) >> > +return ret; >> > + >> > if (!PEM_bytes_read_bio(&data, &len, &nm, PEM_STRING_EVP_PKEY, bp, >> > cb, >> > u)) >> > return NULL; >> > p = data; >> >> -- >> levi...@openssl.org -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte wrote: > This patch doesn't fit the rest... I'm not quite sure I follow why. To allow engines to load PEM encoded engine keys in place of machine processed ones, the hook into the loader has to be in somewhere. This seems to be the most generic place to put the hook. > Generally speaking, I am unsure about your solution. It seems like > hack to fit a specific case where something more general could be of > greater service to others as well. Well, the more adaptable patch set was the previous one that overloaded the meaning of key_id. This one has a specific bio mechanism for loading PEM files, so it only really works for engines that have PEM representable unloaded keys (which, to be fair, is almost all of them, since even the USB crypto keys have a wrapped format). I've tried to make it as generic as possible, but I am conditioned to think to my use case: TPM keys. If you give an example of another use case, it will help me see where it should be more generic. James > Cheers > Richard > > On November 30, 2016 4:27:49 PM GMT+01:00, James Bottomley < > james.bottom...@hansenpartnership.com> wrote: > > Before trying to process the PEM file, hand it to each of the > > loaded > > engines to see if they recognise the PEM guards. This uses the new > > bio based load key callback, so the engine must be loaded and > > implement this callback to be considered. > > > > Signed-off-by: James Bottomley > > --- > > crypto/pem/pem_pkey.c | 4 > > 1 file changed, 4 insertions(+) > > > > diff --git a/crypto/pem/pem_pkey.c b/crypto/pem/pem_pkey.c > > index 04d6319..e3737f0 100644 > > --- a/crypto/pem/pem_pkey.c > > +++ b/crypto/pem/pem_pkey.c > > @@ -85,6 +85,10 @@ EVP_PKEY *PEM_read_bio_PrivateKey(BIO *bp, > > EVP_PKEY > > **x, pem_password_cb *cb, > > int slen; > > EVP_PKEY *ret = NULL; > > > > +/* first check to see if an engine can load the PEM */ > > +if (ENGINE_find_engine_load_key(NULL, &ret, (const char *)bp, > > cb, > > u) == 1) > > +return ret; > > + > > if (!PEM_bytes_read_bio(&data, &len, &nm, PEM_STRING_EVP_PKEY, bp, > > cb, > > u)) > > return NULL; > > p = data; > > -- > levi...@openssl.org -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
This patch doesn't fit the rest... Generally speaking, I am unsure about your solution. It seems like hack to fit a specific case where something more general could be of greater service to others as well. Cheers Richard On November 30, 2016 4:27:49 PM GMT+01:00, James Bottomley wrote: >Before trying to process the PEM file, hand it to each of the loaded >engines to see if they recognise the PEM guards. This uses the new >bio based load key callback, so the engine must be loaded and >implement this callback to be considered. > >Signed-off-by: James Bottomley >--- > crypto/pem/pem_pkey.c | 4 > 1 file changed, 4 insertions(+) > >diff --git a/crypto/pem/pem_pkey.c b/crypto/pem/pem_pkey.c >index 04d6319..e3737f0 100644 >--- a/crypto/pem/pem_pkey.c >+++ b/crypto/pem/pem_pkey.c >@@ -85,6 +85,10 @@ EVP_PKEY *PEM_read_bio_PrivateKey(BIO *bp, EVP_PKEY >**x, pem_password_cb *cb, > int slen; > EVP_PKEY *ret = NULL; > >+/* first check to see if an engine can load the PEM */ >+if (ENGINE_find_engine_load_key(NULL, &ret, (const char *)bp, cb, >u) == 1) >+return ret; >+ >if (!PEM_bytes_read_bio(&data, &len, &nm, PEM_STRING_EVP_PKEY, bp, cb, >u)) > return NULL; > p = data; -- levi...@openssl.org -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
> I suspect IBM will need to sign a CCLA ... they'll definitely need to know who > your lawyers are. We have a CCLA from IBM; contact Christopher Barrett. > I did check those links ... they don't have any governance information about > the actual openssl foundation that I can find. If you want particular details, email the info address I gave you. And again, that organization does not specify how the openssl project works. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Wed, 2016-11-30 at 19:32 +, Salz, Rich wrote: > > OK, so where is the foundation charter and who are your lawyers? > > Wow, this seems to have taken a turn to the unfriendly. I apologize > if I added to that. Sometimes a smiley doesn't wipe out all bad > impressions. No, it's standard if you insist on the CLA route: If you sign a CLA to an organization, you have to understand what the organization does before you understand what you're actually committing to: the actual commitment isn't within the four corners of the CLA. Your current ICLA gives effectively an unlimited perpetual licence to do anything with regard to sublicensing so the scope of that grant is actually governed by the bylaws and restrictions of the foundation itself because it is the grant recipient. For instance, the old OpenStack ICLA was fairly similar to yours so you had to dig into their bylaws to understand what they were actually committing to do with the code (basically sublicense it to all comers under ASL-2). The structure of the organisation matters a lot: unlimited grant to a corporate entity under a CLA is usually a bad idea because they often have nefarious plans to take your code private via a dual licence, or they might mean well, but their intentions become nullified if they get taken over. Foundations are usually better because their charter often restricts what they can actually do with the code and what happens to the grant in the event of dissolution or takeover, which is why reading and understanding the charter (and possibly the bylaws) is important. Usually I don't have to ask, all of this is simply available on the web most of the time. It's just the openssl foundation doesn't appear to have any of this online. I suspect IBM will need to sign a CCLA ... they'll definitely need to know who your lawyers are. > The OpenSSL Software Foundation is incorporated in the the state of > Delaware, United States, as a non-profit corporation. It does not > qualify as a tax-exempt charitable organisation under Section > 501(c)(3) of the U.S. Internal Revenue Code. You can email > i...@opensslfoundation.org with questions. > But do note that openssl open source project itself is not governed > by those entities, but rather by the collection of individuals known > as the development team. You can find more information by clicking > on the "policies" and "community" tab on the website. I did check those links ... they don't have any governance information about the actual openssl foundation that I can find. James -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
> OK, so where is the foundation charter and who are your lawyers? Wow, this seems to have taken a turn to the unfriendly. I apologize if I added to that. Sometimes a smiley doesn't wipe out all bad impressions. The OpenSSL Software Foundation is incorporated in the the state of Delaware, United States, as a non-profit corporation. It does not qualify as a tax-exempt charitable organisation under Section 501(c)(3) of the U.S. Internal Revenue Code. You can email i...@opensslfoundation.org with questions. But do note that openssl open source project itself is not governed by those entities, but rather by the collection of individuals known as the development team. You can find more information by clicking on the "policies" and "community" tab on the website. Hope this helps. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Wed, 2016-11-30 at 17:59 +, Salz, Rich wrote: > > > Plus the DCO is industry best practice: even OpenStack is adopting > > it after a > > long struggle. > > Great. Good for them. > > This is what we're doing. > > :) OK, so where is the foundation charter and who are your lawyers? James -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
> Plus the DCO is industry best practice: even OpenStack is adopting it after a > long struggle. Great. Good for them. This is what we're doing. :) -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
On Wed, 2016-11-30 at 16:04 +, Salz, Rich wrote: > > Groan ... since you're changing licences, I don't suppose you'd > > consider moving to a DCO model. > > Sorry, no. Legal advice and best practices. Interesting: whose legal advice? I assumed you were talking to the SFLC and I thought they were mostly pre-programmed not to recommend CLAs after their FSF experience. I'm asking professionally because the knee jerk reaction of most lawyers is to recommend a CLA and there's a group programme (run by the LF) to help them kick the habit. Plus the DCO is industry best practice: even OpenStack is adopting it after a long struggle. James -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
> Actually, being a kernel developer, email is far easier. I'll send a pull > request > when everyone's OK with the mechanism, plus it will need tests and other > things. Well... okay. I don't know how the community will react. But I *do* know that the team prefers things as PR's. > Groan ... since you're changing licences, I don't suppose you'd consider > moving to a DCO model. Sorry, no. Legal advice and best practices. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
> Thanks for working to improve openssl. You're welcome. > It is probably easier for you to do a GitHub pull request and then > have discussion here, pointing to that PR. Actually, being a kernel developer, email is far easier. I'll send a pull request when everyone's OK with the mechanism, plus it will need tests and other things. > And also, before any of this code could be used, we'll need the > appropriate CLA. Groan ... since you're changing licences, I don't suppose you'd consider moving to a DCO model. It's a whole lot less bureaucratic and much easier to keep track of. James -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys
Thanks for working to improve openssl. It is probably easier for you to do a GitHub pull request and then have discussion here, pointing to that PR. And also, before any of this code could be used, we'll need the appropriate CLA. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev