Re: [openssl-dev] (future) STORE vs X509_LOOKUP_METHOD by_dir
In message <589b86c1.10...@roumenpetrov.info> on Wed, 08 Feb 2017 22:59:45 +0200, Roumen Petrovsaid: openssl> Hi Richard, openssl> openssl> Richard Levitte wrote: openssl> > Hi, openssl> > openssl> > I've some ponderings that I need to bounce a bit with you all. openssl> > openssl> > Some have talked about replace the X509_LOOKUP_METHOD openssl> X.509 lookup method could return certificate , revocation list or openssl> EVP_KEY (structure x509_object_st). openssl> openssl> Unfortunately functionality of EVP_KEY was never implemented. openssl> Another point is specific names of structures - x509_lookup_method_st openssl> , x509_lookup_st, x509_object_st. openssl> Third point is quite specific implementation - functions not just to openssl> retrieve objects( X.509 or CRL) but to fill them into "context of X509 openssl> store". openssl> openssl> Current lookup functionality look like "store" but implementation is openssl> specific to X.509 store. openssl> openssl> openssl> > bit with the openssl> > STORE module I'm building, and while STORE isn't ready for it yet openssl> openssl> I hope that you store functionality will fill gap between load of keys openssl> and load of certificates (+crl). openssl> openssl> Loadable module (engine) has interface to load key(private or public) openssl> but lack load of X.509 certificates or CRL. So far, key parameters, pkeys, certs and crls are covered... oh, and names, if the given URI would return a list of names (file:/foo/bar/ would typically do that, for example). openssl> > , I openssl> > have some thoughts on how the two can approach each other. This would openssl> > involve one or two hooks / callbacks, that a STORE user could specify openssl> > (details later) to pick and choose freely among the objects that the openssl> > STORE module finds (be it on file or whatever else that can be openssl> > represented as a URI). openssl> I think that functionality requires three phases : openssl> 1) instantiation : at this point store is created openssl> 2) specification (optional): set or check capability of store. For openssl> instance store could return only X.509 certificates or to request openssl> store to return only keys. openssl> 3) inquiry: fetch data based on specified criteria. Previous attempts at a STORE effort were along those lines of thinking. Unfortunately, they got stuck in the exact specification of attributes and how to combine them (i.e. forming a "language" of sorts), and then to figure out how to make that pratical, especially for the engines that would have to intrepret it. So this time around, I'm trying to make OpenSSL fairly agnostic and leave the exact spec up to be specified by the user or application in the URI and letting the diverse engines interpret the URI components at their leasure. Objects would then be returned according to those specs in form of one object at a time, be it parameters, keys, certs, crls or names, for the application or other parts of OpenSSL to do whatever they wish with. Given this, I determined yesterday that there would really just be one hook / callback that makes sense, and that's one that can massage or possibly discard returned objects before they make their way all the way to the calling application. Something like this. openssl> > The troublesome part would be to try to mimic by_dir... It highly openssl> > depends on the specified paths to really be directories, and that it openssl> > should find what it wants by adding very specific file names (a hash openssl> > of the subject name with a ".{n}" or ".r{n}" extension for X.509 certs openssl> > and for X.509 CRLs). And sure, that works, but will really only work openssl> > with regular files. openssl> I'm not sure what is issue. openssl> openssl> Lets see X.509 lookup method get_by_subject. openssl> openssl> - by_dir openssl> 2) specification : set directory(path), limit results to X.509 or CRL openssl> and may be to inform store that questions will be performed by openssl> subject. openssl> openssl> For instance URI scheme could befile://path?certificate="name" openssl> openssl> 3) query : from subject calculate hash and then process "{hash}.{n}" openssl> or "{hash}.r{n}" depending from URI openssl> openssl> - Ldap openssl> It is similar, URI is described in RFCs - at point 2) set host, port, openssl> base distinguished name, attribute (for instance cACertificate), openssl> construct filter from specified name. The issue is there, glaring back at you. How will by_dir know exactly how to massage the URI? Sure, we could program in the big known ones plus our own mangling of the file: scheme, but then what? What about the vendor who runs his own scheme foo:? How do we know how to encode their keystore specs into a URI with their scheme? There needs to be a solution where OpenSSL can be specification agnostic. So far, I've this idea of returned names that form a "directory listing", and it wouldn't be
Re: [openssl-dev] (future) STORE vs X509_LOOKUP_METHOD by_dir
Hi Richard, Richard Levitte wrote: Hi, I've some ponderings that I need to bounce a bit with you all. Some have talked about replace the X509_LOOKUP_METHOD X.509 lookup method could return certificate , revocation list or EVP_KEY (structure x509_object_st). Unfortunately functionality of EVP_KEY was never implemented. Another point is specific names of structures - x509_lookup_method_st , x509_lookup_st, x509_object_st. Third point is quite specific implementation - functions not just to retrieve objects( X.509 or CRL) but to fill them into "context of X509 store". Current lookup functionality look like "store" but implementation is specific to X.509 store. bit with the STORE module I'm building, and while STORE isn't ready for it yet I hope that you store functionality will fill gap between load of keys and load of certificates (+crl). Loadable module (engine) has interface to load key(private or public) but lack load of X.509 certificates or CRL. , I have some thoughts on how the two can approach each other. This would involve one or two hooks / callbacks, that a STORE user could specify (details later) to pick and choose freely among the objects that the STORE module finds (be it on file or whatever else that can be represented as a URI). I think that functionality requires three phases : 1) instantiation : at this point store is created 2) specification (optional): set or check capability of store. For instance store could return only X.509 certificates or to request store to return only keys. 3) inquiry: fetch data based on specified criteria. The troublesome part would be to try to mimic by_dir... It highly depends on the specified paths to really be directories, and that it should find what it wants by adding very specific file names (a hash of the subject name with a ".{n}" or ".r{n}" extension for X.509 certs and for X.509 CRLs). And sure, that works, but will really only work with regular files. I'm not sure what is issue. Lets see X.509 lookup method get_by_subject. - by_dir 2) specification : set directory(path), limit results to X.509 or CRL and may be to inform store that questions will be performed by subject. For instance URI scheme could befile://path?certificate="name" 3) query : from subject calculate hash and then process "{hash}.{n}" or "{hash}.r{n}" depending from URI - Ldap It is similar, URI is described in RFCs - at point 2) set host, port, base distinguished name, attribute (for instance cACertificate), construct filter from specified name. What if someone would specify a LDAP URI that can return a bunch of objects? So... my ponderings are going along these lines: 1. Should the directory X509_LOOKUPs be restricted to on disk directories, or should "directory" be redefined as "whatever URI that returns a collection of objects"? The latter would mean that all those objects get loaded and that a hook / callback would then be called to check if it's an object that corresponds to what we search for. I think that replacement of "by_dir" lookup has to be restricted to file system operation. Files could be located on network, memory not only on disk. 2. For on disk directories, should we preserve the rehash file form? In other words, if we decide to load everything we can find, shall we restrict the loading to files matching the regexp [0-9a-f]{8}\.r?[0-9]+ ? If not, are we about to create a new form of key store for ourselves and our users? Should we? For hash-dir please keep current file name format. Quite a lot also depends on what OpenSSL version we aim for. I would very much like to see the STORE module itself become part of 1.1.1, but a new key store to replace our current rehash links will obviously have to wait 'til 1.2.0. Cheers, Richard Roumen -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] (future) STORE vs X509_LOOKUP_METHOD by_dir
On Sun, 2017-02-05 at 16:47 +0100, Richard Levitte wrote: > Hi, > > I've some ponderings that I need to bounce a bit with you all. > > Some have talked about replace the X509_LOOKUP_METHOD bit with the > STORE module I'm building, and while STORE isn't ready for it yet, I > have some thoughts on how the two can approach each other. This > would > involve one or two hooks / callbacks, that a STORE user could specify > (details later) to pick and choose freely among the objects that the > STORE module finds (be it on file or whatever else that can be > represented as a URI). Just to add something to your thinking - so there is a p11-kit-trust PKCS11 module which provides all the CA certificates that should be trusted on the system via individual PKCS11 certificate objects. Could it be somehow accommodated with the STORE module approach? Mozilla NSS and GnuTLS can use this PKCS11 module directly as a trust store, we would like to add the same functionality to OpenSSL. -- Tomas Mraz No matter how far down the wrong road you've gone, turn back. Turkish proverb (You'll never know whether the road is wrong though.) -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
[openssl-dev] (future) STORE vs X509_LOOKUP_METHOD by_dir
Hi, I've some ponderings that I need to bounce a bit with you all. Some have talked about replace the X509_LOOKUP_METHOD bit with the STORE module I'm building, and while STORE isn't ready for it yet, I have some thoughts on how the two can approach each other. This would involve one or two hooks / callbacks, that a STORE user could specify (details later) to pick and choose freely among the objects that the STORE module finds (be it on file or whatever else that can be represented as a URI). The troublesome part would be to try to mimic by_dir... It highly depends on the specified paths to really be directories, and that it should find what it wants by adding very specific file names (a hash of the subject name with a ".{n}" or ".r{n}" extension for X.509 certs and for X.509 CRLs). And sure, that works, but will really only work with regular files. What if someone would specify a LDAP URI that can return a bunch of objects? So... my ponderings are going along these lines: 1. Should the directory X509_LOOKUPs be restricted to on disk directories, or should "directory" be redefined as "whatever URI that returns a collection of objects"? The latter would mean that all those objects get loaded and that a hook / callback would then be called to check if it's an object that corresponds to what we search for. 2. For on disk directories, should we preserve the rehash file form? In other words, if we decide to load everything we can find, shall we restrict the loading to files matching the regexp [0-9a-f]{8}\.r?[0-9]+ ? If not, are we about to create a new form of key store for ourselves and our users? Should we? Quite a lot also depends on what OpenSSL version we aim for. I would very much like to see the STORE module itself become part of 1.1.1, but a new key store to replace our current rehash links will obviously have to wait 'til 1.2.0. Cheers, Richard -- Richard Levitte levi...@openssl.org OpenSSL Project http://www.openssl.org/~levitte/ -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev