Re: X509_get_pubkey() in OpenSSL 3.0?

[Selva Nair]

> X509 cert = SSL_CTX_get0_certificate(ctx);
> EVP_PKEY pkey = X509_get_pubkey(cert);
>

*cert and *pkey ...

Re: X509_get_pubkey() in OpenSSL 3.0?

[Viktor Dukhovni]

On Tue, Nov 02, 2021 at 08:28:01PM +, Jason Schultz wrote:
> Victor-
> 
> I can't seem to find any documentation on SSL_CTX_get0_privatekey(),
> but by the name of it, it sounds like it's getting the private key;
> I'm trying to get the public key.

It does appear to be "under-documented" (i.e. not at all), perhaps
that could be addressed.  Note that you can get a public key from
the private key.

> That said, I should probably explain more of why I'm doing what I'm
> doing, because there may be an easier way all together. Basically, we
> allow configuring RSA or EC certificates/keys, and I want to get the
> public key so I can check the type of key with a call to:
> 
>  EVP_PKEY_base_id(pubkey);

Which you can equally get from the private key, but there's also:

SSL_CTX_get0_certificate(sadly also "under-documented")

which means that you certainly don't need to read the PEM file again.

> So maybe there's a better way? After I call:
> 
>  SSL_CTX_use_certificate_file(ctx,,SSL_FILETYPE_PEM);
> 
> Is there an API I can call passing the ctx that will tell me what type
> of certificate is in use for that ctx? Or something else along those
> lines?

Also once you have an (SSL *) handle you can call SSL_get_certificate(3)
which is documented, and proceed from there.

> It's very possible I'm overcomplicating things with the fopen(),
> PEM_read_X509(), X509_get_pubkey() sequence, so any suggestions on how
> to better accomplish this verification are welcome.

Yes.  Query the data you already have in memory.

Note that a context can be configured with *both* an RSA certificate
chain *and* an EC certificate chain.  The above functions return the one
most recently loaded, but a handshake with a peer may select one of
the other chains.

Most applications only load keys for just one algorithm, but in general
more may be present.

It is surprising that you care which algorithm is in cert/keys.  So
long as it loads, why does it matter?

If you want to log data for each handshake, at that point you can use
SSL_get_certificate(3) for the cert actually selected by this handshake
on the local end:


https://github.com/vdukhovni/postfix/blob/master/postfix/src/tls/tls_misc.c#L950-L953

-- 
Viktor.

Re: X509_get_pubkey() in OpenSSL 3.0?

[Selva Nair]

On Tue, Nov 2, 2021 at 3:42 PM Jason Schultz  wrote:

> I thought I should start a new thread since this question was buried in my
> "FIPS" thread and I dont' think it has anything to do with FIPS and OpenSSL
> providers. I'm hitting another problem that I think is related to the
> migration to OpenSSL 3.0, as this code works with OpenSSL 1.1.1 (and 1.0.2
> before it). When looking at the documentation pages for 1.1.1 vs 3.0, I'm
> not seeing any differences between the OpenSSL APIs I'm calling in the 2
> different release levels.
>
> Here is the sequence, I'm basically setting up my certificate and private
> key, both in PEM format, for the server, then I need to extract some
> information from them:
>
> ctx = SSL_CTX_new_ex(non_fips_libctx, NULL, TLS_method());
> SSL_CTX_use_PrivateKey_file(ctx,,SSL_FILETYPE_PEM);
> SSL_CTX_use_certificate_file(ctx,,SSL_FILETYPE_PEM);
>
SSL_CTX_check_private_key(ctx);
> fp = fopen(, "r");
> mycert = PEM_read_X509(fp, NULL, 0, NULL)
>

>
All functions return good statuses or non-NULL pointers until the last one,
> X509_get_pubkey() returns NULL.
>

You probably do not have any providers loaded in the default libctx (NULL).
As the first 4 calls have succeeded, non_fips_libctx does have a working
provider.

Check your code for what is stopping default provider getting auto-loaded
into the default libctx (config file misconfiguration or explicit provider
loading?). Or try the following after successfully loading the cert to the
SSL context (ctx):

X509 cert = SSL_CTX_get0_certificate(ctx);
EVP_PKEY pkey = X509_get_pubkey(cert);

This should work as the decoding will happen in non_fips_libctx.

Selva

Re: X509_get_pubkey() in OpenSSL 3.0?

[Jason Schultz]

Victor-

I can't seem to find any documentation on SSL_CTX_get0_privatekey(), but by the 
name of it, it sounds like it's getting the private key; I'm trying to get the 
public key.

That said, I should probably explain more of why I'm doing what I'm doing, 
because there may be an easier way all together. Basically, we allow 
configuring RSA or EC certificates/keys, and I want to get the public key so I 
can check the type of key with a call to:

 EVP_PKEY_base_id(pubkey);

I check the return value from that against, for example, EVP_PKEY_EC to verify 
an EC certificate was configured, as opposed to RSA. That's the gist of it, 
without going into too many application specific details.

So maybe there's a better way? After I call:

 SSL_CTX_use_certificate_file(ctx,,SSL_FILETYPE_PEM);

Is there an API I can call passing the ctx that will tell me what type of 
certificate is in use for that ctx? Or something else along those lines?

It's very possible I'm overcomplicating things with the fopen(), 
PEM_read_X509(), X509_get_pubkey() sequence, so any suggestions on how to 
better accomplish this verification are welcome.

Regards,

Jason


> I thought I should start a new thread since this question was buried in my 
> "FIPS" thread and I dont' think it has anything to do with FIPS and OpenSSL 
> providers. I'm hitting another problem that I think is related to the 
> migration to OpenSSL 3.0, as this code works with OpenSSL 1.1.1 (and 1.0.2 
> before it). When looking at the documentation pages for 1.1.1 vs 3.0, I'm not 
> seeing any differences between the OpenSSL APIs I'm calling in the 2 
> different release levels.
>
> Here is the sequence, I'm basically setting up my certificate and private 
> key, both in PEM format, for the server, then I need to extract some 
> information from them:
>
> ctx = SSL_CTX_new_ex(non_fips_libctx, NULL, TLS_method());
> SSL_CTX_use_PrivateKey_file(ctx,,SSL_FILETYPE_PEM);
> SSL_CTX_use_certificate_file(ctx,,SSL_FILETYPE_PEM);
> SSL_CTX_check_private_key(ctx);
> fp = fopen(, "r");
> mycert = PEM_read_X509(fp, NULL, 0, NULL);
> pkey = X509_get_pubkey(mycert);

Without addressing the question of why you're unable to get the public
key handle from the certificate, why not just:

 pkey = SSL_CTX_get0_privatekey(ctx){

and skip reading the cert again?

--
Viktor.

自动回复: Re: X509_get_pubkey() in OpenSSL 3.0?

[562430030 via openssl-users]

您好，您的邮件我已收到，我会尽快阅读，谢谢！

Re: X509_get_pubkey() in OpenSSL 3.0?

[Viktor Dukhovni]

On Tue, Nov 02, 2021 at 07:42:15PM +, Jason Schultz wrote:

> I thought I should start a new thread since this question was buried in my 
> "FIPS" thread and I dont' think it has anything to do with FIPS and OpenSSL 
> providers. I'm hitting another problem that I think is related to the 
> migration to OpenSSL 3.0, as this code works with OpenSSL 1.1.1 (and 1.0.2 
> before it). When looking at the documentation pages for 1.1.1 vs 3.0, I'm not 
> seeing any differences between the OpenSSL APIs I'm calling in the 2 
> different release levels.
> 
> Here is the sequence, I'm basically setting up my certificate and private 
> key, both in PEM format, for the server, then I need to extract some 
> information from them:
> 
> ctx = SSL_CTX_new_ex(non_fips_libctx, NULL, TLS_method());
> SSL_CTX_use_PrivateKey_file(ctx,,SSL_FILETYPE_PEM);
> SSL_CTX_use_certificate_file(ctx,,SSL_FILETYPE_PEM);
> SSL_CTX_check_private_key(ctx);
> fp = fopen(, "r");
> mycert = PEM_read_X509(fp, NULL, 0, NULL);
> pkey = X509_get_pubkey(mycert);

Without addressing the question of why you're unable to get the public
key handle from the certificate, why not just:

 pkey = SSL_CTX_get0_privatekey(ctx){

and skip reading the cert again?

-- 
Viktor.

Re: X509_get_pubkey() in OpenSSL 3.0?

[Jason Schultz]

Sorry, I send this before finishing the Aside. I ended up calling these APIs:

DECODER_ctx = OSSL_DECODER_CTX_new_for_pkey(EVP_PKEY **pkey,
   const char *input_type,
   const char *input_struct,
   const char *keytype, int selection,
   OSSL_LIB_CTX *libctx, const char *propquery);

OSSL_DECODER_from_fp(dctx, fp);

SSL_CTX_set0_tmp_dh_pkey(ctx, dh_pkey);

To get the DH set up. I can't test this until I get past the intial problem, 
but I tried using the above functions for getting the public key from the 
certificate file, and the problem is still the same.

An example of a certificate I'm using that is failing is below:

# openssl x509 -noout -text -in /etc/ssl/certs/servercert.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 286 (0x11e)
Signature Algorithm: ecdsa-with-SHA256
Issuer: C = US, ST = PA, L = City, O = Example, OU = Networking, CN = 
JCS-EC-CA, emailAddress = joe@example.com
Validity
Not Before: Mar  3 18:15:13 2021 GMT
Not After : Nov 28 18:15:13 2023 GMT
Subject: C = US, ST = PA, L = City, O = Example, OU = Networking, CN = 
JCS-EC-server, emailAddress = joe1@example.com
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
04:c2:1a:0c:76:dc:d9:b9:3d:49:f6:60:72:fa:18:
c2:e9:31:bc:64:42:24:02:61:06:80:e0:69:2c:5a:
c3:0e:e4:09:ce:79:64:f8:fd:b0:65:63:e1:e9:35:
34:ff:48:58:aa:72:21:20:c1:ce:f2:7a:90:2b:c7:
e7:b7:76:ad:88
ASN1 OID: prime256v1
NIST CURVE: P-256
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
EF:1E:DA:81:3A:43:AA:12:E4:53:B6:4B:E5:A0:3D:AB:CD:30:6B:2F
X509v3 Authority Key Identifier:
60:A2:DC:A3:7A:FB:78:A9:92:5F:88:B5:1A:03:65:9C:F4:10:CD:38
Signature Algorithm: ecdsa-with-SHA256
Signature Value:
30:46:02:21:00:dc:fc:69:9c:a6:54:e1:fa:3c:e7:65:9f:cc:
d5:a4:a2:3a:a7:b1:37:79:60:d0:61:9e:87:15:79:09:0f:34:
14:02:21:00:fd:29:34:bf:bb:c5:02:0d:9a:04:44:6e:94:22:
52:b4:0e:ab:1f:3d:15:5c:07:47:eb:76:68:80:f9:72:96:f6



From: openssl-users  on behalf of Jason 
Schultz 
Sent: Tuesday, November 2, 2021 7:42 PM
To: openssl-users@openssl.org 
Subject: X509_get_pubkey() in OpenSSL 3.0?

I thought I should start a new thread since this question was buried in my 
"FIPS" thread and I dont' think it has anything to do with FIPS and OpenSSL 
providers. I'm hitting another problem that I think is related to the migration 
to OpenSSL 3.0, as this code works with OpenSSL 1.1.1 (and 1.0.2 before it). 
When looking at the documentation pages for 1.1.1 vs 3.0, I'm not seeing any 
differences between the OpenSSL APIs I'm calling in the 2 different release 
levels.

Here is the sequence, I'm basically setting up my certificate and private key, 
both in PEM format, for the server, then I need to extract some information 
from them:

ctx = SSL_CTX_new_ex(non_fips_libctx, NULL, TLS_method());
SSL_CTX_use_PrivateKey_file(ctx,,SSL_FILETYPE_PEM);
SSL_CTX_use_certificate_file(ctx,,SSL_FILETYPE_PEM);
SSL_CTX_check_private_key(ctx);
fp = fopen(, "r");
mycert = PEM_read_X509(fp, NULL, 0, NULL);
pkey = X509_get_pubkey(mycert);

All functions return good statuses or non-NULL pointers until the last one, 
X509_get_pubkey() returns NULL. I have tried this with RSA and Elliptic Curve 
cert/key pairs. I have tried with pairs created with OpenSSL 1.1.1 as well as 
3.0 via the command line.

This seems like a pretty straightforward operation, but it appears that 
key->pkey is NULL in the code below:

EVP_PKEY *X509_PUBKEY_get0(const X509_PUBKEY *key)
{
if (key == NULL) {
ERR_raise(ERR_LIB_X509, ERR_R_PASSED_NULL_PARAMETER);
return NULL;
}

if (key->pkey == NULL) {
/* We failed to decode the key when we loaded it, or it was never set */
ERR_raise(ERR_LIB_EVP, EVP_R_DECODE_ERROR);
return NULL;
}

return key->pkey;
}

I got to this code from the error information I dumped from OpenSSL:

00079FF8647F:error:0372:digital envelope routines:(unknown 
function):decode error:crypto/x509/x_pubkey.c:444:

I can paste certificates if anyone thinks it will help, but I've tried several 
types of cert/key pairs, and using the command line to do "openssl x509 
-pubkey..." displays the public key just fine with the certificate file in 
question.

Here is an example of the openssl command line to create one of the cert/key 
pairs that hits this error:

X509_get_pubkey() in OpenSSL 3.0?

[Jason Schultz]

I thought I should start a new thread since this question was buried in my 
"FIPS" thread and I dont' think it has anything to do with FIPS and OpenSSL 
providers. I'm hitting another problem that I think is related to the migration 
to OpenSSL 3.0, as this code works with OpenSSL 1.1.1 (and 1.0.2 before it). 
When looking at the documentation pages for 1.1.1 vs 3.0, I'm not seeing any 
differences between the OpenSSL APIs I'm calling in the 2 different release 
levels.

Here is the sequence, I'm basically setting up my certificate and private key, 
both in PEM format, for the server, then I need to extract some information 
from them:

ctx = SSL_CTX_new_ex(non_fips_libctx, NULL, TLS_method());
SSL_CTX_use_PrivateKey_file(ctx,,SSL_FILETYPE_PEM);
SSL_CTX_use_certificate_file(ctx,,SSL_FILETYPE_PEM);
SSL_CTX_check_private_key(ctx);
fp = fopen(, "r");
mycert = PEM_read_X509(fp, NULL, 0, NULL);
pkey = X509_get_pubkey(mycert);

All functions return good statuses or non-NULL pointers until the last one, 
X509_get_pubkey() returns NULL. I have tried this with RSA and Elliptic Curve 
cert/key pairs. I have tried with pairs created with OpenSSL 1.1.1 as well as 
3.0 via the command line.

This seems like a pretty straightforward operation, but it appears that 
key->pkey is NULL in the code below:

EVP_PKEY *X509_PUBKEY_get0(const X509_PUBKEY *key)
{
if (key == NULL) {
ERR_raise(ERR_LIB_X509, ERR_R_PASSED_NULL_PARAMETER);
return NULL;
}

if (key->pkey == NULL) {
/* We failed to decode the key when we loaded it, or it was never set */
ERR_raise(ERR_LIB_EVP, EVP_R_DECODE_ERROR);
return NULL;
}

return key->pkey;
}

I got to this code from the error information I dumped from OpenSSL:

00079FF8647F:error:0372:digital envelope routines:(unknown 
function):decode error:crypto/x509/x_pubkey.c:444:

I can paste certificates if anyone thinks it will help, but I've tried several 
types of cert/key pairs, and using the command line to do "openssl x509 
-pubkey..." displays the public key just fine with the certificate file in 
question.

Here is an example of the openssl command line to create one of the cert/key 
pairs that hits this error:

openssl req -new -x509 -nodes -newkey ec:<(openssl ecparam -name secp384r1) 
-keyout threecert.key -out threecert.crt -days 365

Aside: While I was waiting for an answer on this, I started working on removing 
some of the deprecated functions in my code, for example, PEM_read_DHparams(). 
I ended up finding the DECODER functions and plan on doing something like:


Thanks,

Jason

Re: SSL and "custom" EVP_KEY

[Matt Caswell]

On 02/11/2021 04:42, Alex Dankow wrote:

Matt,

Thank you very much for your response. I understand that the FIPS 
certified OpenSSL module is long awaited and the team was quite limited 
in time to complete all features.
I tried Windows certificates +Openssl because it implements the most 
common scenario: you can get a certificate to Openssl in DER format, but 
you can't get the private key. HSMs, KMIP servers or Windows don't let 
you export the private key.  It can be only virtualized (You sure know 
it, but I'm writing it also for our readers.)
So, when I load X509 from storage it is handled by OpenSSL directly and 
gets type "RSA".When the private key is supplied, it has type of 
"MYTYPE" and actually represents a handle. If I skip the key matching 
part, "openssl ca" works and a cert request can be signed with such a 
handle.


As per Tomas's comment if it really is an RSA key then you should call 
it that. It does not matter that the private key cannot be extracted - 
it is perfectly valid to have a provider that does not export its 
private keys. We make no requirements on provider authors as to how keys 
should be represented internally to the provider. If its a handle rather 
than an "RSA" structure - then that's fine - OpenSSL doesn't care.




As I understand, Openssl doesn't handle it completely yet.


OpenSSL doesn't handle "pluggable" signatures. So you can't add a 
signature algorithm that libssl doesn't know about and get it to work. 
It should work just fine for RSA keys though since libssl knows what to 
do with those.


Matt


But it was 
planned so and maybe we will see it in the future.

If ENGINE is now deprecated (is it?), what HSM vendors should do?
--
Alex Dankow













On Fri, Oct 29, 2021 at 10:11 PM Matt Caswell > wrote:


Hi Alex,

On 29/10/2021 14:32, Alex Dankow wrote:
 > Hi OpenSSL team!
 >
 > I wrote a provider for Windows certificates and implemented
"openssl ca".
 > Now, I think it would be fun to see a HTTPS server using
certificates
 > installed in Windows storage.
Nice!

 >
 > Certificate is loaded using load_cert_pass (taken from apps.c) with
 > custom uri "wincert://11:22:33",  private key is loaded with
 > load_key from apps.c too. It works, but ...
 > When I use  SSL_CTX_use_PrivateKey(ctx, myprivk)  the key is
declined.
 > OpenSSL compares strings and expects "rsaEncryption", and so on
instead
 > of "MYKEY". Why ?

It's not entirely clear to me what you are attempting here. Are your
certificates/keys stored in Windows storage standard RSA/ECDSA etc
certs? Or are they using some custom algorithm?

If they are standard RSA/ECSDA certs then you should be using the
correct standard algorithm names in you keymgmt etc and it should all
"just work".

Unfortunately, in 3.0, libssl only supports standard algorithms. We
have
discussed a "pluggable" signature scheme mechanism which would enable
plugging in arbitrary algorithms but it didn't make it for 3.0:

https://github.com/openssl/openssl/issues/10512


I'd still like to get back to that at some point but we don't have it
yet. It should be entirely possible with the new provider
architecture -
and in fact we *did* add pluggable kex/kem support for libssl. But we
just ran out of time with pluggable signatures.

https://github.com/openssl/openssl/pull/11914

https://github.com/openssl/openssl/pull/13018



Matt

 > Maybe I'm missing something, but if you built a key management
system,
 > sign interface, ciphers that allows key virtualization, why not go
 > further ? I'm ready to implement the encryption interface, but why
 > OpenSSL still care about key type name. In the new era of version
3, it
 > can check if the key provides necessary interfaces.
 >
 > --
 > Alex Dankow
 >
 >

Re: SSL and "custom" EVP_KEY

[Tomas Mraz]

On Tue, 2021-11-02 at 11:42 +0700, Alex Dankow wrote:
> Matt, 
> 
> Thank you very much for your response. I understand that the FIPS
> certified OpenSSL module is long awaited and the team was quite
> limited in time to complete all features.
> I tried Windows certificates +Openssl because it implements the most
> common scenario: you can get a certificate to Openssl in DER format,
> but you can't get the private key. HSMs, KMIP servers or Windows
> don't let you export the private key.  It can be only virtualized
> (You sure know it, but I'm writing it also for our readers.)
> So, when I load X509 from storage it is handled by OpenSSL directly
> and gets type "RSA".When the private key is supplied, it has type of
> "MYTYPE" and actually represents a handle. If I skip the key matching

It should not have a type "MYTYPE" if it is really an RSA key. There
are some pecularities with how the openssl-3.0.0 works however 3.0.1
will contain a fix that should make this work correctly for
unexportable private keys of any common type. Your provider will be
invoked for operations with the private key.

> part, "openssl ca" works and a cert request can be signed with such a
> handle.
> As I understand, Openssl doesn't handle it completely yet. But it was
> planned so and maybe we will see it in the future.
> If ENGINE is now deprecated (is it?), what HSM vendors should do?
> --
> Alex Dankow
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Fri, Oct 29, 2021 at 10:11 PM Matt Caswell 
> wrote:
> > Hi Alex,
> > 
> > On 29/10/2021 14:32, Alex Dankow wrote:
> > > Hi OpenSSL team!
> > > 
> > > I wrote a provider for Windows certificates and implemented
> > "openssl ca".
> > > Now, I think it would be fun to see a HTTPS server using
> > certificates 
> > > installed in Windows storage.
> > Nice!
> > 
> > > 
> > > Certificate is loaded using load_cert_pass (taken from apps.c)
> > > with
> > > custom uri "wincert://11:22:33",  private key is loaded with 
> > > load_key from apps.c too. It works, but ...
> > > When I use  SSL_CTX_use_PrivateKey(ctx, myprivk)  the key is
> > declined. 
> > > OpenSSL compares strings and expects "rsaEncryption", and so on
> > instead 
> > > of "MYKEY". Why ?
> > 
> > It's not entirely clear to me what you are attempting here. Are
> > your 
> > certificates/keys stored in Windows storage standard RSA/ECDSA etc 
> > certs? Or are they using some custom algorithm?
> > 
> > If they are standard RSA/ECSDA certs then you should be using the 
> > correct standard algorithm names in you keymgmt etc and it should
> > all
> > "just work".
> > 
> > Unfortunately, in 3.0, libssl only supports standard algorithms. We
> > have 
> > discussed a "pluggable" signature scheme mechanism which would
> > enable
> > plugging in arbitrary algorithms but it didn't make it for 3.0:
> > 
> > https://github.com/openssl/openssl/issues/10512
> > 
> > I'd still like to get back to that at some point but we don't have
> > it
> > yet. It should be entirely possible with the new provider
> > architecture - 
> > and in fact we *did* add pluggable kex/kem support for libssl. But
> > we
> > just ran out of time with pluggable signatures.
> > 
> > https://github.com/openssl/openssl/pull/11914
> > https://github.com/openssl/openssl/pull/13018
> > 
> > 
> > Matt
> > 
> > > Maybe I'm missing something, but if you built a key management
> > system, 
> > > sign interface, ciphers that allows key virtualization, why not
> > > go 
> > > further ? I'm ready to implement the encryption interface, but
> > > why 
> > > OpenSSL still care about key type name. In the new era of version
> > 3, it 
> > > can check if the key provides necessary interfaces.
> > > 
> > > --
> > > Alex Dankow
> > > 
> > > 

-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]