Unfortunately the check breaks code which doesn't know nor need to know the keysize. The engine takes care of allocating buffers required.
Leaving it set to 0 has not broken anything yet. I supposed we could try to somehow set it to an arbitrary non-zero value to please the == 0 check. michael On 04/11/2017 03:47 PM, Dr. Stephen Henson wrote: > On Tue, Apr 11, 2017, Michael Reilly wrote: > >> Hi, >> >> commit 222333cf01e2fec4a20c107ac9e820694611a4db added a check that the size >> returned by EVP_PKEY_size(ctx->pkey) in M_check_autoarg() in >> crypto/evp/pmeth_fn.c is != 0. >> >> We are in the process of upgrading from 1.0.2j to 1.0.2k and discovered that >> the >> if (pksize == 0) check added in 1.0.2k breaks some of our applications. >> >> We use an engine for the RSA sign operation. The applications do not know >> anything about the keypair being used. The keypair is kept private by the >> engine so the application couldn't determine the attributes of the keypair >> if it >> wanted to do so. >> >> If this check is necessary is there a way to bypass it when the application >> does >> not have the keypair but the engine being used is holding the keypair? >> >> I know we can simply remove this line from our copy of the code but we like >> to >> avoid modifying the openssl distributed code if at all possible. >> > > Well the point of that code is so an application knows how large a buffer to > allocate for the signature. If it returns zero I can't see how applications > can do that. > > Note that you don't have to return the *precise* length of the signature just > an upper bound is sufficient. > > Steve. > -- > Dr Stephen N. Henson. OpenSSL project core developer. > Commercial tech support now available see: http://www.openssl.org > -- ---- ---- ---- Michael Reilly micha...@cisco.com Cisco Systems Arizona -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
