Steve, all, here's another idea in draft stage. Please give me feedback before I start working out the details.
In obj_xref.txt, we define rsassaPss undef rsaEncryption We add two components for pss to rsa_st struct rsa_st { [...] const EVP_MD *pssDigest; /* pointer ok or should we store the NID? */ int pssSaltlen; }; and define a function to decode the parameters of the algorithm used in the X.509 certificate static int rsa_param_decode(EVP_PKEY *pkey, const unsigned char **pder, int derlen) { /* TODO: parse and check paramters */ if (pkey && pkey->type == EVP_PKEY_RSA) { pkey->pkey.rsa->pssDigest = EVP_sha1(); pkey->pkey.rsa->pssSaltlen = 20; } return 1; } In ASN1_item_verify(), we use the EVP_DigestVerify...() API and call the pkey's param_decode() method ... if (pkey->ameth->param_decode) { /* TODO: pass real algorithm parameters */ pkey->ameth->param_decode(pkey, NULL, 0); } else printf("no param_decode method defined\n"); if (!EVP_DigestVerifyInit(&mctx,&pctx,md, NULL, pkey)) ... md may be NULL if message digest is undef is obj_xref.txt (some low-level routines must be changed to get this result) In do_sigver_init(), we check for type==NULL and set the pss parameters in this case ... } else if (EVP_PKEY_sign_init(ctx->pctx) <= 0) return 0; } if (type == NULL) { if (pkey && pkey->type == EVP_PKEY_RSA && pkey->pkey.rsa->pssDigest) { type = pkey->pkey.rsa->pssDigest; EVP_PKEY_CTX_set_rsa_padding(ctx->pctx, RSA_PKCS1_PSS_PADDING); EVP_PKEY_CTX_set_rsa_pss_saltlen(ctx->pctx, pkey->pkey.rsa->pssSaltlen); } else { int def_nid; if (EVP_PKEY_get_default_digest_nid(pkey, &def_nid) > 0) type = EVP_get_digestbynid(def_nid); } } if (type == NULL) { EVPerr(EVP_F_DO_SIGVER_INIT, EVP_R_NO_DEFAULT_DIGEST); ... Does this approch look ok? Best regards, Martin ______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org