I'm trying to perform a fairly simple operation. I have a 20 byte hash. I want to PKCS1_OAEP pad it to 256 bytes so I can RSA encrypt it. Later, after RSA decrypting, I want to remove the pad to get the 20 bytes back.
However, the implementation of RSA_padding_check_PKCS1_OAEP() almost immediately tests that the 5th parameter (my 256 byte RSA modulus) is greater than the 4th parameter (my 256 byte decrypted but still padded data). At the suggestion of the list, I've looked at the pkcs1 specification. No help. At another suggestion, I changed the 5th parameter to 257, but still get an error. If I patch the openSSL code to remove the test, this code works fine, and I get my hash back. Anyone want to suggest where my code is wrong? Or point me to sample code? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #include <stdio.h> #include <stdlib.h> #include <string.h> #include <openssl/rsa.h> #include <openssl/err.h> int main(int argc, char **argv) { int rc = 0; unsigned char data_in[256]; unsigned char data_pad[256]; unsigned char data_out[256]; unsigned long error; const char *file; int line; const char *data; int flags; /* data to pad */ memset(data_in, 0x55, 256); rc = RSA_padding_add_PKCS1_OAEP(data_pad, /* unsigned char *to */ 256, /* int tlen */ data_in, /* unsigned char *f */ 20, /* int fl */ NULL, /* unsigned char *p */ 0); /* int pl */ printf("RSA_padding_add_PKCS1_OAEP rc %d\n", rc); if (rc == 1) { /* 1 is success, 0 is error */ rc = RSA_padding_check_PKCS1_OAEP(data_out, /* unsigned char *to */ 256, /* int tlen */ data_pad, /* unsigned char *f */ 256, /* int fl */ 256, /* int rsa_len */ NULL, /* unsigned char *p */ 0); /* int pl */ /* -1 is error */ printf("RSA_padding_check_PKCS1_OAEP rc %d\n", rc); error = ERR_get_error_line_data(&file, &line, &data, &flags); printf("error %08lx file %s line %d data %s flags %08x\n", error, file, line, data, flags); } return EXIT_SUCCESS; } -- Ken Goldman [EMAIL PROTECTED] 914-784-7646 ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]