[LibReSSL] Allow key generation to use arbitrary public exponents

[Benny Baumann]

Hi,

I'd like to propose to include the following additional two command line
arguments for the openssl binary when creating RSA keys. While the patch
is written to apply to LibReSSL 2.0.5 it should apply to genrsa.c of
OpenSSL 1.0.1 just fine too.

While the default of 65537 is a sane default it's not strictly forced by
any standard. In contrast when looking at NIST SP-800-56B section 6.2.1
bullet 2b it is described as "an odd positive integer such that 65537 <=
e < 2**256"

As the plain RSA only requires e to be co-prime to both p-1 and q-1 and
given the obvious limitation for e=1 yielding no security, there is no
mathematical backing for any upper bound for e (except the obvious one
given by p*q-1).

The change only affects the key generation and extends the possibility
to use custom public exponents as has been done in certain areas
previously. Implementations conforming to the mathematical foundation
should be unaffected as otherwise they would have been broken for
decryption all along.

Kind regards,
Benny Baumann

diff -r -U3 '--exclude=*.log' '--exclude=*.o' libressl-2.0.5/apps/genrsa.c libressl-2.0.5-benbe1/apps/genrsa.c
--- libressl-2.0.5/apps/genrsa.c	2014-08-06 05:27:41.000000000 +0200
+++ libressl-2.0.5-benbe1/apps/genrsa.c	2014-08-09 01:18:37.906651418 +0200
@@ -101,6 +101,7 @@
 	const EVP_CIPHER *enc = NULL;
 	unsigned long f4 = RSA_F4;
 	char *outfile = NULL;
+	char *pubval = NULL;
 	char *passargout = NULL, *passout = NULL;
 #ifndef OPENSSL_NO_ENGINE
 	char *engine = NULL;
@@ -127,6 +128,13 @@
 			if (--argc < 1)
 				goto bad;
 			outfile = *(++argv);
+		} else if (strcmp(*argv, "-pub") == 0) {
+			if (--argc < 1)
+				goto bad;
+			pubval = *(++argv);
+			f4 = 1;
+		} else if (strcmp(*argv, "-pubrand") == 0) {
+			f4 = 0;
 		} else if (strcmp(*argv, "-3") == 0)
 			f4 = 3;
 		else if (strcmp(*argv, "-F4") == 0 || strcmp(*argv, "-f4") == 0)
@@ -193,6 +201,8 @@
 		BIO_printf(bio_err, " -passout arg    output file pass phrase source\n");
 		BIO_printf(bio_err, " -f4             use F4 (0x10001) for the E value\n");
 		BIO_printf(bio_err, " -3              use 3 for the E value\n");
+		BIO_printf(bio_err, " -pub hex        use the given hex number for the E value\n");
+		BIO_printf(bio_err, " -pubrand        use a randomly generated number half the size of N for the E value\n");
 #ifndef OPENSSL_NO_ENGINE
 		BIO_printf(bio_err, " -engine e       use engine e, possibly a hardware device.\n");
 #endif
@@ -227,7 +237,28 @@
 	if (!rsa)
 		goto err;
 
-	if (!BN_set_word(bn, f4) || !RSA_generate_key_ex(rsa, num, bn, &cb))
+	switch( f4 ) {
+		case 0:
+			if(!BN_rand(bn, num >> 1, 0, 1))
+				goto err;
+			break;
+
+		case 1:
+{
+			BIGNUM *tmp;
+			if(!BN_hex2bn(&tmp, pubval))
+				goto err;
+			if(!BN_copy(bn, tmp))
+				goto err;
+			break;
+}
+		default:
+			if (!BN_set_word(bn, f4))
+				goto err;
+			break;
+	}
+
+	if (!RSA_generate_key_ex(rsa, num, bn, &cb))
 		goto err;
 
 	/*

signature.asc
Description: OpenPGP digital signature