rsa_get_n() only rejects a zero-length INTEGER, so an all-zero
modulus (e.g. a single 0x00 byte) is accepted and leaves num_bits
at 0 after the leading-zero-stripping loop. With num_bits == 0, n
is allocated as a single header word, so br_i32_ninv32(n[1]) reads
one word past the end of n, and modulus is allocated via malloc(0).
Reject num_bits == 0 with -EINVAL right after stripping the leading
zeroes, before any of the buffers sized from it are allocated.
Fixes: e0d310b098b1 ("lib: rsa: generate additional parameters for public key")
Suggested-by: Simon Glass <[email protected]>
Signed-off-by: Prasad Kummari <[email protected]>
---
lib/rsa/rsa-keyprop.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/rsa/rsa-keyprop.c b/lib/rsa/rsa-keyprop.c
index ca186a1440c..e74eea18977 100644
--- a/lib/rsa/rsa-keyprop.c
+++ b/lib/rsa/rsa-keyprop.c
@@ -681,6 +681,10 @@ int rsa_gen_key_prop(const void *key, uint32_t keylen,
struct key_prop **prop)
for (i = 0; i < rsa_key.n_sz && !rsa_key.n[i]; i++)
;
(*prop)->num_bits = (rsa_key.n_sz - i) * 8;
+ if (!(*prop)->num_bits) {
+ ret = -EINVAL;
+ goto out;
+ }
(*prop)->modulus = malloc(rsa_key.n_sz - i);
if (!(*prop)->modulus) {
ret = -ENOMEM;
--
2.43.0
