type of request : enhancement version of OpenSSL affected : since 0.9.8a Hello,
When calling BN_GF2m_mod_inv with an "irreductible polynome" parameter "p"
that is not actually irreductible, it seems to cause an infinite loop.
(gdb) bt
#0 BN_GF2m_mod_inv (r=0x1e5100, a=0x1e5128, p=0x1e5114, ctx=0x1e52d8) at
bn_gf2m.c:605
(gdb) p *a
$237 = {d = 0x1e50a0, top = 6, dmax = 6, neg = 0, flags = 0}
(gdb) p /x *a->d@a->top
$238 = {0xc148f877, 0x14576c9f, 0x47c9316b, 0x7ef5a452, 0x4a2053f3, 0x40}
(gdb) p *p
$239 = {d = 0x1e50c0, top = 6, dmax = 6, neg = 0, flags = 0}
(gdb) p /x *p->d@p->top
$240 = {0x800001, 0x0, 0x0, 0x0, 0x0, 0x80}
(gdb) show endian
The target is assumed to be big endian
(gdb) c
Continuing.
Program received signal SIGSTOP, Stopped (signal).
BN_GF2m_mod_inv (r=0x1e5100, a=0x1e5128, p=0x1e5114, ctx=0x1e52d8) at
bn_gf2m.c:309
309 bn_correct_top(r);
(gdb) p *u
$245 = {d = 0x1e5040, top = 0, dmax = 6, neg = 0, flags = 0}
(gdb) p /x *u->d@u->top
Invalid number 0 of repetitions.
I suggest a test on u :
while (1)
{
while (!BN_is_odd(u))
{
+ if (BN_is_zero(u)) goto err;
if (!BN_rshift1(u, u)) goto err;
BN_GF2m_mod_inv appeared in version openssl-0.9.8a.tar.gz. It did not
seems to have changed since.
Best regards
Emmanuel
type of request : enhancement
version of OpenSSL affected : since 0.9.8a
Hello,
When calling BN_GF2m_mod_inv with an "irreductible polynome" parameter "p" that is not actually irreductible, it seems to cause an infinite loop.
(gdb) bt
#0 BN_GF2m_mod_inv (r=0x1e5100, a=0x1e5128, p=0x1e5114, ctx=0x1e52d8) at bn_gf2m.c:605
(gdb) p *a
$237 = {d = 0x1e50a0, top = 6, dmax = 6, neg = 0, flags = 0}
(gdb) p /x *a->d@a->top
$238 = {0xc148f877, 0x14576c9f, 0x47c9316b, 0x7ef5a452, 0x4a2053f3, 0x40}
(gdb) p *p
$239 = {d = 0x1e50c0, top = 6, dmax = 6, neg = 0, flags = 0}
(gdb) p /x *p->d@p->top
$240 = {0x800001, 0x0, 0x0, 0x0, 0x0, 0x80}
(gdb) show endian
The target is assumed to be big endian
(gdb) c
Continuing.
Program received signal SIGSTOP, Stopped (signal).
BN_GF2m_mod_inv (r=0x1e5100, a=0x1e5128, p=0x1e5114, ctx=0x1e52d8) at bn_gf2m.c:309
309 bn_correct_top(r);
(gdb) p *u
$245 = {d = 0x1e5040, top = 0, dmax = 6, neg = 0, flags = 0}
(gdb) p /x *u->d@u->top
Invalid number 0 of repetitions.
I suggest a test on u :
while (1)
{
while (!BN_is_odd(u))
{
+ if (BN_is_zero(u)) goto err;
if (!BN_rshift1(u, u)) goto err;
BN_GF2m_mod_inv appeared in version openssl-0.9.8a.tar.gz. It did not seems to have changed since.
Best regards
Emmanuel
