I think the above patch is good, but incomplete.

(As a niggle, it uses jl, which I think is correct because the
argument is signed, but the rest of the file is using jb. The best
answer would be to fix the file to use jl before applying it, but I've
used jb for consistency below.)

Once the crash is fixed, the code still accesses out of bounds of the
modulus when the modulus is a single limb because modulus[1] is
accessed unconditionally. Depending on the phase of the moon, that
might crash the address space.

I've fixed that in the attached and have included a test case. (The
patch to bn_test.c will need to be applied manually but that's
simple.)

However, even with that fix there's still and out-of-bounds access of
the precomputed table in bn_mul_mont_gather5. This leads to the wrong
result being calculated. This only affects cases where the modulus is
>= 312 bits because that leads to a window size of 5 and the mont5
code being used. Sadly, that code is too odd for me to figure out what
it's doing so I don't have a patch for that.


Cheers

AGL

Attachment: patch
Description: Binary data

Reply via email to