The following code will make BN_GF2m_mod_arr() into infinite loop. int main(int argc, char *argv[]) { BIGNUM *bn = NULL, *res = NULL, *p = NULL;
BN_hex2bn(&bn3, "448692853686179295b477565726f6e5d"); BN_hex2bn(&p, "100000000000000000000000000000087"); res = BN_new(); BN_GF2m_mod(res, bn3, p); } Because in final round of reduction d0 == 0 and z[dN] != 0, which makes z[dN] can not be changed for ever. This is fixed by set z[dn] = 0 if d0 == 0. This patch is based on openssl SNAPSHOT 20080519, and has been tested on x86_64 with openssl/test/bntest.c and above program. Signed-off-by: Huang Ying <[EMAIL PROTECTED]> --- crypto/bn/bn_gf2m.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) --- a/crypto/bn/bn_gf2m.c +++ b/crypto/bn/bn_gf2m.c @@ -322,7 +322,11 @@ int BN_GF2m_mod_arr(BIGNUM *r, const BIG if (zz == 0) break; d1 = BN_BITS2 - d0; - if (d0) z[dN] = (z[dN] << d1) >> d1; /* clear up the top d1 bits */ + /* clear up the top d1 bits */ + if (d0) + z[dN] = (z[dN] << d1) >> d1; + else + z[dN] = 0; z[0] ^= zz; /* reduction t^0 component */ for (k = 1; p[k] != 0; k++)
The following code will make BN_GF2m_mod_arr() into infinite loop. int main(int argc, char *argv[]) { BIGNUM *bn = NULL, *res = NULL, *p = NULL; BN_hex2bn(&bn3, "448692853686179295b477565726f6e5d"); BN_hex2bn(&p, "100000000000000000000000000000087"); res = BN_new(); BN_GF2m_mod(res, bn3, p); } Because in final round of reduction d0 == 0 and z[dN] != 0, which makes z[dN] can not be changed for ever. This is fixed by set z[dn] = 0 if d0 == 0. This patch is based on openssl SNAPSHOT 20080519, and has been tested on x86_64 with openssl/test/bntest.c and above program. Signed-off-by: Huang Ying <[EMAIL PROTECTED]> --- crypto/bn/bn_gf2m.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) --- a/crypto/bn/bn_gf2m.c +++ b/crypto/bn/bn_gf2m.c @@ -322,7 +322,11 @@ int BN_GF2m_mod_arr(BIGNUM *r, const BIG if (zz == 0) break; d1 = BN_BITS2 - d0; - if (d0) z[dN] = (z[dN] << d1) >> d1; /* clear up the top d1 bits */ + /* clear up the top d1 bits */ + if (d0) + z[dN] = (z[dN] << d1) >> d1; + else + z[dN] = 0; z[0] ^= zz; /* reduction t^0 component */ for (k = 1; p[k] != 0; k++)
OpenSSL self-test report: OpenSSL version: 0.9.9-dev Last change: To support arbitrarily-typed thread IDs, deprecate the ... Options: no-gmp no-krb5 no-mdc2 no-rc5 no-rfc3779 no-shared no-zlib no-zlib-dynamic static-engine OS (uname): Linux caritas-dev 2.6.24-1-amd64 #1 SMP Fri Apr 18 23:08:22 UTC 2008 x86_64 GNU/Linux OS (config): x86_64-whatever-linux2 Target (default): linux-x86_64 Target: linux-x86_64 Compiler: Using built-in specs. Target: x86_64-linux-gnu Configured with: ../src/configure -v --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.2 --program-suffix=-4.2 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --disable-libmudflap --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 4.2.4 (Debian 4.2.4-1) Test passed.