[levitte - Fri Nov 29 17:31:16 2002]: > Another question: have you tried defining the logical name > OPENSSL_NO_ASM with the value YES before building? If you build it > that way, it should work. I'm assuming the problem lies in > [.CRYPTO.BN.ASM]VMS.MAR, and if you can verify that it runs all > tests correctly when OPENSSL_NO_ASM if defined, it will definitely > narrow down the space I need to look into.
Confirmed, it's the assembler version of bn_div_words that doesn't work for some values, basically when the quotient gets the high bit set. The reason is that EDIV treats that as an integer overflow, since the beginning of the routine makes sure all arguments to EDIV are positive (i.e. don't have the high bit set). The result in such cases is that the quotient really is the remainder, and we get whacked result. An example: bn_div_words(57DEDEDE,DEC0003F,80000000) -> 6F60001F because EDIV is be handed this (the numbers are different because everything is shifted down one step to make sure all numbers are positive): 2BEF6F6F6F60001F as dividend 40000000 as divisor Making the operation 2BEF6F6F6F60001F / 40000000 result in the quotient AFBDBDBD, which is signed, and therefore makes EDIV think it's an error since that's a negative number... I'm starting to believe that the divisor doesn't really need to get shifted at all, which would simplify the fiddling, and deal with the kind of situation that gets us the current bug. I'll do some research in the next few days. If I don't get it in time for 0.9.7-beta5, I'll simply revert to the routines that worked (the 0.9.6 ones). -- Richard Levitte ______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
