the following error was found on a i686-linux (SuSE 7.2) system with P4 CPU and 2GB of RAM.
When calling the MD5 function on very large data sets (around 2GB) in memory or from a memory map, the computed MD5 sum is false and even worth, can cause the program to crash with a seg-fault. By tracking down the calculation of the sum, I found out that this behaviour occurs when the data pointer crosses the address 0X80000000L in the MD5_Update() function, thus wrapping around from a positive to negative integer. The reason for this error lies in the comparison of two signed numbers rather than two unsigned numbers in the assembly code found in crypto/md5/asm/md5-586.pl at line 296. Instead of a "jge" (greater equal) instruction, there should be a "jae" (above equal) instruction for evaluating an unsigned compare.
An even better fix, IMHO, would be to omit the 64 bytes subtraction from the target address register right at the begining of the function call and then just compare the pointers for equality (jne).
In order to compile, the "jae" instruction must also be added somewhere in the crypto/perlasm/x86unix.pl and x86ms.pl perl scripts.
Cheers, Eric
______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
