The t-sqrlo test invokes undefined behavior. The mpn_ptr scratch as well as the boundary values of the mpn_ptr pp are uninitialized values. In t-sqrlo.c their values are recorded (lines 92,93,100,101) before mpn_sqrlo and mpn_sqr are called. Their values are subsequently checked (lines 105,106) to see if the values at these locations in memory have changed.
I understand the programmers logic here- the code appears as if it simply checks that some uninitialized set of bits remains the same uninitialized set of bits after a function call. However, the C standard doesn't represent the value at that pointer as being a set of bits- it represents it as being something like a floating point NaN- something that ideally shouldn't be operated upon, a placeholder waiting for a real value. I came across this undefined behavior when compiling tests/mpn/t-sqrlo.c with clang and linking it to the otherwise correctly compiled gmp library. I am an LLVM/Clang developer, so I at first suspected that the issue was a compiler bug. Clang assumes that the memory at and around pp is initialized within the function calls; however, there is nothing to suggest that the memory at scratch is ever initialized. At -O2 optimization, clang optimizes the check for scratch[-1] != s_before into an undefined check, which is later treated as being always false, causing the test case to fail. Although this may seem like an issue with clang, it is within the right of the compiler to do this since controlling the flow of the program based on uninitialized memory is undefined behavior. I was able to get the tests to pass by simply memset()ting scratch to any value, and it should be trivially fixable by doing basically this but the proper way one might do it if they were a GMP developer. I would also suggest doing the same with pp[-1] and pp[n] in this case, with care that whatever values those are set to are unlikely to be values that a buggy mpn_sqrlo() call would set those to. GMP version 6.1.2 Investigated on RHEL on PowerPC, but recreated on FreeBSD on x86-64. uname -a output: Linux -----.----.---.--- 4.14.0-115.el7a.ppc64le #1 SMP Tue Sep 25 12:28:39 EDT 2018 ppc64le ppc64le ppc64le GNU/Linux config.guess: power9-unknown-linux-gnu configfsf.guess: powerpc64le-unkown-linux-gnu The bug was triggered with an experimental version of LLVM/Clang _______________________________________________ gmp-bugs mailing list [email protected] https://gmplib.org/mailman/listinfo/gmp-bugs
