On Sun, Feb 03, 2008 at 01:22:02PM -0800, Paul Charlton wrote: > Perhaps GW should comment on the accuracy of Prime95 convolution. Last > time that I looked it was using transforms 2^21 and 2^22 in length for > exponents up to 33,554,432; on a 16-bit representation stored in a > [sign]+23-bit mantissa single-precision float. For all intents and > purposes with 16-bit numerical precision, FP32_cos(PI/2^n) == ONE, for > n>=10.
I might be missing something, but I was pretty sure Prime95 used double-precision floats all the way? /* Steinar */ -- Homepage: http://www.sesse.net/ _______________________________________________ Prime mailing list [email protected] http://hogranch.com/mailman/listinfo/prime
