Hi, I've been using numpy's float96 class lately, and I've run into some strange precision errors. See example below:
>>> import numpy >>> numpy.version.version '1.5.0' >>> sys.version '3.1.2 (release31-maint, Jul 8 2010, 01:16:48) \n[GCC 4.4.4]' >>> x = numpy.array( [0.01] , numpy.float32 ) >>> y = numpy.array( [0.0001] , numpy.float32 ) >>> x[0]*x[0] - y[0] 0.0 >>> x = numpy.array( [0.01] , numpy.float64 ) >>> y = numpy.array( [0.0001] , numpy.float64 ) >>> x[0]*x[0] - y[0] 0.0 >>> x = numpy.array( [0.01] , numpy.float96 ) >>> y = numpy.array( [0.0001] , numpy.float96 ) >>> x[0]*x[0] - y[0] -6.286572655403010329e-22 I would expect the float96 calculation to also produce 0.0 exactly as found in the float32 and float64 examples. Why isn't this the case? Slightly off-topic: why was the float128 class dropped? Thanks in advance for any thoughts/feedback, Mike _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
