If I recall correctly the intel chip has an "extended precision" format that uses 80 bits when doing calculation. These are used during calculation regardless of the original floating point format. So if the values in the calculation are single precision (one's compliment 24 bit mantissa with an 8 bit exponent) the full 80 bits are used as 'guard bits' to increase the precision of the calculation. Of course the result is then truncated to single precision. In High Level Assembly (Randall Hyde's interesting fusion of assembly and high level language constructs) you can declare an 80 bit value.
~joe -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Peter K. Stys Sent: Friday, July 28, 2006 11:36 AM To: REALbasic NUG Subject: Re: Re: Math Error? Actually we're both right and wrong: In[55]:=RealDigits[0.036,2,100] Out[55]={{1,0,0,1,0,0,1,1,0,1,1,1,0,1,0,0,1,0,1,1,1,1,0,0,0,1,1,0,1,0,1,0,0, 1,1,1,1, 1,1,0,1,1,1,1,1,0,0,1,1,1,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},-4} Shows that 0.036 can be represented exactly in binary, but requires 51 significant digits (if I counted correctly to the last 1). So with a 64 bit double, it probably just missed the # of digits in the mantissa to represent 0.036 exactly. Nuf time wasted on this, just a curiosity. P. -- ---------------------------------------------------------------------------- --- Peter K. Stys, MD Professor of Medicine(Neurology), Senior Scientist Ottawa Health Research Institute, Div. of Neuroscience Ottawa Hospital / University of Ottawa Ontario, CANADA tel: (613)761-5444 fax: (613)761-5330 http://www.ohri.ca/profiles/stys.asp realsoftware.com/listarchives/lists.html> _______________________________________________ Unsubscribe or switch delivery mode: <http://www.realsoftware.com/support/listmanager/> Search the archives of this list here: <http://support.realsoftware.com/listarchives/lists.html>
