That makes me ask the question Why would I use REAL for anything?? Thank you
On 23 Jun 2004 at 11:33, Alan Wolfe wrote: > Hey Victor. > > Double has 2 times the amount of accuracy as Real does because its twice the > size in memory. > > This is because analog floating point numbers are being stored in a binary > computer. > > Check out this oversimplified example :P > > if you only had 2 bits to store the number to the right of a decimal point, > thats only 4 combinations so youd have: > 0 > 0.25 > 0.5 > 0.75 > > so if you made a variable of that type and tried to set it to 0.14, it would > have to set it to 0.25 since that was the closest match that it can store. > > if you had 3 bits, it doubles the number of combinations to 8 so you would > have: > 0 > 0.125 > 0.25 > 0.375 > 0.5 > 0.625 > 0.75 > 0.875 > > so now if you had a variable of that type and you tried to store 0.14, it > would really store 0.125 since that was the closest match you can store. > > if you had 4 bits, the number of combinations go up to 16, with 5 bits it > becomes 32 combinations and so on. > > the more bits you have, the more accurate the number you can store is, but > it can never be infinitely accurate. > > Thats why double gives a more precise calculation while real gives an > incorect one, cause double has twice the number of bits and squared the > amount of accuracy as real does. > > > ----- Original Message ----- > From: "Victor Timmons" <[EMAIL PROTECTED]> > To: "RBASE-L Mailing List" <[EMAIL PROTECTED]> > Sent: Wednesday, June 23, 2004 11:08 AM > Subject: [RBASE-L] - Rounding REAL vs DOUBLE > > > > I am doing a very simple formula that does some rounding. > > > > R>SET VAR vUpdateAmount REAL = .01 > > R>SET VAR vBookListPrice CURRENCY = 3.50 > > R>SET VAR vBookListPrice2 DOUBLE = NULL > > R>SET VAR vBookListPrice2 = .vBookListPrice > > R>SET VAR vBookListPrice2 = ((.vBookListPrice2 * > > .vUpdateAmount)+.vBookListPrice2) > > R>SHO VAR vBookListPrice2 > > 3.5399999921769 > > R>SET VAR vBookListPrice2 = (ROUND(.vBookListPrice2,2)) > > R>SHO VAR vBookListPrice2 > > 3.53 > > > > If I change vUpdateAmount to a DOUBLE I get the correct results 3.54 > > Why is that?? Isnt .01 = .01 wheather is a REAL or a DOUBLE?? Could > > someone run this a let me know if they get the same results? > > > > R>SET VAR vUpdateAmount DOUBLE = .01 > > R>SET VAR vBookListPrice CURRENCY = 3.50 > > R>SET VAR vBookListPrice2 DOUBLE = NULL > > R>SET VAR vBookListPrice2 = .vBookListPrice > > R>SET VAR vBookListPrice2 = ((.vBookListPrice2 * > > .vUpdateAmount)+.vBookListPrice2) > > R>SHO VAR vBookListPrice2 > > 3.535 > > R>SET VAR vBookListPrice2 = (ROUND(.vBookListPrice2,2)) > > R>SHO VAR vBookListPrice2 > > 3.54 > > > > > > > > > > Victor Timmons > > Tiz's Door Sales, Inc > > 425-258-2391 > > > > Victor Timmons Tiz's Door Sales, Inc 425-258-2391
