Thanks CV, Norman and Russ, the updated project is here http://download.yousendit.com/027DCFE46629C029
works as expected now. Thanks again. Lennox. CV <[EMAIL PROTECTED]> wrote: On Nov 23, 2006, at 5:47 PM, Lennox Jacob wrote: > Hello, > > I have an app in which I perform some calculations. > > The problem is that when I use large numbers like 60000.23 + > 25000.23 + 25000.23 I get 110000.70 instead of 110000.69 > Actually, Lennox, the result of that addition in Rb is 110000.69000000, which is accurate to 15 or 16 significant figures as one would expect per the IEEE floating point standard for a double. You are viewing a rounded result, rather than the full precision result stored in memory. Here is how you can view the result of that calculation correctly to the precision of a double: dim r as string = Format(60000.23 + 25000.23 + 25000.23, "#.###########") MsgBox r // shows 110000.69 You can see a decimal representation of the numerical result stored in memory by adding a few more #'s past the decimal point above, like this: dim r as string = Format(60000.23 + 25000.23 + 25000.23, ""#.################"") MsgBox r // shows 110000.6900000000023283 This result, 110000.6900000000023283, is the floating point number which Rb is using to approximate the result of the calculation. As you can see, it is exact to 17 significant digits in this case. In other words, the 'problem' you are seeing is a result of the manner in which the result is displayed. It doesn't reflect the precision of the result as calculated and represented internally by Rb. Best, Jack _______________________________________________ Unsubscribe or switch delivery mode: Search the archives of this list here: --------------------------------- Cheap Talk? Check out Yahoo! Messenger's low PC-to-Phone call rates. _______________________________________________ Unsubscribe or switch delivery mode: <http://www.realsoftware.com/support/listmanager/> Search the archives of this list here: <http://support.realsoftware.com/listarchives/lists.html>
