Hi Lennox.
Your observation may be correct - but in floating point, you need to
know that 2.00000... is the same thing as 1.99999... And this applies
to all numbers. The fraction is not precise, particularly when
passing through a decimal to binary or binary to decimal conversion.
If you want to study up on it, there is a whole area of interest
called numerical analysis.
Russ
On Nov 23, 2006, at 10:09 PM, Lennox Jacob wrote:
Thanks Russ, I will try that but I should point out one observation:-
With the current code, if the sum is exactly 100,000.00 or less,
all calculations are correct but once I go above 100,000.00 then
that problem occurs.
Thanks again Russ.
Lennox.
Russ Jones <[EMAIL PROTECTED]> wrote:
On Nov 23, 2006, at 9:01 PM, Lennox Jacob wrote:
The problem is that when I use large numbers like 60000.23 +
25000.23 + 25000.23 I get 110000.70 instead of 110000.69
Hi Lennox.
You have come up against a "problem" that has existed in computer
science for at least the last 45 years - since that is how long I
have been doing this stuff - and it probably existed before that.
The problem is that you are using "Floating point" (or "double")
variables to store your numbers, and expect to get fixed-point
results. Floating point numbers are represented by a fraction,
multiplied by an exponent. Works pretty well if all is in decimal,
but computers seem to prefer binary numbers - and there is not a 1-
to-1 correspondence between decimal fractions and binary fractions.
If you really need the "exact" result, as is often the case in
financial calculations, you will need to avoid floating point.
You can do all your arithmetic in long integers, if you first
multiply all incoming numbers by 100, an at the output end, divide
the results by 100.
HTH
Russ
Russ Jones
[EMAIL PROTECTED]
RB 2006r4 MacOS 10.4.8
_______________________________________________
Unsubscribe or switch delivery mode:
Search the archives of this list here:
---------------------------------
Everyone is raving about the all-new Yahoo! Mail beta.
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>
Search the archives of this list here:
<http://support.realsoftware.com/listarchives/lists.html>
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>
Search the archives of this list here:
<http://support.realsoftware.com/listarchives/lists.html>
