Original conundrum...
>Type this in the message box...
>put 5.45 - 4 = 1.45 FALSE
So we have a sever dose of the floating point accuracies, huh John?
>On the surface, it appears to me that this is an interesting example of how
>roundoff errors in computer calculations can cause programming headaches if
>your logic is testing for absolute equality.
Claude agrees:
>As a workaround you can use round(myNumber,nDigits)
>
>put round((5.45-4),10)-round((1.45),10)=0 TRUE
John's observation below hints at a curious solution, however...
>The moral of the story is to never build logic into numerical programs
>that relies on recognizing precise equality of two floating point numbers.
put VALUE(5.45 - 4) = 1.45 TRUE
Presumably, 'value' applies a different set of rules.
At least I have a solution for my accounts validation check, but I do feel
a computer should manage basic subtraction. There was a discussion about
this a while ago, essentially explaining how computers behave this way.
No-one to the best of my recollection actually explained why they have to,
however.
The moral of this story is to remember your values.
/H
Hugh Senior
The Flexible Learning Company
Consultant Programming & Software Solutions
Fax/Voice: +44 (0)1483.27 87 27
Email: [EMAIL PROTECTED]
Web: www.flexibleLearning.com
Archives: http://www.mail-archive.com/[email protected]/
Info: http://www.xworlds.com/metacard/mailinglist.htm
Please send bug reports to <[EMAIL PROTECTED]>, not this list.
