And just to add to Martin's excellent answer - the normal workround for any
computer system is to use integral values, scaling and descaling as
required.
In case you think this is a U2 specific limitation, here's the same in C# as
a console app -
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
float A = 3176.79f;
float B = 106.19f;
float C = 3282.98f;
float D = 920.11f;
A = A + D;
C = C + D ;
if((A+B) != C ){
Console.WriteLine("You lie, A+B = {0}, C= {1}",(A+B), C);
} else{
Console.WriteLine("you rock");
}
pressAnyKey();
}
static void pressAnyKey()
{
Console.Write("Press any key ...");
Console.ReadLine();
}
}
}
Result -
You lie, A+B = 4203.09, C= 4203.09
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Martin Phillips
Sent: 29 September 2011 13:46
To: U2 Users List
Subject: Re: [U2] The math just doesn't work.
Hi George,
As a general rule in programming, comparison of floating point values for
equality should be avoided. This is because, just as we cannot write the
number one third accurately in decimal notation, so the IEEE floating point
format used by computer systems cannot store numbers accurately. The example
that I use when teaching training courses is 14.2 which actually ends up as
something close to 14.19999999997.
UniVerse gets around this with a wonderful concept called "wide zero" that
says, when testing for equality of floating point numbers, they must be
within some specified value of being equal rather than strictly equal. The
default wide zero error tolerance, set in IEEE format with the WIDEZERO
configuration parameter, is 2.91 * 10^-11 (2^ 035) which is good for most
business applications but occasionally needs adjusting.
Unidata has a command, SET.WIDEZERO, to serve the same purpose but defaults
to 0.0 for backward comaptibility.
Martin Phillips
Ladybridge Systems Ltd
17b Coldstream Lane, Hardingstone, Northampton NN4 6DB, England
+44 (0)1604-709200
----- Original Message -----
From: "George Hammerle" <[email protected]>
To: <[email protected]>
Sent: Thursday, September 29, 2011 1:30 PM
Subject: [U2] The math just doesn't work.
> Can anybody please help?
>
> For some reason A + B does not equal C in the comparison below. Is there
> any trick to get the comparisons to work properly?
>
> Unidata 7.2 on Hp Unix 11+
>
>
> Top of "TEST.COMP" in "RMH.MAIN", 13 lines, 263 characters.
> *--: P
> 001: A = 3176.79
> 002: B = 106.19
> 003: C = 3282.98
> 004: D = 920.11
> 005: A = A + D
> 006: C = C + D
> 007: IF (A+B) # C THEN
> 008: CRT '(A+B) # C? YOU LIE'
> 009: CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C
> 010: END ELSE
> 011: CRT '(A+B) = C? YOU ROCK'
> 012: CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C
> 013: END
> Bottom.
> *--: FIBR
> Filed "TEST.COMP" in file "RMH.MAIN" unchanged.
>
> Compiling Unibasic: /db1/ud1/PGM/RMH.MAIN/TEST.COMP in mode 'u'.
> compilation finished
>
> (A+B) # C? YOU LIE
> A = 4096.9, B = 106.19, (A+B) = 4203.09, C = 4203.09
>
>
>
>
> George Hammerle
> Programming Dude
> Hubert Company LLC.
> 9555 Dry Fork Road
> Harrison, Ohio 45030
> 513-367-8974
> zhammerle@hubertREMOVE_THIS.com
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
