Brett:
When I run the following code on UD v7.2.7, it works fine.
*
** Test program
EXECUTE \SET.WIDEZERO 0.01\
*
NUL = ''
CM = ','
A = -409071.8775: CM: 475000: CM: -652413: CM: 652413: CM: -475000: CM:
409071.8775
*
TEST.VMC = 1
TEST.TOTAL = 0
*
LOOP UNTIL FIELD(A, CM, TEST.VMC) = NUL DO
TEST.TOTAL += FIELD(A, CM, TEST.VMC)
TEST.VMC += 1
REPEAT
*
IF TEST.TOTAL = 0 THEN
PRINT 'OK'
END ELSE
PRINT TEST.TOTAL, (TEST.TOTAL = 0), TEST.TOTAL - 0,
(OCONV(TEST.TOTAL, 'MD0') = 0)
END
*2 Dev (0)-> RUN BP BILL
OK*
HTH,
Bill
------------------------------------------------------------------------
----- Original Message -----
*From:* [email protected]
*To:* [email protected] <[email protected]>
*Date:* 10/10/2011 1:16 AM
*Subject:* Re: [U2] The math just doesn't work.
I also thought: "great explanation" and nodded sagely when I read this.
However, consider this code:
NUL = ''
CM = ','
A = -409071.8775: CM: 475000: CM: -652413: CM: 652413: CM: -475000: CM:
409071.8775
*
*
TEST.VMC = 1
TEST.TOTAL = 0
*
LOOP UNTIL FIELD(A, CM, TEST.VMC) = NUL DO
TEST.TOTAL += FIELD(A, CM, TEST.VMC)
TEST.VMC += 1
REPEAT
*
IF TEST.TOTAL = 0 THEN
PRINT 'OK'
END ELSE
PRINT TEST.TOTAL, (TEST.TOTAL = 0), TEST.TOTAL - 0,
(OCONV(TEST.TOTAL, 'MD0') = 0)
END
Running this on our Universe system 11.1.1 produces:
0 0 0 1
So, in other words zero is the total result in TEST.TOTAL but this does not
equate to zero.
Now I know I can get round this by rounding the answer but am not sure why I
should have to. Any ideas?
Thanks
Brett
"Martin Phillips"<[email protected]<mailto:[email protected]>>
wrote in message
news:<2636297EDC484501AD1A5AC277EE76A7@lbs8<news:%3c2636297EDC484501AD1A5AC277EE76A7@lbs8>>...
Hi George,
As a general rule in programming, comparison of floating point values
for equality should be avoided. This is because, just as we cannot
write the number one third accurately in decimal notation, so the IEEE
floating point format used by computer systems cannot store numbers
accurately. The example that I use when teaching training courses is
14.2 which actually ends up as something close to 14.19999999997.
UniVerse gets around this with a wonderful concept called "wide zero"
that says, when testing for equality of floating point numbers, they
must be within some specified value of being equal rather than
strictly equal. The default wide zero error tolerance, set in IEEE
format with the WIDEZERO configuration parameter, is 2.91 * 10^-11 (2^
035) which is good for most business applications but occasionally needs
adjusting.
Unidata has a command, SET.WIDEZERO, to serve the same purpose but
defaults to 0.0 for backward comaptibility.
Martin Phillips
Ladybridge Systems Ltd
17b Coldstream Lane, Hardingstone, Northampton NN4 6DB, England
+44 (0)1604-709200
----- Original Message -----
From: "George Hammerle"<[email protected]<mailto:[email protected]>>
To:<[email protected]<mailto:[email protected]>>
Sent: Thursday, September 29, 2011 1:30 PM
Subject: [U2] The math just doesn't work.
Can anybody please help?
For some reason A + B does not equal C in the comparison below. Is
there any trick to get the comparisons to work properly?
Unidata 7.2 on Hp Unix 11+
Top of "TEST.COMP" in "RMH.MAIN", 13 lines, 263 characters.
*--: P
001: A = 3176.79
002: B = 106.19
003: C = 3282.98
004: D = 920.11
005: A = A + D
006: C = C + D
007: IF (A+B) # C THEN
008: CRT '(A+B) # C? YOU LIE'
009: CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C
010: END ELSE
011: CRT '(A+B) = C? YOU ROCK'
012: CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C
013: END
Bottom.
*--: FIBR
Filed "TEST.COMP" in file "RMH.MAIN" unchanged.
Compiling Unibasic: /db1/ud1/PGM/RMH.MAIN/TEST.COMP in mode 'u'.
compilation finished
(A+B) # C? YOU LIE
A = 4096.9, B = 106.19, (A+B) = 4203.09, C = 4203.09
George Hammerle
Programming Dude
Hubert Company LLC.
9555 Dry Fork Road
Harrison, Ohio 45030
513-367-8974
zhammerle@hubertREMOVE_THIS.com<mailto:zhammerle@hubertREMOVE_THIS.com>
_______________________________________________
U2-Users mailing list
[email protected]<mailto:[email protected]>
http://listserver.u2ug.org/mailman/listinfo/u2-users
This message contains information that may be privileged or confidential and is
the property of GPM Development Ltd. It is intended only for the person to whom
it is addressed. If you are not the intended recipient ,you are not authorized
to read, print, retain, copy, disseminate, distribute, or use this message or
any part thereof. If you receive this message in error, please notify the
sender immediately and delete all copies of this message.
This e-mail was sent to you by GPM Development Ltd. We are incorporated under
the laws of England and Wales (company no. 2292156 and VAT registration no. 523
5622 63). Our registered office is 6th Floor, AMP House, Croydon, Surrey CR0
2LX.
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
