# Re: [U2] The math just doesn't work.

```Hi Brett,

I think my discussion on this topic still holds. Run your program with
PRECISION 14
```
at the top and you can see the rounding errors creeping in. This is exactly why comparing floating point values is generally considered a bad idea.
```

Martin Phillips
17b Coldstream Lane, Hardingstone, Northampton NN4 6DB, England
+44 (0)1604-709200

```
----- Original Message ----- From: "Brett Callacher" <bre...@gpmdev.co.uk>
```To: <u2-users@listserver.u2ug.org>
Sent: Monday, October 10, 2011 9:16 AM
Subject: Re: [U2] The math just doesn't work.

```
I also thought: "great explanation" and nodded sagely when I read this. However, consider this code:
```

NUL = ''

CM = ','

```
A = -409071.8775: CM: 475000: CM: -652413: CM: 652413: CM: -475000: CM: 409071.8775
```
*

*

TEST.VMC = 1

TEST.TOTAL = 0

*

LOOP UNTIL FIELD(A, CM, TEST.VMC) = NUL DO

TEST.TOTAL += FIELD(A, CM, TEST.VMC)

TEST.VMC += 1

REPEAT

*

IF TEST.TOTAL = 0 THEN

PRINT 'OK'

END ELSE

```
PRINT TEST.TOTAL, (TEST.TOTAL = 0), TEST.TOTAL - 0, (OCONV(TEST.TOTAL, 'MD0') = 0)
```
END

Running this on our Universe system 11.1.1 produces:

0         0         0         1

```
So, in other words zero is the total result in TEST.TOTAL but this does not equate to zero.
```

```
Now I know I can get round this by rounding the answer but am not sure why I should have to. Any ideas?
```

Thanks

Brett

```
```
```
```Hi George,
```
```
```
```
```
```
```
```As a general rule in programming, comparison of floating point values
```
```
```
```for equality should be avoided. This is because, just as we cannot
```
```
```
```write the number one third accurately in decimal notation, so the IEEE
```
```
```
```floating point format used by computer systems cannot store numbers
```
```
```
```accurately. The example that I use when teaching training courses is
```
```
```
```14.2 which actually ends up as something close to 14.19999999997.
```
```
```
```
```
```
```
```UniVerse gets around this with a wonderful concept called "wide zero"
```
```
```
```that says, when testing for equality of floating point numbers, they
```
```
```
```must be within some specified value of being equal rather than
```
```
```
```strictly equal. The default wide zero error tolerance, set in IEEE
```
```
```
```format with the WIDEZERO configuration parameter, is 2.91 * 10^-11 (2^
```
```
```
035) which is good for most business applications but occasionally needs adjusting.
```
```
```
```
```
```
```Unidata has a command, SET.WIDEZERO, to serve the same purpose but
```
```
```
```defaults to 0.0 for backward comaptibility.
```
```
```
```
```
```
```
```
```
```
```
```Martin Phillips
```
```
```
```Ladybridge Systems Ltd
```
```
```
```17b Coldstream Lane, Hardingstone, Northampton NN4 6DB, England
```
```
```
```+44 (0)1604-709200
```
```
```
```
```
```
```
```
```
```
```
```----- Original Message -----
```
```
```
From: "George Hammerle" <zhamme...@hubert.com<mailto:zhamme...@hubert.com>>
```
```
```To: <u2-users@listserver.u2ug.org<mailto:u2-users@listserver.u2ug.org>>
```
```
```
```Sent: Thursday, September 29, 2011 1:30 PM
```
```
```
```Subject: [U2] The math just doesn't work.
```
```
```
```
```
```
```
```
```
```
```
```> Can anybody please help?
```
```
```
```>
```
```
```
```> For some reason A + B does not equal C in the comparison below. Is
```
```
```
```> there any trick to get the comparisons to work properly?
```
```
```
```>
```
```
```
```> Unidata 7.2 on Hp Unix 11+
```
```
```
```>
```
```
```
```>
```
```
```
```> Top of "TEST.COMP" in "RMH.MAIN", 13 lines, 263 characters.
```
```
```
```> *--: P
```
```
```
```> 001: A = 3176.79
```
```
```
```> 002: B = 106.19
```
```
```
```> 003: C = 3282.98
```
```
```
```> 004: D = 920.11
```
```
```
```> 005: A = A + D
```
```
```
```> 006: C = C + D
```
```
```
```> 007: IF (A+B) # C THEN
```
```
```
```> 008:   CRT '(A+B) # C? YOU LIE'
```
```
```
```> 009:   CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C
```
```
```
```> 010: END ELSE
```
```
```
```> 011:   CRT '(A+B) = C? YOU ROCK'
```
```
```
```> 012:   CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C
```
```
```
```> 013: END
```
```
```
```> Bottom.
```
```
```
```> *--: FIBR
```
```
```
```> Filed "TEST.COMP" in file "RMH.MAIN" unchanged.
```
```
```
```>
```
```
```
```> Compiling Unibasic: /db1/ud1/PGM/RMH.MAIN/TEST.COMP in mode 'u'.
```
```
```
```> compilation finished
```
```
```
```>
```
```
```
```> (A+B) # C? YOU LIE
```
```
```
```> A = 4096.9, B = 106.19, (A+B) = 4203.09, C = 4203.09
```
```
```
```>
```
```
```
```>
```
```
```
```>
```
```
```
```>
```
```
```
```>            George Hammerle
```
```
```
```>            Programming Dude
```
```
```
```>            Hubert Company LLC.
```
```
```
```>            9555 Dry Fork Road
```
```
```
```>            Harrison, Ohio 45030
```
```
```
```>            513-367-8974
```
```
```
> > zhammerle@hubertREMOVE_THIS.com<mailto:zhammerle@hubertREMOVE_THIS.com>
```
```
```
```
```
```
```_______________________________________________
```
```
```
```U2-Users mailing list
```
```
```
```U2-Users@listserver.u2ug.org<mailto:U2-Users@listserver.u2ug.org>
```
```
```
```http://listserver.u2ug.org/mailman/listinfo/u2-users
```
```
```
```
```
```
```
This message contains information that may be privileged or confidential and is the property of GPM Development Ltd. It is intended only for the person to whom it is addressed. If you are not the intended recipient ,you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.
```
```
This e-mail was sent to you by GPM Development Ltd. We are incorporated under the laws of England and Wales (company no. 2292156 and VAT registration no. 523 5622 63). Our registered office is 6th Floor, AMP House, Croydon, Surrey CR0 2LX.
```

_______________________________________________
U2-Users mailing list
U2-Users@listserver.u2ug.org
```
http://listserver.u2ug.org/mailman/listinfo/u2-users
```
_______________________________________________
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
```