Thanks Wally and Colin,
That works a treat. I am surprised though that Unidata 'out of the box'
has this problem and you have to work round it.
regards
Martin Hutchinson
Wally Terhune wrote:
look up the ECL statement FLOAT.PRECISION
Folks are often happiest with FLOAT.PRECISION 4
Wally Terhune
U2 Support Architect
IBM Information Management Software
Tel: (303) 773-7969 T/L 656-7969
Mobile: (303) 807-6222
Email: [email protected]
http://www.ibm.com/software/data/u2/support
From: "Martin Hutchinson" <[email protected]>
To: "u2 user group" <[email protected]>
Date: 05/01/2009 05:21 AM
Subject: [U2] Unidata calculations incorrect
Sent by: [email protected]
Hi All,
We have Unidata 7.2 running on Windows 2008 server making incorrect
calculations when numbers are close to zero. I understand there is a
'WIDEZERO' parameter on Univers but I don't think there is an equivalent
on Unidata. Below we have an example program and output.
001: BAL = 856.19
002: VAL1 = 287.31
003: VAL2 = 568.88
004: BAL = BAL - VAL1
005: BAL = BAL - VAL2
006: *
007: CRT "BAL [":BAL:"]"
008: *
009: IF BAL > 0 THEN
010: CRT "BAL > 0"
011: END ELSE
012: CRT "BAL <= 0"
013: END
014: *
015: IF INT(BAL) = BAL THEN
016: CRT "Matches integer"
017: END ELSE
018: CRT "Does not match integer"
019: END
020: *
021: IF ICONV(BAL,"MCN") > 0 THEN
022: CRT "MCN BAL > 0"
023: END ELSE
024: CRT "MCN BAL <= 0"
025: END
When run this gives:
BAL [0]
BAL > 0
Does not match integer
MCN BAL <= 0
Has anyone come across this one or have any ideas?
regards
Martin Hutchinson
________________________
Web Control (UK) Ltd
Atlantic House,
Michigan Drive,
Tongwell,
Milton Keynes,
MK15 8HQ
T: +44(0)1908 216700 x4
F: +44(0)1908 216300
http://www.webcontrolltd.co.uk
Web Control (UK) Ltd is a company registered in England and Wales with
company number 4118558. VAT number: 770 3363 38
-------
u2-users mailing list
[email protected]
To unsubscribe please visit http://listserver.u2ug.org/
[demime 1.01d removed an attachment of type image/gif which had a name of
graycol.gif]
[demime 1.01d removed an attachment of type image/gif which had a name of
ecblank.gif]
-------
u2-users mailing list
[email protected]
To unsubscribe please visit http://listserver.u2ug.org/
________________________
Web Control (UK) Ltd
Atlantic House,
Michigan Drive,
Tongwell,
Milton Keynes,
MK15 8HQ
T: +44(0)1908 216700 x4
F: +44(0)1908 216300
http://www.webcontrolltd.co.uk
Web Control (UK) Ltd is a company registered in England and Wales with company
number 4118558. VAT number: 770 3363 38
-------
u2-users mailing list
[email protected]
To unsubscribe please visit http://listserver.u2ug.org/
