I got the following for UD v7.2.2 on Windows 2K3 Server w/all SPs :FLOAT.PRECISION FLOAT.PRECISION mode 0 :RUN BP TEST BAL [0] BAL > 0 Does not match integer MCN BAL <= 0 :FLOAT.PRECISION 4 :RUN BP TEST BAL [0] BAL <= 0 Matches integer MCN BAL <= 0 Bill ______________________________________________________________________
From: Horn, John <[email protected]> Sent: 5/1/2009 7:11 AM To: [email protected] Subject: Re: [U2] Unidata calculations incorrect Interesting. This works as expected on our system. Unidata 6.something on AIX. I wonder what is different in our settings. :FLOAT.PRECISION FLOAT.PRECISION mode 3 :RUN FIXBP JH.TST BAL [0] BAL <= 0 Matches integer MCN BAL <= 0 John M. Horn | Solutions Engineer, Advisor Sr. | HealthLink | 314-989-6050 | [email protected] -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Wally Terhune Sent: Friday, May 01, 2009 8:00 AM To: [email protected] Subject: Re: [U2] Unidata calculations incorrect look up the ECL statement FLOAT.PRECISION Folks are often happiest with FLOAT.PRECISION 4 Wally Terhune U2 Support Architect IBM Information Management Software Tel: (303) 773-7969 T/L 656-7969 Mobile: (303) 807-6222 Email: [email protected] http://www.ibm.com/software/data/u2/support From: "Martin Hutchinson" <[email protected]> To: "u2 user group" <[email protected]> Date: 05/01/2009 05:21 AM Subject: [U2] Unidata calculations incorrect Sent by: [email protected] Hi All, We have Unidata 7.2 running on Windows 2008 server making incorrect calculations when numbers are close to zero. I understand there is a 'WIDEZERO' parameter on Univers but I don't think there is an equivalent on Unidata. Below we have an example program and output. 001: BAL = 856.19 002: VAL1 = 287.31 003: VAL2 = 568.88 004: BAL = BAL - VAL1 005: BAL = BAL - VAL2 006: * 007: CRT "BAL [":BAL:"]" 008: * 009: IF BAL > 0 THEN 010: CRT "BAL > 0" 011: END ELSE 012: CRT "BAL <= 0" 013: END 014: * 015: IF INT(BAL) = BAL THEN 016: CRT "Matches integer" 017: END ELSE 018: CRT "Does not match integer" 019: END 020: * 021: IF ICONV(BAL,"MCN") > 0 THEN 022: CRT "MCN BAL > 0" 023: END ELSE 024: CRT "MCN BAL <= 0" 025: END When run this gives: BAL [0] BAL > 0 Does not match integer MCN BAL <= 0 Has anyone come across this one or have any ideas? regards Martin Hutchinson ________________________ Web Control (UK) Ltd Atlantic House, Michigan Drive, Tongwell, Milton Keynes, MK15 8HQ T: +44(0)1908 216700 x4 F: +44(0)1908 216300 http://www.webcontrolltd.co.uk Web Control (UK) Ltd is a company registered in England and Wales with company number 4118558. VAT number: 770 3363 38 ------- u2-users mailing list [email protected] To unsubscribe please visit http://listserver.u2ug.org/ [demime 1.01d removed an attachment of type image/gif which had a name of graycol.gif] [demime 1.01d removed an attachment of type image/gif which had a name of ecblank.gif] ------- u2-users mailing list [email protected] To unsubscribe please visit http://listserver.u2ug.org/ CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and priv ileged information or otherwise protected by law. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipien t, please contact the sender by reply e-mail and destroy all copies of the orig inal message. ------- u2-users mailing list [email protected] To unsubscribe please visit http://listserver.u2ug.org/ ------- u2-users mailing list [email protected] To unsubscribe please visit http://listserver.u2ug.org/
