Running Martin's code on a Universe system, got: (A+B) = C? YOU ROCK A = 4096.9, B = 106.19, (A+B) = 4203.09, C = 4203.09
Interesting - Universe does better than Unidata at this particular task. Harold Oaks Clark County -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Brian Leach Sent: Thursday, September 29, 2011 6:21 AM To: 'U2 Users List' Subject: Re: [U2] The math just doesn't work. And just to add to Martin's excellent answer - the normal workround for any computer system is to use integral values, scaling and descaling as required. In case you think this is a U2 specific limitation, here's the same in C# as a console app - namespace ConsoleApplication1 { class Program { static void Main(string[] args) { float A = 3176.79f; float B = 106.19f; float C = 3282.98f; float D = 920.11f; A = A + D; C = C + D ; if((A+B) != C ){ Console.WriteLine("You lie, A+B = {0}, C= {1}",(A+B), C); } else{ Console.WriteLine("you rock"); } pressAnyKey(); } static void pressAnyKey() { Console.Write("Press any key ..."); Console.ReadLine(); } } } Result - You lie, A+B = 4203.09, C= 4203.09 -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Martin Phillips Sent: 29 September 2011 13:46 To: U2 Users List Subject: Re: [U2] The math just doesn't work. Hi George, As a general rule in programming, comparison of floating point values for equality should be avoided. This is because, just as we cannot write the number one third accurately in decimal notation, so the IEEE floating point format used by computer systems cannot store numbers accurately. The example that I use when teaching training courses is 14.2 which actually ends up as something close to 14.19999999997. UniVerse gets around this with a wonderful concept called "wide zero" that says, when testing for equality of floating point numbers, they must be within some specified value of being equal rather than strictly equal. The default wide zero error tolerance, set in IEEE format with the WIDEZERO configuration parameter, is 2.91 * 10^-11 (2^ 035) which is good for most business applications but occasionally needs adjusting. Unidata has a command, SET.WIDEZERO, to serve the same purpose but defaults to 0.0 for backward comaptibility. Martin Phillips Ladybridge Systems Ltd 17b Coldstream Lane, Hardingstone, Northampton NN4 6DB, England +44 (0)1604-709200 ----- Original Message ----- From: "George Hammerle" <zhamme...@hubert.com> To: <u2-users@listserver.u2ug.org> Sent: Thursday, September 29, 2011 1:30 PM Subject: [U2] The math just doesn't work. > Can anybody please help? > > For some reason A + B does not equal C in the comparison below. Is there > any trick to get the comparisons to work properly? > > Unidata 7.2 on Hp Unix 11+ > > > Top of "TEST.COMP" in "RMH.MAIN", 13 lines, 263 characters. > *--: P > 001: A = 3176.79 > 002: B = 106.19 > 003: C = 3282.98 > 004: D = 920.11 > 005: A = A + D > 006: C = C + D > 007: IF (A+B) # C THEN > 008: CRT '(A+B) # C? YOU LIE' > 009: CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C > 010: END ELSE > 011: CRT '(A+B) = C? YOU ROCK' > 012: CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C > 013: END > Bottom. > *--: FIBR > Filed "TEST.COMP" in file "RMH.MAIN" unchanged. > > Compiling Unibasic: /db1/ud1/PGM/RMH.MAIN/TEST.COMP in mode 'u'. > compilation finished > > (A+B) # C? YOU LIE > A = 4096.9, B = 106.19, (A+B) = 4203.09, C = 4203.09 > > > > > George Hammerle > Programming Dude > Hubert Company LLC. > 9555 Dry Fork Road > Harrison, Ohio 45030 > 513-367-8974 > zhammerle@hubertREMOVE_THIS.com _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users This e-mail and related attachments and any response may be subject to public disclosure under state law. _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
