Hello Paul, As Uffe has pointed out, the problem is with floating point numbers not representing the results of your calculation exactly. If you enter the following in the MapBasic window, it will demonstrate what's happening:
print format$((145.2-fix(145.2))*60, "0.0000000000000000000") Produces: 11.99999999999318 So, that last fix() rounds it down to 11. My advice it to leave it out. Regards, Warren Vick Europa Technologies Ltd. http://www.europa-tech.com -----Original Message----- From: Uffe Kousgaard [mailto:[EMAIL PROTECTED] Sent: 18 January 2005 07:26 To: Mapinfo-L Subject: Re: MI-L When 12 = 11 ( Error maybe in mapbasic/mapinfo) Welcome to the world of floating point math ! Floating point numbers are usually expressed as binary numbers internally, which are not always exact. In this case 0.2 may be stored as 0.1999999. Multiply that with 60 and you get something a bit smaller than 12. Use the fix funtion and you get 11. Similarly 145.2 - 145 <> 0.2, when using floating point numbers. So you mat get the correct result using straight 0.2, but not using 145.2 - 145. Here is an explanation (Delphi based, but same principle): http://www.guidogybels.net/index.html?floats.html Kind regards Uffe Kousgaard www.routeware.dk ----- Original Message ----- From: "Paul Smith" <[EMAIL PROTECTED]> To: <[email protected]> Sent: Tuesday, January 18, 2005 8:08 AM Subject: MI-L When 12 = 11 ( Error maybe in mapbasic/mapinfo) Hi List, I have been working on some of my own tools and in a equation working with Lat's and Long's I have found something odd. What I was trying to do was take a float and give the result of the float without the preceding integer ie. 145.2 = 0.2 (Remove the 145) Then multiply the result by 60 ie. 0.2 x 60 = 12 But wait fo it: In mapbasic 12 = 11 Yes you all think I am crazy but here is my proof. My equation is: print fix((abs(fltvariable)-fix(abs(fltvariable)))*60) subtituting fltvariable for 145.2 (makes not different if you use any other number as long as it has 0.2 as the decimal) Type this equation into the mapbasic window and even try them on paper yourself. print fix((abs(145.2)-fix(abs(145.2)))*60) Working through the equation we would have print fix((abs(145.2)-fix(abs(145.2)))*60) = 11 ? print fix((145.2-fix(145.2))*60) = 11 ? print fix((145.2-145)*60) = 11 ? print fix(0.2*60) = 12 Correct print fix(12) = 12 Correct The answer should be 12 but mapbasic is giving 11 as the result for the first 3 equations. It this a bug that should be fixed or been fixed? Paul Smith --------------------------------------------------------------------- List hosting provided by Directions Magazine | www.directionsmag.com | To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Message number: 14895 --------------------------------------------------------------------- List hosting provided by Directions Magazine | www.directionsmag.com | To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Message number: 14898
