New topic: Ceil bug, or am I just missing something?
<http://forums.realsoftware.com/viewtopic.php?t=47868> Page 1 of 1 [ 6 posts ] Previous topic | Next topic Author Message clarusmoof Post subject: Ceil bug, or am I just missing something?Posted: Mon May 13, 2013 6:15 pm Joined: Wed Feb 13, 2013 10:02 pm Posts: 7 Make a new RB project and put this in the open event of the window: dim d as double=1.1 MsgBox cstr(ceil(d*100)/100) You should receive a message box with "1.1", but instead it returns "1.11". Any other double (2.1, 3.1, 3.3, etc) returns the proper ceil that you'd expect. Is this a bug? Is it correct? And how do I account for this when rounding? Top timhare Post subject: Re: Ceil bug, or am I just missing something?Posted: Mon May 13, 2013 7:08 pm Joined: Fri Jan 06, 2006 3:21 pm Posts: 12314 Location: Portland, OR USA It is correct. 1.1 is represented in floating point notation as roughly 1.100000000001 d*100 is then something like 110.00000000001 ceil(110.00000000001) is 111 Floating point fractional values are not precise. How you account for it depends a lot on what you're doing with the result. Top clarusmoof Post subject: Re: Ceil bug, or am I just missing something?Posted: Mon May 13, 2013 7:47 pm Joined: Wed Feb 13, 2013 10:02 pm Posts: 7 Thanks Tim, very interesting. I'm trying to round for currency, taking one number (eg. 1.1) and multiplying it by another number (eg 125.50) and coming up with a result rounded up to the nearest cent. Top Jason_Adams Post subject: Re: Ceil bug, or am I just missing something?Posted: Mon May 13, 2013 8:15 pm Joined: Fri Nov 10, 2006 4:10 pm Posts: 1831 Location: Michigan, USA Instead of using a Double, you might look into the Currency type. _________________ Windows 8 x64 Windows XP Pro SP3 Ubuntu 11.04 via Virtual Box RS Enterprise 2012r1.1 Programming Tutorials & Free Projects: http://www.JasonTheAdams.com "Christianity has not been tried and found wanting; it has been found difficult and not tried." - G.K. Chesterton Top clarusmoof Post subject: Re: Ceil bug, or am I just missing something?Posted: Mon May 13, 2013 8:20 pm Joined: Wed Feb 13, 2013 10:02 pm Posts: 7 I do use the currency variable. I was just using a double for a quick demonstration of my question. Changing the type to currency does not change the rounding behavior though: dim d as currency=1.1 MsgBox cstr(ceil(d*100)/100) Still produces 1.11. Top timhare Post subject: Re: Ceil bug, or am I just missing something?Posted: Tue May 14, 2013 12:27 am Joined: Fri Jan 06, 2006 3:21 pm Posts: 12314 Location: Portland, OR USA Um, wouldn't you use Round() instead of Ceil()? MsgBox cstr(round(d*100)/100) or better MsgBox format(d, "#.00") Top Display posts from previous: All posts1 day7 days2 weeks1 month3 months6 months1 year Sort by AuthorPost timeSubject AscendingDescending Page 1 of 1 [ 6 posts ] -- Over 1500 classes with 29000 functions in one REALbasic plug-in collection. The Monkeybread Software Realbasic Plugin v9.3. http://www.monkeybreadsoftware.de/realbasic/plugins.shtml [email protected]
