You could always use the NumberFormatter class and use the precision property:
var numberFormatter:NumberFormatter = new NumberFormatter; numberFormatter.precision = 2; var myNum:Number = Number( numberFormatter.format( myOldNum ) ); This will give you the old number formatted to 2 decimal places (or however many you need). From here you could use your "tolerance" to compare the 2 numbers. --- In [email protected], "Tim Hoff" <[EMAIL PROTECTED]> wrote: > > > Hi fumeng, > > You could work this out using Math.round(). Although this method rounds > to the nearest integer (doesn't use a precision parameter), with a > little work this could help you compare apples to apples. > > -TH > > --- In [email protected], "fumeng5" <fumeng5@> wrote: > > > > Hi, > > > > I'm trying to compare 3 points but because of double precision in AS3 > > i'm running into some issues with trailing digits causing the numbers > > to not be equal, i.e. 1.25 != 1.2499999999999 > > > > what i'd like to figure out how to do is introduce a tolerance of say > > .001. then i'd compare the test point against the other two points > > saying: is testPoint.x within the range of point1.x and point2.x and > > is testPoint.y within the range of point1.y and point2.y. if so, > > return true. > > > > i just can't figure it out. here's what i have so far: > > > > public static function isEndPoint(p:Point, Lp1:Point, > Lp2:Point):Boolean { > > var tolerance:int = .001; > > var newLp1:Point = new Point(tolerance * Lp1.x,tolerance * Lp1.y); > > var newLp2:Point = new Point(tolerance * Lp2.x,tolerance * Lp2.y); > > return (p.equals(newLp1) || p.equals(newLp2)); > > } > > > > I guess I'm seeing that the equals() method at the bottom of this code > > is not really what i want. Plus, multiplying everything by .001 > > doesn't solve the problem as i'll essentially be using the same > > numbers i started with. > > > > Can someone perhaps point me in the right direction here, please? I'm > > not sure where exactly to go from here. > > > > Thanks, > > > > fumeng. > > >
