Hi Nargis,
I wrote a small program that checks
how precise the precision method really is.
And the results are as follows.
conside a scenario
num = 2 //please note that if we omit the decimal after the number it will
be
//treated as integer and in this case the result is ok
trace(myNum.toPrecision(18))
the result is 2.00000000000000000 // now all the places filled with 0s
num = 2.2 //now the number is floating point
if precision level is <=16 results are same like we are treating integer
numbers
trace(myNum.toPrecision(16))
the result is 2.200000000000000
num = 2.2 //now the number is floating point
if precision level is (>or equal than)17 then i start getting weird results
trace(myNum.toPrecision(18));
trace(myNum.toPrecision(19));
the result with precision level 18 is = 2.200000000000000*18* // notice the
different behavior
the result with precision level 19 is =2.200000000000000*178*// notice the
different behavior again
I will try to check why it is giving results in such a way but for now may
be this will be of some help.
On Tue, Mar 23, 2010 at 3:29 PM, Arindam Dhar <[email protected]> wrote:
> Is it mandatory that getMaxValue() has to return a Number ? If not and ur
> ultimate goal is to populate txtOutput.text u can return a string ,
>
> private function getMaxValue():String{
>
> return
> "9999999999999.99999";
>
> }
>
> just a thought.
>
> regards,
>
> A.
>
>
> On Fri, Mar 19, 2010 at 12:20 PM, Nargis <[email protected]> wrote:
>
>> Hi,
>>
>> I am getting some issue with number data type.
>> Please check out following
>>
>>
>> <?xml version="1.0" encoding="utf-8"?>
>> <mx:Application xmlns:mx="http://www.adobe.com/2006/mxml";
>> layout="vertical">
>> <mx:Script>
>> <![CDATA[
>> private function getMaxValue():Number{
>>
>> return Number(9999999999999.99999);
>> }
>> private function clickHandler():void{
>> var num:Number = getMaxValue();
>> trace(num);
>> txtOutput.text = getMaxValue().toString();
>> }
>> ]]>
>> </mx:Script>
>> <mx:Text id="txtOutput" />
>> <mx:Button label="Get Value" click="clickHandler()"/>
>> </mx:Application>
>>
>>
>> Here i am expecting out put as (9999999999999.99999) but the function
>> returns 10000000000000.
>>
>> Please let me know if any one has solution for this.
>>
>> -Nargis
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Flex India Community" group.
>> To post to this group, send email to [email protected].
>> To unsubscribe from this group, send email to
>> [email protected]<flex_india%[email protected]>
>> .
>>
>> For more options, visit this group at
>> http://groups.google.com/group/flex_india?hl=en.
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Flex India Community" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected]<flex_india%[email protected]>
> .
> For more options, visit this group at
> http://groups.google.com/group/flex_india?hl=en.
>
--
Regards
Nakul
--
You received this message because you are subscribed to the Google Groups "Flex
India Community" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/flex_india?hl=en.
