[Flashcoders] problem with adding two digits
Hello, First of all, I want to say Hi to everyone here. This is my first post. Please, take a look at this part of code: trace (0.27 + 0.03); // output -- 0.30004 Can anyone tell me why I get this weird result in output window? Regards Adrian ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
RE: [Flashcoders] problem with adding two digits
It has to do with the fact that computers have to convert all numbers to binary then perform math on those numbers and then convert back to decimal. In the process of this conversion you get results like these. If you need the addition to be accurate and you know that the numbers will always be decimals you can multiply the numbers by 100 and then you will add 27 + 3 which will return 30 and then divide it by 100 to get your .3 (or at least I think that will work as I have not tried it) -Original Message- From: flashcoders-boun...@chattyfig.figleaf.com [mailto:flashcoders-boun...@chattyfig.figleaf.com] On Behalf Of Adrian Zajac Sent: Tuesday, December 14, 2010 8:43 AM To: Flash Coders List Subject: [Flashcoders] problem with adding two digits Hello, First of all, I want to say Hi to everyone here. This is my first post. Please, take a look at this part of code: trace (0.27 + 0.03); // output -- 0.30004 Can anyone tell me why I get this weird result in output window? Regards Adrian ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
same here compiling for flash player 10 and flash player 9, 8 and below give 0.3 as expected On 14 December 2010 15:42, Adrian Zając zajac.adr...@gmail.com wrote: trace (0.27 + 0.03); ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
If you need the addition to be accurate and you know that the numbers will always be decimals you can multiply the numbers by 100 and then you will add 27 + 3 which will return 30 and then divide it by 100 to get your .3 (or at least I think that will work as I have not tried it) Yes Steve, probably this is the best solution. But still I want to know why is it working in that way. Funny thing with flash player versions... Tom, thanks for checking it out. W dniu 2010-12-14 16:15, tom rhodes pisze: same here compiling for flash player 10 and flash player 9, 8 and below give 0.3 as expected On 14 December 2010 15:42, Adrian Zajączajac.adr...@gmail.com wrote: trace (0.27 + 0.03); ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
It's an issue with floating point accuracy/calculations, Here is some info: http://joshblog.net/2007/01/30/flash-floating-point-number-errors/ saw more about it somewhere else... don't think it's an issue with the FP tho'. - Karim On 14 Dec 2010, at 15:15, tom rhodes wrote: same here compiling for flash player 10 and flash player 9, 8 and below give 0.3 as expected On 14 December 2010 15:42, Adrian Zając zajac.adr...@gmail.com wrote: trace (0.27 + 0.03); ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
I like to quote this, from PHP.net's Floating Point documentation: Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error propagation must be considered when several operations are compounded. Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9991118 So never trust floating number results to the last digit, and never compare floating point numbers for equality. Additional, interesting read: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems On Tue, Dec 14, 2010 at 10:15 AM, tom rhodes tom.rho...@gmail.com wrote: same here compiling for flash player 10 and flash player 9, 8 and below give 0.3 as expected On 14 December 2010 15:42, Adrian Zając zajac.adr...@gmail.com wrote: trace (0.27 + 0.03); ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
Howdy Adrian, Here is one way. trace (int((0.27 + 0.03)*100)/100) // output -- 0.3 Later, John on 12/14/10 8:42 AM, Adrian Zając at zajac.adr...@gmail.com wrote: Hello, First of all, I want to say Hi to everyone here. This is my first post. Please, take a look at this part of code: trace (0.27 + 0.03); // output -- 0.30004 Can anyone tell me why I get this weird result in output window? Regards Adrian John R. Sweeney Jr. Interactive Multimedia Developer OnDemand Interactive Inc 945 Washington Blvd. Hoffman Estates, IL 60169 Office/Fax: 847.310.5959 Cellular: 847.651.4469 www.ondemandinteractive.com ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
yup interesting that the old AVM gives you what you'd think... On 14 December 2010 16:36, Zeh Fernando z...@zehfernando.com wrote: I like to quote this, from PHP.net's Floating Point documentation: Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error propagation must be considered when several operations are compounded. Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9991118 So never trust floating number results to the last digit, and never compare floating point numbers for equality. Additional, interesting read: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems On Tue, Dec 14, 2010 at 10:15 AM, tom rhodes tom.rho...@gmail.com wrote: same here compiling for flash player 10 and flash player 9, 8 and below give 0.3 as expected On 14 December 2010 15:42, Adrian Zając zajac.adr...@gmail.com wrote: trace (0.27 + 0.03); ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
tom rhodes skriver: yup interesting that the old AVM gives you what you'd think... Pure luck, you might have compiled against something that happened to drop the decimal instead . ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
Thanks Zeh, now I understand. So do we have to keep an eye on our variables so we don't have for example:7.998 * 0,3004 ? Because I think it is a little heavier for processors to count than: 8 * 0.3 W dniu 2010-12-14 16:36, Zeh Fernando pisze: I like to quote this, from PHP.net's Floating Point documentation: Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error propagation must be considered when several operations are compounded. Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9991118 So never trust floating number results to the last digit, and never compare floating point numbers for equality. Additional, interesting read: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems On Tue, Dec 14, 2010 at 10:15 AM, tom rhodestom.rho...@gmail.com wrote: same here compiling for flash player 10 and flash player 9, 8 and below give 0.3 as expected On 14 December 2010 15:42, Adrian Zajączajac.adr...@gmail.com wrote: trace (0.27 + 0.03); ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] problem with adding two digits
Adrian Zając wrote: trace (0.27 + 0.03); // output -- 0.30004 Can anyone tell me why I get this weird result in output window? As people have said, it's a problem with decimals. It's not a problem with Flash--it's a problem with binary numbers. Integers are accurate because all integers are a multiple of 1, and binary does just fine with combining 0's and 1's, the only possibile values of a digit in the binary system. If you think of decimals as fractions, maybe it will help you understand the issue. In binary, a digit can only be a multiple of 2: 1/2, 1/4, 1/8/ 1/16, 1/32, and so on. You can create numbers that aren't multiples of two by adding multiple digits. For example, 1/4 + 1/8 = 3/8, or .375. This works pretty well, until you get numbers with a lot of decimal places. At some point, you will find a decimal that is impossible to make using powers of 2. Cordially, Kerry Thompson ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
