to prevent anyone answering on this thread:
after toying with it again, and reading the docs + wikipedia I figured
out that I can only trust a result of 15 digits or less, unless I have
the string be returned instead of the number.
n 12/18/2010 8:34 PM, Anthony Pace wrote:
function
whoa... hold on Help Please.
After I posted yesterday, (or the night before??? can't remember) as I
realized how badly the code sucked, I was just messing around with this
code below to see if I could separate the halves and recompose them
properly; however,
if you pop the code below on
I wrote the functions below in literally just a few min, so, even though
they do seem to work for me, I wouldn't necessarily say they are
production ready.
Also, I would only use these if you will know that the numbers past the
mantissa are of small amounts.
Oh yeah, and please tell me if I
Oh yeah, I forgot to give some examples to test it with, so...
trace(floatSum(-0.1,0.9155,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1));
//vs
trace(-0.1+0.9155+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1);
//and
trace(floatProduct(2.1,0.1,1.0001,.0001));
//vs
trace(2.1*0.1*1.0001*.0001);
On 12/15/2010 2:25 PM,
It has to do with the fact that computers have to convert all numbers to binary
then perform math on those numbers and then convert back to decimal. In the
process of this conversion you get results like these.
If you need the addition to be accurate and you know that the numbers will
always
same here compiling for flash player 10 and flash player 9, 8 and below give
0.3 as expected
On 14 December 2010 15:42, Adrian Zając zajac.adr...@gmail.com wrote:
trace (0.27 + 0.03);
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
If you need the addition to be accurate and you know that the numbers will
always be decimals you can multiply the numbers by 100 and then you will add 27
+ 3 which will return 30 and then divide it by 100 to get your .3 (or at least
I think that will work as I have not tried it)
Yes
It's an issue with floating point accuracy/calculations,
Here is some info:
http://joshblog.net/2007/01/30/flash-floating-point-number-errors/
saw more about it somewhere else... don't think it's an issue with the FP tho'.
- Karim
On 14 Dec 2010, at 15:15, tom rhodes wrote:
same here
I like to quote this, from PHP.net's Floating Point documentation:
Floating point numbers have limited precision. Although it depends on the
system, PHP typically uses the IEEE 754 double precision format, which will
give a maximum relative error due to rounding in the order of 1.11e-16. Non
Howdy Adrian,
Here is one way.
trace (int((0.27 + 0.03)*100)/100) // output -- 0.3
Later,
John
on 12/14/10 8:42 AM, Adrian Zając at zajac.adr...@gmail.com wrote:
Hello,
First of all, I want to say Hi to everyone here. This is my first post.
Please, take a look at this part of
yup interesting that the old AVM gives you what you'd think...
On 14 December 2010 16:36, Zeh Fernando z...@zehfernando.com wrote:
I like to quote this, from PHP.net's Floating Point documentation:
Floating point numbers have limited precision. Although it depends on the
system, PHP
tom rhodes skriver:
yup interesting that the old AVM gives you what you'd think...
Pure luck, you might have compiled against something that happened to
drop the decimal instead .
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
Thanks Zeh, now I understand.
So do we have to keep an eye on our variables so we don't have for
example:7.998 * 0,3004 ?
Because I think it is a little heavier for processors to count than: 8
* 0.3
W dniu 2010-12-14 16:36, Zeh Fernando pisze:
I like to quote
Adrian Zając wrote:
trace (0.27 + 0.03); // output -- 0.30004
Can anyone tell me why I get this weird result in output window?
As people have said, it's a problem with decimals. It's not a problem with
Flash--it's a problem with binary numbers.
Integers are accurate
14 matches
Mail list logo
