[cfaussie] Re: Strange calculation behavior on CF 7.02

[Gareth Edwards]

Sorry to complain about it, it just seems strange, I won't complain anymore. Does it do the same thing in Actionscript? (don't have time to check right now) What do you think is the best way to handle this in coldfusion? Also, I'm curious, how many people on this list are aware of this issue? Cheers Gareth. Robin Hilliard wrote: It's easier to see the pattern without the random stuff and the extra additions and subtractions, note val() etc have nothing to do with it: <cfloop from="1" to="10000" index="a"> <cfset c = a + 0.1> <cfset d = a - c> <cfoutput>#a# - #c# = #d#<br/></cfoutput> </cfloop> 1019 - 1019.1 = -0.1 1020 - 1020.1 = -0.1 1021 - 1021.1 = -0.1 1022 - 1022.1 = -0.1 1023 - 1023.1 = -0.1 1024 - 1024.1 = -0.0999999999999 1025 - 1025.1 = -0.0999999999999 1026 - 1026.1 = -0.0999999999999 1027 - 1027.1 = -0.0999999999999 1028 - 1028.1 = -0.0999999999999 ... 4090 - 4090.1 = -0.0999999999999 4091 - 4091.1 = -0.0999999999999 4092 - 4092.1 = -0.0999999999999 4093 - 4093.1 = -0.0999999999999 4094 - 4094.1 = -0.0999999999999 4095 - 4095.1 = -0.0999999999999 4096 - 4096.1 = -0.1 4097 - 4097.1 = -0.1 4098 - 4098.1 = -0.1 4099 - 4099.1 = -0.1 4100 - 4100.1 = -0.1 4101 - 4101.1 = -0.1 4102 - 4102.1 = -0.1 4103 - 4103.1 = -0.1 4104 - 4104.1 = -0.1 I get errors for the range 1024 <= a < 4096. Note 1024 = 2 ^ 10 and 4096 = 2 ^ 12 - significant numbers in the binary world. These sorts of errors are caused because you cannot represent all fractional amounts in a finite number of binary digits - for instance 0.1 is an endlessly repeating 1100110011001100... pattern in binary (just like 1/3 is an endlessly repeating series in decimal). Replace 0.1 in the above code with 0.125 (1/8) or another fraction that can be represented in a finite number of binary digits and you'll see the error disappears, or if you use 0.7 you'll see the error kick in at a >= 8192 (2 ^ 13). Without further analysis, it's probably something to do with the way negative numbers (see "two's complement") are represented in binary, and the point in the repeating sequence of binary digits where a Java floating point number runs out of precision. As Ryan suggested, these rounding errors happen on any digital computer/language and your code needs to deal with them gracefully. Cheers, Robin ______________ Robin Hilliard Director - RocketBoots Pty Ltd Consulting . Recruitment . Software Licensing . Training http://www.rocketboots.com.au For schedule/availability call Pamela Higgins: w +61 7 5451 0362 m +61 419 677 151 f +61 3 9923 6261 e [EMAIL PROTECTED] or Direct: m +61 418 414 341 e [EMAIL PROTECTED] On 20/02/2007, at 12:04 PM, Gareth Edwards wrote: If you are running coldfusion 7.02, or maybe 7.01 or 7.0? (don't know) Run this script below, and tell me if you get the same results for each line. The way I see it, d should equal 0.1 on every row. But this isn't the case. Is this a bug introduced by Coldfusion 7.02? <cfset a = 0> <cfset b = 0> <cfset c = 0> <cfloop from="1" to="1000" index="i"> <cfset a = RandRange(10000,20000)> <cfset b = a*2> <cfset c = "#ListFirst(Trim(a),".")#.00"> <cfset c = c + 0.1> <cfset d = a - val(b) + val(c)> <cfoutput>#d#<br /></cfoutput> </cfloop> Cheers Gareth. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to cfaussie@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en -~----------~----~----~----~------~----~------~--~---