Hi Robin, What exactly is a 'CF number'? It seems to be CF can store it as a string or a double / float depending on the way it is constructed or what methods are used on the var. I agree with Gareth in that CF needs to make untyped numbers in CF more consistent. The values generated from Gareth's code shouldn't happen and no I don't agree with the opinion it is a fault / known issue of the Java language, it is simply the way CF was written to use Java. They decided to go with a smaller memory footprint for storing numeric values. This is a very annoying issue when dealing with very small numbers in CF and has driven me nuts plenty of times before.
Obviously using BigDecimal for every numeric in CF would use too much memory but maybe some option for the construction of 'CF numbers' so developers can choose to be more precise. Rod -----Original Message----- From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Robin Hilliard Sent: Wednesday, 21 February 2007 8:03 AM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Hi Rod, Big Decimal has far more precision than a CF number (only 12 significant digits) and has built in rounding - you can set the precision with it's setScale(scale, rounding-mode) method. One point to note if you are using BigDecimal - you should use the string constructor i.e. init("0.1") because otherwise ColdFusion will convert the literal 0.1 to a ColdFusion number (with it's more limited precision) before passing it to Java. Robin ______________ Robin Hilliard Director - RocketBoots Pty Ltd Consulting . Recruitment . Software Licensing . Training http://www.rocketboots.com.au For schedule/availability call Pamela Higgins: w +61 7 5451 0362 m +61 419 677 151 f +61 3 9923 6261 e [EMAIL PROTECTED] or Direct: m +61 418 414 341 e [EMAIL PROTECTED] On 20/02/2007, at 6:43 PM, rod higgins wrote: > > If you use the java class and methods that handle these types of > numbers I don't see the problem?? > > <cfloop from="1" to="1000" index="i"> > <cfset diff = CreateObject("Java", "java.math.BigDecimal").init(0.1)> > <cfset a = CreateObject("Java", > "java.math.BigDecimal").init(RandRange(100,10000))> > <cfset b = CreateObject("Java", > "java.math.BigDecimal").init(a*2)> > <cfset c = CreateObject("Java", > "java.math.BigDecimal").init(val(a))> > <cfset c = c.add(diff)> > <cfset d = a.subtract(b).add(c)> > <cfoutput>#a#-#b#+#c# = #d#<br /></cfoutput> > </cfloop> > > It seems CF is using some sort of toString() value during the > calculation at some stage?? > > hth > Rod > > On Feb 20, 12:04 pm, Gareth Edwards <[EMAIL PROTECTED]> wrote: >> If you are running coldfusion 7.02, or maybe 7.01 or 7.0? (don't >> know) >> >> Run this script below, and tell me if you get the same results for >> each >> line. The way I see it, d should equal 0.1 on every row. But this >> isn't >> the case. Is this a bug introduced by Coldfusion 7.02? >> >> <cfset a = 0> >> <cfset b = 0> >> <cfset c = 0> >> >> <cfloop from="1" to="1000" index="i"> >> <cfset a = RandRange(10000,20000)> >> <cfset b = a*2> >> <cfset c = "#ListFirst(Trim(a),".")#.00"> >> <cfset c = c + 0.1> >> <cfset d = a - val(b) + val(c)> >> <cfoutput>#d#<br /></cfoutput> >> </cfloop> >> >> Cheers >> Gareth. > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to cfaussie@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en -~----------~----~----~----~------~----~------~--~---
