[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-21 Thread Joel Cass
I remember someone raising this issue yonks ago in CF 4.5 or something A quick fix: #round((1024-1024.1)*10)/10# On Wed, 2007-02-21 at 13:38 +1100, Robin Hilliard wrote: What exactly is a 'CF number'? All simple CF variables are stored as strings. When used as numbers in an

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-20 Thread Steve Armstrong
4223 Mobile: 0404 998 273 *From:* cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] *On Behalf Of *Gareth Edwards *Sent:* Tuesday, 20 February 2007 2:40 PM *To:* cfaussie@googlegroups.com *Subject:* [cfaussie] Re: Strange calculation behavior on CF 7.02 Indicating you know how

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-20 Thread Robin Hilliard
Hi Rod, Big Decimal has far more precision than a CF number (only 12 significant digits) and has built in rounding - you can set the precision with it's setScale(scale, rounding-mode) method. One point to note if you are using BigDecimal - you should use the string constructor i.e.

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-20 Thread Andrew Scott
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Steve Armstrong Sent: Tuesday, 20 February 2007 8:47 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Andrew, To say that it has nothing to do with ColdFusion is incorrect

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-20 Thread Andrew Scott
Hmm, interesting you used to be able to select that option in previous versionJ Andrew Scott Senior Coldfusion Developer Aegeon Pty. Ltd. www.aegeon.com.au Phone: +613 8676 4223 Mobile: 0404 998 273 --~--~-~--~~~---~--~~ You received this message because you

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-20 Thread Robin Hilliard
What exactly is a 'CF number'? All simple CF variables are stored as strings. When used as numbers in an expression they have 12 significant digits according to the documentation, which I assume is a java Float (a Double would have more). As for a solution in CFML, I wouldn't mind an

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-20 Thread Gareth Edwards
Andrew, What option? Cheers Gareth. Andrew Scott wrote: Hmm, interesting you used to be able to select that option in previous versionJ Andrew Scott Senior Coldfusion Developer Aegeon Pty. Ltd. www.aegeon.com.au Phone:+613 8676 4223 Mobile: 0404 998 273

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-20 Thread Andrew Scott
February 2007 1:41 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Andrew, What option? Cheers Gareth. Andrew Scott wrote: Hmm, interesting you used to be able to select that option in previous versionJ Andrew Scott Senior Coldfusion Developer

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Barry Beattie
not having a working CF server handy at the moment to test, I'm just curious you're not getting grief with treating strings as numbers, are you? perhaps a val( #ListFirst(Trim(a),.)#.00 ) is needed? just a quick thought b On 2/20/07, Gareth Edwards [EMAIL PROTECTED] wrote: If you are

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
If it were a string, and it was having troubles the end result should be consistent. Check this out. 9314-18628+9314.1 = 0.1 4730-9460+4730.1 = 0.1 9913-19826+9913.1 = 0.1 3913-7826+3913.1 = 0.0 6770-13540+6770.1 = 0.1 3000-6000+3000.1 = 0.0 Cheers Gareth. Barry

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
Just an update... the new code below does the same thing. cfset a = 0 cfset b = 0 cfset c = 0 cfloop from="1" to="1000" index="i" cfset a = RandRange(100,1) cfset b = a*2 cfset c = val("#ListFirst(Trim(a),".")#.00") cfset c = c + 0.1 cfset d = a - b + c cfoutput#a#-#b#+#c# = #d#br

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Ryan Sabir
They looks like pretty standard floating point rounding errors. You get them in any programming language where you deal with fractions. The trick is to expect that this will happen and plan your application around it. -Original Message- From: cfaussie@googlegroups.com

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
It hasn't been a problem previously, why would it be a problem now? CFML is not a strong typed language and there for should evaluate situations like this correctly. Cheers Gareth. Ryan Sabir wrote: They looks like pretty standard floating point rounding errors. You get them in any

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Ryan Sabir
[mailto:[EMAIL PROTECTED] On Behalf Of Gareth Edwards Sent: Tuesday, 20 February 2007 12:37 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 It hasn't been a problem previously, why would it be a problem now? CFML is not a strong typed language

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
] Re: Strange calculation behavior on CF 7.02 It hasn't been a problem previously, why would it be a problem now? CFML is not a strong typed language and there for should evaluate situations like this correctly. Cheers Gareth. Ryan Sabir wrote: They looks like pretty

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Brett Payne-Rhodes
*Subject:* [cfaussie] Re: Strange calculation behavior on CF 7.02 It hasn't been a problem previously, why would it be a problem now? CFML is not a strong typed language and there for should evaluate situations like this correctly. Cheers Gareth. Ryan Sabir wrote

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Ryan Sabir
PROTECTED] On Behalf Of Gareth Edwards Sent: Tuesday, 20 February 2007 12:53 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 What about 7.01 or 7.00? or 6.1 or 6? CF5 is not a fair comparison. Cheers Gareth. Ryan Sabir wrote: I just tried

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
the rounding errors, then it won't matter. From: cfaussie@googlegroups.com [mailto:cfaussie@googlegroups.com] On Behalf Of Gareth Edwards Sent: Tuesday, 20 February 2007 12:53 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
: [cfaussie] Re: Strange calculation behavior on CF 7.02 So why the mixed result? Cheers Gareth. Ryan Sabir wrote: Just remembered I had a 6.1 server lying around. And yes it did the same thing: 0.1 0.1 0.1 0.09985 0.09985 0.1

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Andrew Scott
[mailto:[EMAIL PROTECTED] On Behalf Of Gareth Edwards Sent: Tuesday, 20 February 2007 1:42 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Now you show me in the Coldfusion Documentation where this is outlined as a known issue? Cheers Gareth

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
Of Gareth Edwards Sent: Tuesday, 20 February 2007 1:42 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Now you show me in the Coldfusion Documentation where this is outlined as a known issue? Cheers Gareth. Ryan Sabir wrote

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Ryan Sabir
] On Behalf Of Gareth Edwards Sent: Tuesday, 20 February 2007 1:42 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Now you show me in the Coldfusion Documentation where this is outlined as a known issue? Cheers Gareth. Ryan Sabir wrote: Doesn't

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
groups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Now you show me in the Coldfusion Documentation where this is outlined as a known issue? Cheers Gareth. Ryan Sabir wrote: Doesn't really matter why... Maybe there's a gremlin

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Andrew Scott
. www.aegeon.com.au Phone: +613 8676 4223 Mobile: 0404 998 273 From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Gareth Edwards Sent: Tuesday, 20 February 2007 2:14 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Who's missing my point

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
o:cfaussie@googlegroups.com] On Behalf Of Gareth Edwards Sent: Tuesday, 20 February 2007 2:14 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 Who's missing my point? One shouldn't have to worry about typing in Coldfusion. If in an evaluation

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Robin Hilliard
It's easier to see the pattern without the random stuff and the extra additions and subtractions, note val() etc have nothing to do with it: cfloop from=1 to=1 index=a cfset c = a + 0.1 cfset d = a - c cfoutput#a# - #c# = #d#br//cfoutput /cfloop 1019 - 1019.1 = -0.1 1020 -

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Robin Hilliard
Actually check out the last example in the Addition section of this article: http://en.wikipedia.org/wiki/Floating_point This is what we are seeing subtracting two similar numbers - the result of the subtraction winds up with a lot of leading 0s - then the exponent is adjusted to

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Gareth Edwards
Sorry to complain about it, it just seems strange, I won't complain anymore. Does it do the same thing in Actionscript? (don't have time to check right now) What do you think is the best way to handle this in coldfusion? Also, I'm curious, how many people on this list are aware of this

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Barry Beattie
to part answer that, what were you doing to bring this matter to your attention? On 2/20/07, Gareth Edwards [EMAIL PROTECTED] wrote: Sorry to complain about it, it just seems strange, I won't complain anymore. Does it do the same thing in Actionscript? (don't have time to check right now)

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread skateboard.com.au
2007 14:27:34 +1000 Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 !DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01 Transitional//EN html head meta content=text/html;charset=ISO-8859-1 http-equiv=Content-Type /head body bgcolor=#ff text=#00 Sorry to complain about

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Andrew Scott
Restrict the decimal paces and force a round off! Andrew Scott Senior Coldfusion Developer Aegeon Pty. Ltd. www.aegeon.com.au Phone: +613 8676 4223 Mobile: 0404 998 273 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread Steve Onnis
Hilliard Sent: Tuesday, 20 February 2007 3:54 PM To: cfaussie@googlegroups.com Subject: [cfaussie] Re: Strange calculation behavior on CF 7.02 On 20/02/2007, at 3:27 PM, Gareth Edwards wrote: Sorry to complain about it, Don't worry, you were somewhat provoked over the course of the thread (I

[cfaussie] Re: Strange calculation behavior on CF 7.02

2007-02-19 Thread rod higgins
If you use the java class and methods that handle these types of numbers I don't see the problem?? cfloop from=1 to=1000 index=i cfset diff = CreateObject(Java, java.math.BigDecimal).init(0.1) cfset a = CreateObject(Java, java.math.BigDecimal).init(RandRange(100,1))