I think I would write a UDF that would round all of your floating point calculations. Don't forget that you have to worry about +- 0.00000000001 as well as +- 0.009999999999991
Try playing around with this <cfset val=-0.0000999991> <cfset decimal=3> Make this part a UDF with val and decimal as arguments. <cfset multiplier=10^decimal> <cfset newval=round(multiplier*val)/multiplier> Output or return this new value <cfoutput>#newval#</cfoutput> ------------------------------------ Gerry Gurevich Application Development NIEHS ITSS Contractor Lockheed Martin Information Technology 919-361-5444 ext 311 ________________________________ From: Howard Fore [mailto:[EMAIL PROTECTED] Sent: Friday, December 28, 2007 12:55 PM To: [email protected] Subject: Re: [ACFUG Discuss] Negative plus positive equals less than zero? I may try that. My hackish solution was that if the resulting number was less than 0 but greater than -0.01 to just return zero. I'm dealing with money amounts here with no partial cents so I'm not really losing any significant data. Decimalformat takes care of this but I need to have the negative sign to the right of the number (client request) so I have to use numberFormat instead, Thanks. On 12/28/07, Cameron Childress <[EMAIL PROTECTED]> wrote: I ran into this a number of times on a eCom system I was working on recently. Gerry is right in that it's just a really small floating point number. I've seen some financial systems deal with this problem by shifting the decimal when dealing with numbers and shifting it back only for display. In that case. For example 62355.57 would be stored as 6235557 or even as 623555700 in financial systems that use 4 decimal points. You'd conduct all math on these whole numbers and avoid floating point problems and then just x/100 or x/10000 to get it back to a decimal format (maybe in a UDF) for display. Another more hackish solution I've seen is to numberFormat() all inputs to any math operation, which will strip off all the extra float and give you a straighter answer. I'm not sure I would actually suggest this cause it makes your code look like crap - but if you do it the problem goes away. Try this out and see if your problem goes away... <cfscript> total = 0; numberList = "62355.57,-62355.57,-333.01,261.09 ,17.98,35.96,17.98"; for (i = 1;i lte listLen(numberList);i = i + 1) { writeOutput("#total# + #ListGetAt(numberList,i)# = "); total = numberFormat(total,"999999999.99") + numberFormat(ListGetAt(numberList,i),"999999999.99"); writeOutput("#total#<br>"); } writeOutput('numberFormat(total,"999,999,999.99") = ' & numberFormat(total,"999,999,999.99")); </cfscript> -Cameron On Dec 28, 2007 11:20 AM, Howard Fore < [EMAIL PROTECTED]> wrote: > Hey, > > Can someone confirm this seemingly odd behavior? If I execute the following > code: > > <cfscript> > > total = 0; > numberList = "62355.57,-62355.57,-333.01,261.09 ,17.98,35.96,17.98"; > for (i = 1;i lte listLen(numberList);i = i + 1) > { > writeOutput("#total# + #ListGetAt(numberList,i)# = "); > total = total + ListGetAt(numberList,i); > writeOutput("#total#<br>"); > > } > writeOutput('numberFormat(total,"999,999,999.99") = ' & > numberFormat(total,"999,999, 999.99")); > </cfscript> > > I get the following output: > > 0 + 62355.57 = 62355.57 > 62355.57 + -62355.57 = 0 > 0 + -333.01 = -333.01 > -333.01 + 261.09 = -71.92 > -71.92 + 17.98 = -53.94 > -53.94 + 35.96 = -17.98 > -17.98 + 17.98 = -1.06581410364E-014 > numberFormat(total,"999,999, 999.99") = -0.00 > > I've tried using javacasts to make sure that there wasn't some odd string to > number conversion thing going on too and still got the same result. Any > ideas? > > -- > Howard Fore, [EMAIL PROTECTED] > "Whether you believe you can do a thing or not, you are right." -- Henry > Ford > ------------------------------------------------------------- > Annual Sponsor - Figleaf Software > > To unsubscribe from this list, manage your profile @ > http://www.acfug.org?fa=login.edituserform > > For more info, see http://www.acfug.org/mailinglists > Archive @ http://www.mail-archive.com/discussion%40acfug.org/ > List hosted by FusionLink > ------------------------------------------------------------- -- Cameron Childress Sumo Consulting Inc http://www.sumoc.com --- cell: 678.637.5072 aim: cameroncf email: [EMAIL PROTECTED] ------------------------------------------------------------- Annual Sponsor FigLeaf Software - http://www.figleaf.com To unsubscribe from this list, manage your profile @ http://www.acfug.org?fa=login.edituserform For more info, see http://www.acfug.org/mailinglists Archive @ http://www.mail-archive.com/discussion%40acfug.org/ List hosted by http://www.fusionlink.com ------------------------------------------------------------- -- Howard Fore, [EMAIL PROTECTED] "Whether you believe you can do a thing or not, you are right." -- Henry Ford ------------------------------------------------------------- Annual Sponsor - Figleaf Software <http://www.figleaf.com> To unsubscribe from this list, manage your profile @ http://www.acfug.org?fa=login.edituserform For more info, see http://www.acfug.org/mailinglists Archive @ http://www.mail-archive.com/discussion%40acfug.org/ List hosted by FusionLink <http://www.fusionlink.com> ------------------------------------------------------------- ------------------------------------------------------------- Annual Sponsor FigLeaf Software - http://www.figleaf.com To unsubscribe from this list, manage your profile @ http://www.acfug.org?fa=login.edituserform For more info, see http://www.acfug.org/mailinglists Archive @ http://www.mail-archive.com/discussion%40acfug.org/ List hosted by http://www.fusionlink.com -------------------------------------------------------------
