David, It all depends on what method are you using: Math.Round or Math.Truncate. Round rounds the double to the nearest integer. So
Math.Round( 13788.999 ) will return 13789 as opposed to Math.Truncate who will return 13788. So far so good. The problem will appear when the number is 13788.5. This will be rounded to the nearest even number so Math.Round will return 13788 (incorrect). Hopefully this will never happen, because as some of you suggested, that means that the initial amount of $137.89 is stored in a double as 137.885000, off by 0.005, a very big precision error. My point is that adding anything will not help me because I can introduce the error. Just for the sake of argument, let's assume that a precision error of 0.005 can happen. If that, then an error of 0.0045 can happen too. If my amount of $137.89 is actually stored as $137.8945 and I add 0.001 I get 137.8955 which will round up to 137.90 - wrong number. We have already established that Math.Truncate is not good so I am only left with Math.Round and with the hope that a precision error of 0.005 is too big to actually happen. Thanks to everyone who chimed in and shared experience. Best regards, Eddie -----Original Message----- From: Discussion of advanced .NET topics. [mailto:[EMAIL PROTECTED] Behalf Of David Nicholson Sent: Wednesday, August 27, 2008 8:04 PM To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM Subject: Re: [ADVANCED-DOTNET] Converting doubles into integers without rounding errors Since there seems to be no clear solution so far, I've gone back to the beginning. I suggest you get more information by writing some code to display your numbers to higher precision. My expectation is that the 137.89 is actually 137.889999 (probably more nine's). So your code below converts this to 13788.9999, then truncates to 137.88. So add a small value (e.g. .0001) first. Then the outcomes for numbers above and below the intended number look something like: 137.889999 -> 137.890099 -> 13789.0099 -> 13789 137.890000 -> 137.890100 -> 13789.0100 -> 13789 137.890001 -> 137.890101 -> 13789.0101 -> 13789 David. On Mon, 25 Aug 2008 12:27:52 -0400, Eddie Lascu <[EMAIL PROTECTED]> wrote: >Hello everyone, > >I have some objects that contain an amount field that is declared as double. >Since it contains amounts, it always has only two decimal digits that are >significant. During the process I need to convert that double into an >integer by removing the decimal point. For example, $78.59 should be >converted to integer 7859 and $101.53 to 10153. in my code I have >uint nIntAmount = (uint)(objMyObject.Amount * 100); > >The problem I am facing is that sometimes, very rarely, there is a rounding >error that is introduced and the integer obtained is off by a cent (plus or >minus). For example, this is a line that was traced in my log file: > >"Updating the batch with $137.89 as the amount in the transaction. This >amount was converted to 13788." > >Can either of you suggest a different way to convert the amounts in integers >without this nagging rounding error? > >Any help will be appreciated, > >Eddie > >=================================== >This list is hosted by DevelopMentor® http://www.develop.com > >View archives and manage your subscription(s) at http://discuss.develop.com =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
