.005 is ALOT for a double to be off ... usually its more like .0000000001 off
On Mon, Aug 25, 2008 at 10:49 AM, Eddie Lascu <[EMAIL PROTECTED]> wrote: > The problem is that I have to multiply by 100 before rounding, so the double > must no be off by more than 0.005. For example, if 4.56 is actually stored > as 4.5610, the number multiplied by 100 will be 456.100 which gets to > 456.600 when adding 0.5 which rounds up to 457. > > I feel like my brain is about to explode... > > > -----Original Message----- > From: Discussion of advanced .NET topics. > [mailto:[EMAIL PROTECTED] Behalf Of Terry Griffin > Sent: Monday, August 25, 2008 1:27 PM > To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > Subject: Re: [ADVANCED-DOTNET] Converting doubles into integers without > rounding errors > > > The problem with going from a double to an int is that the final digit can > be cut off because the double is inexact. But the double will be close. > All you need to do is add 0.5 to the value before doing the conversion to > an int. The "extra" 0.5 will make a value like 4.999999999 get converted > to 5 instead of 4. Unless your original numbers are off by a large amount > (close to 0.5) this won't introdcue any error. > > int intVal; > double dblVal; > > intVal = (int)(dbleVal * 100 + 0.5); > > > > Terry Griffin > Sr. Software Engineer > Carl Zeiss SMT Inc. > ALIS Business Unit > 1 Corporation Way > Peabody, MA 01960 USA > [EMAIL PROTECTED] > (978) 826-1569 > > > > > Daniel Barla-Szabo <[EMAIL PROTECTED]> > Sent by: "Discussion of advanced .NET topics." > <ADVANCED-DOTNET@DISCUSS.DEVELOP.COM> > 08/25/2008 01:10 PM > Please respond to > "Discussion of advanced .NET topics." > <ADVANCED-DOTNET@DISCUSS.DEVELOP.COM> > > > To > ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > cc > > Subject > Re: [ADVANCED-DOTNET] Converting doubles into integers without rounding > errors > > > > > > > Personally, I've never used it before, but I see in framework 2.0 there is > the option of: > > (int) Double.Truncate(objMyObject.Amount * 100); > > HTH > > -----Original Message----- > From: Discussion of advanced .NET topics. > [mailto:[EMAIL PROTECTED] On Behalf Of Eddie Lascu > Sent: 25 August 2008 06:28 PM > To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > Subject: [ADVANCED-DOTNET] Converting doubles into integers without > rounding > errors > > Hello everyone, > > I have some objects that contain an amount field that is declared as > double. > Since it contains amounts, it always has only two decimal digits that are > significant. During the process I need to convert that double into an > integer by removing the decimal point. For example, $78.59 should be > converted to integer 7859 and $101.53 to 10153. in my code I have > uint nIntAmount = (uint)(objMyObject.Amount * 100); > > The problem I am facing is that sometimes, very rarely, there is a > rounding > error that is introduced and the integer obtained is off by a cent (plus > or > minus). For example, this is a line that was traced in my log file: > > "Updating the batch with $137.89 as the amount in the transaction. This > amount was converted to 13788." > > Can either of you suggest a different way to convert the amounts in > integers > without this nagging rounding error? > > Any help will be appreciated, > > Eddie > > =================================== > This list is hosted by DevelopMentorR http://www.develop.com > > View archives and manage your subscription(s) at > http://discuss.develop.com > > =================================== > This list is hosted by DevelopMentor(R) http://www.develop.com > > View archives and manage your subscription(s) at > http://discuss.develop.com > > > > > ---------------------------------------- > This message is intended for a particular addressee only and may contain > business or company secrets. If you have received this email in error, > please contact the sender and delete the message immediately. Any use of > this email, including saving, publishing, copying, replication or forwarding > of the message or the contents is not permitted. > > =================================== > This list is hosted by DevelopMentor(R) http://www.develop.com > > View archives and manage your subscription(s) at http://discuss.develop.com > > =================================== > This list is hosted by DevelopMentor(R) http://www.develop.com > > View archives and manage your subscription(s) at http://discuss.develop.com > -- It is the mark of an educated mind to be able to entertain a thought without accepting it. =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
