Thank you for your help. Your response and the sample code are greatly
appreciated!
> -----Original Message-----
> From: Richard Yee [SMTP:[EMAIL PROTECTED]]
> Sent: Thursday, April 20, 2000 11:01 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Java Division Problem
>
> Geneva,
>
> This isn't a problem. The same behavior exists in C, Pascal, or most any
> other programming language. It is caused by the limitations of
> representing
> floating point numbers using a binary number. Floating point numbers
> should
> not be tested for equality. If you need the result of your example to be
> exactly 2.05, then you must add some code to round the number to the 0.01
> decimal place.
>
> Here is some code that trims a double so that it does not have values
> beyond
> the 4th decimal.
>
> /*
> * trimDouble trims a double value to two significant digits to the
> right of the
> * decimal place.
> * @param d the value to trim
> * @return the trimmed value. Returns a double rounded to the nearest
> 0.0001
> * @author Richard Yee
> */
> protected double trimDouble(double d)
> {
> double dValue;
> long lValue;
> if (d < 0.00001)
> return 0.0;
> // round up if >= .00005, round down if less
> lValue = (long) ((d + 0.00005) * 10000);
> dValue = lValue / 10000.00;
> return dValue;
> }
>
> If you are dealing w/ dollar amounts, change the constants accordingly.
>
> -Richard Yee
>
> -----Original Message-----
> From: Davis, Geneva [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, April 20, 2000 8:49 AM
> To: [EMAIL PROTECTED]
> Subject: Java Division Problem
>
>
> Sorry if this isn't the right place to post this question. Has anyone run
> into a problem when dividing numbers where you end up losing precision in
> your result, or in effect, get an incorrect result? For example 2.050/100
> results in something like 0.0204999997.
>
> Thank You,
> Geneva
>
> ==========================================================================
> =
> To unsubscribe: mailto [EMAIL PROTECTED] with body: "signoff
> JSP-INTEREST".
> Some relevant FAQs on JSP/Servlets can be found at:
>
> http://java.sun.com/products/jsp/faq.html
> http://www.esperanto.org.nz/jsp/jspfaq.html
> http://www.jguru.com/jguru/faq/faqpage.jsp?name=JSP
> http://www.jguru.com/jguru/faq/faqpage.jsp?name=Servlets
>
> ==========================================================================
> =
> To unsubscribe: mailto [EMAIL PROTECTED] with body: "signoff
> JSP-INTEREST".
> Some relevant FAQs on JSP/Servlets can be found at:
>
> http://java.sun.com/products/jsp/faq.html
> http://www.esperanto.org.nz/jsp/jspfaq.html
> http://www.jguru.com/jguru/faq/faqpage.jsp?name=JSP
> http://www.jguru.com/jguru/faq/faqpage.jsp?name=Servlets
===========================================================================
To unsubscribe: mailto [EMAIL PROTECTED] with body: "signoff JSP-INTEREST".
Some relevant FAQs on JSP/Servlets can be found at:
http://java.sun.com/products/jsp/faq.html
http://www.esperanto.org.nz/jsp/jspfaq.html
http://www.jguru.com/jguru/faq/faqpage.jsp?name=JSP
http://www.jguru.com/jguru/faq/faqpage.jsp?name=Servlets
