Geneva,
This isn't a problem. The same behavior exists in C, Pascal, or most any
other programming language. It is caused by the limitations of representing
floating point numbers using a binary number. Floating point numbers should
not be tested for equality. If you need the result of your example to be
exactly 2.05, then you must add some code to round the number to the 0.01
decimal place.
Here is some code that trims a double so that it does not have values beyond
the 4th decimal.
/*
* trimDouble trims a double value to two significant digits to the
right of the
* decimal place.
* @param d the value to trim
* @return the trimmed value. Returns a double rounded to the nearest
0.0001
* @author Richard Yee
*/
protected double trimDouble(double d)
{
double dValue;
long lValue;
if (d < 0.00001)
return 0.0;
// round up if >= .00005, round down if less
lValue = (long) ((d + 0.00005) * 10000);
dValue = lValue / 10000.00;
return dValue;
}
If you are dealing w/ dollar amounts, change the constants accordingly.
-Richard Yee
-----Original Message-----
From: Davis, Geneva [mailto:[EMAIL PROTECTED]]
Sent: Thursday, April 20, 2000 8:49 AM
To: [EMAIL PROTECTED]
Subject: Java Division Problem
Sorry if this isn't the right place to post this question. Has anyone run
into a problem when dividing numbers where you end up losing precision in
your result, or in effect, get an incorrect result? For example 2.050/100
results in something like 0.0204999997.
Thank You,
Geneva
===========================================================================
To unsubscribe: mailto [EMAIL PROTECTED] with body: "signoff
JSP-INTEREST".
Some relevant FAQs on JSP/Servlets can be found at:
http://java.sun.com/products/jsp/faq.html
http://www.esperanto.org.nz/jsp/jspfaq.html
http://www.jguru.com/jguru/faq/faqpage.jsp?name=JSP
http://www.jguru.com/jguru/faq/faqpage.jsp?name=Servlets
===========================================================================
To unsubscribe: mailto [EMAIL PROTECTED] with body: "signoff JSP-INTEREST".
Some relevant FAQs on JSP/Servlets can be found at:
http://java.sun.com/products/jsp/faq.html
http://www.esperanto.org.nz/jsp/jspfaq.html
http://www.jguru.com/jguru/faq/faqpage.jsp?name=JSP
http://www.jguru.com/jguru/faq/faqpage.jsp?name=Servlets
