Hi Richard, > Uhhh that doesnt sound right to me. I subtracted 3 different values > from another value and got the same result twice.
Oops, I didn't see your results were identical for the first two cases. I tried it here; 2156737.51 - 16123.81 => 2140613.6999999997 2156737.51 - 16123.83 => 2140613.6799999997 <-- Note: .6_7_999... 2156737.51 - 16123.82 => 2140613.69 Could you try it again to make sure you transcribed the values correctly? > Also two of the results are flat out wrong. Right, but they're pretty close to correct, which is what you get with floating-point, if I understand things correctly. > I could understand if I had used some kind of hokey notation but its > a straight up math problem X-Y. When performed in a calculator > application I dont get that result? The calculator (at least the one in Windows) probably doesn't use floating-point to store its numbers, but again, I'm a bit out of my depth here. Cheers, - Kim ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ To unsubscribe visit: https://lists.sourceforge.net/lists/listinfo/dqsd-users DQSD-Users@lists.sourceforge.net http://sourceforge.net/mailarchive/forum.php?forum_id=8601
