[EMAIL PROTECTED] wrote:
T&B <[EMAIL PROTECTED]> wrote:
select cast( 141.70 * 100 as integer)
gives the incorrect 14169
14169 is the correct answer, believe it or not. There is
no such thing as the number 141.70 in a 64-bit IEEE float.
The closest you can get is
141.69
T&B <[EMAIL PROTECTED]> wrote:
>
> >> select cast( 141.70 * 100 as integer)
>
> gives the incorrect 14169
14169 is the correct answer, believe it or not. There is
no such thing as the number 141.70 in a 64-bit IEEE float.
The closest you can get is
141.6886313162278383970260620
Hi Mike,
well, first of all you must have a typo, since the sql you show will
return 14170, not 0.69 or anything like it.
Yes, typo, sorry. The short version returns 14169 but should give
14170, as per my correction a minute ago.
however, one thing springs out:
Total * 100 + 100 is wrong
oops, sorry, slight correction:
When I try:
select cast( 141.70 * 100 as integer)
I get 14169, but should get 14170
Tom
-
To unsubscribe, send email to [EMAIL PROTECTED]
Cariotoglou Mike wrote:
Total * 100 + 100 is wrong IMHO, unless you are looking for CEILING
functionallity.
"round" would need : total * 100 +50 (which rounds to nearest integer at
two decimal points, not to the nearest LARGER integer,
which is what your sample does)
Shouldn't that be
t
to nearest integer at
two decimal points, not to the nearest LARGER integer,
which is what your sample does)
> -Original Message-
> From: T&B [mailto:[EMAIL PROTECTED]
> Sent: Thursday, December 13, 2007 4:27 PM
> To: sqlite-users@sqlite.org
> Subject: [sqlite] Casting bug
T&B wrote:
When I try:
select cast( 141.70 * 100 as integer)
I get 0.69, but should get 0.70
Assuming you mean you got 14169, maybe you should look at
http://sqlite.org/faq.html#q16
HTH,
Gerry
-
To unsubscribe,
When I try:
select cast( 141.70 * 100 as integer)
I get 0.69, but should get 0.70
What's the problem? Seems like a bug.
I tried some other numbers in place of 141.70, and they worked OK,
though I imagine there are others that have the bug that I just
haven't tried.
The above is the isola
8 matches
Mail list logo
