The problem is that you should have wrapped 123.45678 in quotes
when you assigned it to 'a'. In this lazy modern world of variant
data types, you really need to explicitly declare types when
assigning variables, or else instead of crashing with an overflow
or underflow, this particular compiler acts like Candide, and
gives you the best of all possible worlds. In other words, it's
pleased to hand you the wrong answer when the right one isn't
completely spelled out.
Try this:
dim a as string
a = 123.45678
print Format$(val(a), "###.######")
(this results in 123.457, as you saw)
a = "123.45678"
print Format$(val(a), "###.######")
(this results in 123.45678 as desired.)
This sort of thing that you discovered can also appear in a much
more subtle and dangerous form (from a programmer's point of
view) when one uses text edit controls in dialogs. If you ask the
user to fill in a float variable appearing in a dialog box, you
should first convert it to a string with all its precision using
'Format$()' or you will lose the full precision in the edit
control.
Using mapbasic, you can prove to yourself this is so. Just run
the following and look what it does to the number!
dim a as float
a = -123.456789
dialog title "float test"
control edittext width 100
value a into a
control OKbutton
note "a is now " + format$(a, "###.######")
If the user alters the value in the dialog box, it comes out all
right, but going into the textedit control, the precision is
stripped just as you found out when converting a number into a
string without explicitly converting the number to a string
first.
Just imagine how many dialog boxes are out there where a lat or a
lon value gets presented to a user already snipped!
Another interesting thing about MapInfo data types is how it
stores a 'decimal' type in a table. Looks to me like these are
just stored as strings in the table, but interpreted as floats
internally. It may seem like a neat way to avoid the negative
range of large signed integer types, but if you want to do that,
use an 8-byte float type instead of wasting the 10 bytes of a
decimal type to store the full range of an unsigned integer (0 to
2^31-1).
A decimal type might also seem like a clever way to store
extremely large or small numbers, but I think the only way you
can do this is to actually code your high-precision numbers as
strings and manipulate them yourself. For example, create a table
with a field called 'Id' as decimal (20,0), then fill it with
'12345678901234567890' and look what happens. The number remains
20 digits long, but the significant digits are dropped to 17 (16
is the limit of a double precision IEEE floating point number, so
I think this extra digit just got lucky and isn't really there).
Also, if you look in the *.dat file of the table, there's your
number staring back out at you occupying 20 bytes of disk space,
but it's really only containing 8 bytes of information.
The only possible value a decimal type might have is to make
browsers look pretty. And even then, the table's owner had better
take care that these values remain within the limits of
double-precision floating point numbers.
- Bill Thoen
----------------------------------------------------------------------
To unsubscribe from this list, send e-mail to [EMAIL PROTECTED] and put
"unsubscribe MAPINFO-L" in the message body, or contact [EMAIL PROTECTED]
