Piers,
Thanks for the insight and detailed explanation.
Barry Rogen
PNY Technologies, Inc.
Senior Programmer/Analyst
(973) 515 - 9700 ext 5327
[EMAIL PROTECTED]
-
Far better it is to dare mighty things, to win
glorious triumphs even though checkered by
failure, than to rank with those poor spirits who
neither enjoy nor suffer much because they live
in the gray twilight that knows neither victory
nor defeat
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Piers Angliss
Sent: Friday, January 06, 2006 5:35 PM
To: u2-users@listserver.u2ug.org
Subject: [Potentially SPAM Header Checking] - RE: [U2] [UV] Totaling
oddity on UniVerse - Email has different SMTP TO: and MIME TO: fields in
the email addresses
Barry,
Search the archives for anything on wide-zero for detailed explanation,
but
I'll have a go anyway.
It is all to do with binary representation of floating point numbers. If
you
convert an integer from decimal to binary the resultant value is always
precise and unambiguous, the binary can always be converted back to
decimal
with the correct result. If you try this with a floating point number
that
isn't always true, a small percentage of decimal values between 0 and 1
do
not have a precise binary representation, you can get close but you
can't
get there exactly.
So, if you perform arithmetic operations on these binary representations
you
stand the chance of getting very small inaccuracies, or rounding errors.
Many programming environments (including Unidata) will resolve these for
you - I'm not sure how (help me Ken ?) but possibly by converting fp
values
to integers first, then calculating, then converting back
Universe performs the calculation on the fp numbers themselves and uses
a
tunable parameter (WIDE-ZERO) to allow you to specify the point at which
very small values can be considered equivalent to zero. The default
value is
an industry-standard (IEEE) value of 2.91E-11 which is roughly 1 / 2 pwr
35.
By totalling values in external (OCONV) format in Universe you will
occasionally fall foul of this and find that the total is "out" by a
small
amount - although 0.05 is larger than I would expect, may depend on
the
number of values you are totalling.
You can "tune" your system by tweaking the value of the wide-zero
parameter
(default 0x3dc0) or, as recommended by many gurus on this list, you
should always perform arithmetic on the internal (ICONV) values so if
you
modify your F55 definition to ICONV the value with an MD2 conversion
then I
think you'll find the total to be as expected ( but 100 times too large)
Hth
Piers
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of Barry Rogen
The output I get is quite bizarre. Instead of the expected value
of
134878167.90I get 134878167.890005
Any thoughts ???
Barry Rogen
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
NOT INTENDED AS A SUBSTITUTE FOR A WRITING
NOTHING IN THIS E-MAIL, IN ANY E-MAIL THREAD OF WHICH IT MAY BE A PART, OR IN
ANY ATTACHMENTS THERETO, SHALL CONSTITUTE A BINDING CONTRACT, OR ANY
CONTRACTUAL OBLIGATION BY PNY, OR ANY INTENT TO ENTER INTO ANY BINDING
OBLIGATIONS, NOTWITHSTANDING ANY ENACTMENT OF THE UNIFORM ELECTRONIC
TRANSACTIONS ACT, THE FEDERAL E-SIGN ACT, OR ANY OTHER STATE OR FEDERAL LAW OF
SIMILAR SUBSTANCE OR EFFECT. THIS EMAIL MESSAGE, ITS CONTENTS AND ATTACHMENTS
ARE NOT INTENDED TO REPRESENT AN OFFER OR ACCEPTANCE OF AN OFFER TO ENTER INTO
A CONTRACT. NOTHING IN THIS E-MAIL, IN ANY E-MAIL THREAD OF WHICH IT MAY BE A
PART, OR IN ANY ATTACHMENTS THERETO SHALL ALTER THIS DISCLAIMER.
This e-mail message from PNY Technologies, Inc. is for the sole use of the
intended recipient(s) and may contain confidential and privileged information.
Any unauthorized review, use, disclosure or distribution is prohibited. If you
are not the intended recipient, please contact the sender by reply e-mail and
destroy all copies of the original message.
---
u2-users mailing list
u2-users@listserver.u2ug.org
To unsubscribe please visit http://listserver.u2ug.org/
