In news:[email protected],
Don Daugherty <[email protected]> typed:
On 3/3/2010 1:36 PM, AG wrote:
JOE Conner wrote:
On 3/3/2010 10:52 AM, AG wrote:
11.73 + 7 + 15.75 + 24.68 + 17.5 + 21.18 + 20.65 + 17.85
+ 19.25 + 23.1 + 12.6 + 10.5 + 23.8
TOOLS -> OPTIONS -> OPENOFFICE.ORG CALC -> CALCULATE ->
Right Panel, do you have PRECISION AS SHOWN checked on
one of the machines? Joe Conner, Poulsbo, WA USA
------------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Joe
Cheers for the rapid response. Neither of our machines
had that option enabled. However, even after enabling the
"precision as shown" the error persists on my partner's
version of Calc. Thanks
AG
From my reading of Help, you wouldn't want that option
enabled anyway since it would tend to result in more
roundoff's. I'm thinking that JOE wasn't recommending it
as a cure but as the possible problem.
I know when messing with Floating Point math in about any program, you can
get results similar to the original one stated here. Excel seems to use
double-precision no matter what you display and usually avoids this issue
but it can still be made to happen even in Excel. I'ts very easy to
duplicate in most development languages too if you aren't watching your
precision.
Apparently, and I don't know this for a fact, Calc only uses the displayed
digits for its math from a couple quick tests I just did. Somehow it seems
to be 4 places of precision for calculations unless you program it
differently.
Personally I haven't encountered such a problem in Calc, but I think
that's because I have a habit of always using at least two positions more of
precision than is necessary, three if they aren't ridiculous numbers. There
is a diminishing point of no return when you for instance set pi to be 3.14
or 3.14xx or 3.14xxxxx, and so forth.
It's a complex subject for program authors and one that results in many
debates around some circles about which way to do them. Even my invoices,
which need two decimal places of precision, are calculated on 5 decimal
places, but ... use the standard currency formatting of only two decimal
places. No one wants a bill for $133.3333. But when I'm multiplying $5.35
by 10,013 pcs, the differences between that result and using only two places
amounts to a lot of money at the end of the year over all the invoices. It
comes out more accurate for me and more accurate for the customer too.
Dunno if that makes any sense; it's the best I can do right now though so I
hope it helps someone out.
HTH,
Twayne`
--
Newsgroups are great places to get assistance.
But always verify important information with
other sources to be certain you have a clear
understanding of it and that it is accurate.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
