>
> Gives 636 as expected. Then it starts getting confusing:
>
> Frac(AOverNotation) gives 0.2 (As expected)
> Frac(AOverNotation) * 10 gives 2 (As expected)
> Trunc(Frac(AOverNotation) * 10) gives 1 !!!!! Why is that?
>
> Can someone point me in the right direction here please?
Ah, the joys of floating point arithmetic. I think you will find that
Frac(AOverNotation) * 10 isn't actually giving 2. It will be returning
1.999999999784 for example. Your Trunc call then correctly notices that this
value is less than 2 and returns 1.
There are probably many ways you could solve this problem. One of the
simplest is to replace Trunc(Frac(AOverNotation) * 10) with
Trunc(Frac(AOverNotation) * 10 + 0.000001). If your variables are
sufficiently high precision (double's work well) then this should be
sufficient to give you the correct result without any errors resulting from
adding the small offset of 0.000001).
David.
---------------------------------------------------------------------------
New Zealand Delphi Users group - Delphi List - [EMAIL PROTECTED]
Website: http://www.delphi.org.nz
To UnSub, send email to: [EMAIL PROTECTED]
with body of "unsubscribe delphi"
