On 07.01.19 13:24, Peter P. wrote: > Hi, > > when I send the numer 0.7 to both inlets of a [* ] object (yes, in > correct order) the result is printed as 0.49 > If I now subtract [- 0.49] the result is -2.98023e-08 > While this is no major concern for me right now, I am still wondering > which step in this simple calculation is triggering this. If I remember > my CS lecture correctly it is the subtraction, right?
no. it's the representation of any floating point number. neither "0.7" nor "0.49" nor "x*x" (with x="0.7") can be represented *exactly* in a base-2 floating point representation. (honestly, i'm just making that up now without checking the representability of these exact numbers; however, i'm pretty sure it *is* valid though). because both numbers are inaccurately represented, they happen to be different. fgmasdr IOhannes
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
