Yo Hal! On Sun, 26 Mar 2017 19:04:22 -0700 Hal Murray <[email protected]> wrote:
> [email protected] said: > >> Isn't that as simple as convert a 64 bit integer to float and > >> divide by 1<<32 to get the right scale? > > Nope. Loss of precision. A double only has 52 bits of significant > > digits. A float only 23. To fully encode a 64 bit int you need a > > long double with (usually) 80 significant bits. > > Is there something I can say in python to get more precision? How > widely is it supported? You can use gmpy or numpy. Core Python is a bit sloppy on type precision. Pulling in either once would annoy a lot of minimalists. OTOH there are a lot of NUMPY fans. RGDS GARY --------------------------------------------------------------------------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 [email protected] Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can’t measure it, you can’t improve it." - Lord Kelvin
pgpfcAL1rKhDL.pgp
Description: OpenPGP digital signature
_______________________________________________ devel mailing list [email protected] http://lists.ntpsec.org/mailman/listinfo/devel
