On 2 February 2012 12:16, Victor Stinner <victor.stin...@haypocalc.com> wrote:
> Let's take an NTP timestamp in format (c): (sec=0,
> floatpart=100000000, divisor=2**32):
>
>>>> Decimal(100000000) * Decimal(10)**-10
> Decimal('0.0100000000')
>>>> Decimal(100000000) / Decimal(2)**32
> Decimal('0.023283064365386962890625')
>
> You have an error of 57%. Or do you mean that not only 2**32 should be
> modified, but also 100000000? How do you adapt 100000000 (floatpart)
> when changing the divisor (2**32 => 10**-10)? The format (c) avoids an
> operation (base^exponent) and avoids loosing precision.Am I missing something? If you're using the fixed point form (fraction, exponent) then 0.023283064365386962890625 would be written as (23283064365386962890625, -23). Same precision as the (100000000, base=2, exponent=32) format. Confused, Paul _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
