Re: [Tutor] Unexpected result from decimal

[Orri Ganel]

Jacob S. wrote: Hi all,    I'm having a problem that is ticking me off. (to put it lightly) Why does decimal do this --  I thought that getcontext().prec was number of decimal places? import decimal decimal.getcontext().prec = 2 a = decimal.Decimal(2) b = decimal.Decimal(3) 100*a/b Decimal("67") print 100*a/b 67 Why does it do this? It's really, really, messing things up for me because results are not interpreted in my programs correctly. Jacob _______________________________________________ Tutor maillist  -  Tutor@python.org http://mail.python.org/mailman/listinfo/tutor Well, from looking at the documentation, decimal.getcontext().prec *tells* you the precision, so assigning to it is meaningless. Let's see what you have to do to actually change it: >>> a = decimal.Decimal(2) >>> b = decimal.Decimal(3) >>> 100*a/b Decimal("66.66666666666666666666666667") >>> decimal.BasicContext Context(prec=9, rounding=ROUND_HALF_UP, Emin=-999999999, Emax=999999999, capitals=1, flags=[], traps=[Clamped, DivisionByZero, InvalidOperation, Overflow, Underflow]) >>> decimal.DefaultContext Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999, capitals=1, flags=[], traps=[DivisionByZero, InvalidOperation, Overflow]) >>> decimal.ExtendedContext Context(prec=9, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999, capitals=1, flags=[], traps=[]) >>> decimal.setcontext(decimal.ExtendedContext) >>> decimal.getcontext().prec 9 >>> decimal.setcontext(decimal.Context(prec=50, rounding=decimal.ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999, capitals=1, flags=[], traps=[decimal.DivisionByZero, decimal.InvalidOperation, decimal.Overflow])) >>> decimal.getcontext().prec 50 >>> 100*a/b Decimal("66.666666666666666666666666666666666666666666666667") Hold on a second . . . it looks like prec doesn't refer to how many digits there will be after the decimal point, but how many total digits . . . bit of an odd way to do things, but that explains your problem. Scratch what i said before about assigning to decimal.getcontext().prec, by the way, it looks like that works after all: >>> decimal.getcontext().prec = 4 >>> 100 * a/b Decimal("66.67") HTH, Orri -- Email: singingxduck AT gmail DOT com AIM: singingxduck Programming Python for the fun of it.

_______________________________________________
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor