Re: [Python-Dev] Why is nan != nan?
Am 24.03.2010 22:47, schrieb Mark Dickinson:
> On Wed, Mar 24, 2010 at 10:36 PM, Alexander Belopolsky
> wrote:
>> On Wed, Mar 24, 2010 at 6:31 PM, Mark Dickinson wrote:
>> ..
>>> Neither is necessary, because Python doesn't actually use == as the
>>> equivalence relation for containment testing: the actual equivalence
>>> relation is: x equivalent to y iff id(x) == id(y) or x == y. This
>>> restores the missing reflexivity (besides being a useful
>>> optimization).
>>
>> No, it does not:
>>
> float('nan') in [float('nan')]
>> False
>
> Sure, but just think of it as having two different nans there. (You
> could imagine thinking of the id of the nan as part of the payload.)
That's interesting. Thinking of each value created by float('nan') as
a different nan makes sense to my naive mind, and it also explains
nicely the behavior present right now. Each nan comes from a different
operation and therefore is a "different" non-number.
Of course, float being an immutable type, there is some reason to expect
that all values created by float('nan') should be identical, but after all,
datetime is an immutable type as well, but you wouldn't expect
datetime.now() in [datetime.now()]
to be true. The only wart left is that you can't distinguish different
nans by their string representation -- this could be remedied by making
it ``"nan-%s" % id(self)``, but that looks a bit ugly to me.
Georg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r79397 - in python/trunk: Doc/c-api/capsule.rst Doc/c-api/cobject.rst Doc/c-api/concrete.rst Doc/data/refcounts.dat Doc/extending/extending.rst Include/Python.h Incl
larry.hastings wrote: > Author: larry.hastings > Date: Thu Mar 25 01:54:54 2010 > New Revision: 79397 > > Log: > Backported PyCapsule from 3.1, and converted most uses of > CObject to PyCapsule. Backporting PyCapsule is fine, but the changes you made to all those PyCObject uses does not look backwards compatible. The C APIs exposed by the modules (e.g. the datetime module) are used in lots of 3rd party extension modules and changing them from PyCObject to PyCapsule is a major change in the module API. > Added: >python/trunk/Doc/c-api/capsule.rst >python/trunk/Include/pycapsule.h >python/trunk/Objects/capsule.c > Modified: >python/trunk/Doc/c-api/cobject.rst >python/trunk/Doc/c-api/concrete.rst >python/trunk/Doc/data/refcounts.dat >python/trunk/Doc/extending/extending.rst >python/trunk/Include/Python.h >python/trunk/Include/cStringIO.h >python/trunk/Include/cobject.h >python/trunk/Include/datetime.h >python/trunk/Include/py_curses.h >python/trunk/Include/pyexpat.h >python/trunk/Include/ucnhash.h >python/trunk/Lib/test/test_sys.py >python/trunk/Makefile.pre.in >python/trunk/Misc/NEWS >python/trunk/Modules/_ctypes/callproc.c >python/trunk/Modules/_ctypes/cfield.c >python/trunk/Modules/_ctypes/ctypes.h >python/trunk/Modules/_cursesmodule.c >python/trunk/Modules/_elementtree.c >python/trunk/Modules/_testcapimodule.c >python/trunk/Modules/cStringIO.c >python/trunk/Modules/cjkcodecs/cjkcodecs.h >python/trunk/Modules/cjkcodecs/multibytecodec.c >python/trunk/Modules/cjkcodecs/multibytecodec.h >python/trunk/Modules/datetimemodule.c >python/trunk/Modules/pyexpat.c >python/trunk/Modules/socketmodule.c >python/trunk/Modules/socketmodule.h >python/trunk/Modules/unicodedata.c >python/trunk/Objects/object.c >python/trunk/Objects/unicodeobject.c >python/trunk/PC/VS7.1/pythoncore.vcproj >python/trunk/PC/VS8.0/pythoncore.vcproj >python/trunk/PC/os2emx/python27.def >python/trunk/PC/os2vacpp/python.def >python/trunk/Python/compile.c >python/trunk/Python/getargs.c -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 25 2010) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-checkins mailing list python-check...@python.org http://mail.python.org/mailman/listinfo/python-checkins ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Thu, 25 Mar 2010 05:26:12 am Alexander Belopolsky wrote: > Mark, I wonder if you could describe an algorithm off the top of your > head that relies on NaN == NaN being false. I don't know whether "relies on" is appropriate, but consider: def myfunc(x, y): if x == y: return 1.0 else: return something_complicated**(x-y) Optimising floating point code is fraught with dangers (the above fails for x=y=INF as well as NAN) but anything that make Not A Numbers pretend to be numbers is a bad thing. I'd like to turn the question around ... what algorithms are there that rely on NaN == NaN being True? -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Steven D'Aprano wrote: > On Thu, 25 Mar 2010 03:22:29 am Mark Dickinson wrote: >> The obvious way to do this nan interning for floats would be to put >> the interning code into PyFloat_FromDouble. I'm not sure whether >> this would be worth the cost in terms of added code (and possibly >> reduced performance, since the nan check would be done every time a >> float was returned), but I'd be willing to review a patch. > > I hope that it's obvious from my previous post that I do NOT want such > interning done, but since I put the idea in people's heads, I wish to > reiterate that I'm against the idea: -1 on interning NaNs. For the rare > application where it might be useful, it is easy to do in the > application code. Yep, and I'll freely admit I didn't know about the potential additional state on NaN values, or I wouldn't have suggested interning automatically. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __pycache__ creation
Ron Adam wrote: > h... unless there is a __pycache__ *file* located there first. ;-) Just a specific reason why attempting to create __pycache__ can fail (which has defined behaviour in the PEP - running directly from the source without caching the bytecode file). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
Filip Gruszczyński wrote: > I would appreciate any advice on this topic, even if this ticket would > be dismissed altogether, as I would like to learn as much as possible > on practices on developing Python. Raymond's use case is valid, but the currently proposed method name is way too long to be usable and the use of *args is a definite misfeature. I suggest approaching the issue with the exact spec needed to cover the problem Raymond described: Add a method to strings that allows a format string to be applied to an existing mapping without implicitly converting the mapping to a dict first. From that spec, a straightforward API falls out: def format_mapping(self, kwds): # Method body actually written in C, so it can # easily invoke the internal formatting operation return do_string_format(self, NULL, kwds) Sure, you can't mix positional and keyword arguments the way you can with .format(), but you can't mix and match those with mod-formatting either. *If* support for a sequence were to be added (and that's a big if), it should be added as an explicit second argument, not as a *args argument. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Mark Dickinson wrote: > Here's an interesting recent blog post on this subject, from the > creator of Eiffel: > > http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-civilization/ Interesting. So the natural tweak that would arise from that perspective is for us to restore reflexivity by declaring that any given NaN is equal to *itself* but not to any other NaN (even one with the same payload). With NaN (in general) not being interned, that would actually fit the idea of a NaN implicitly carrying the operation that created the NaN as part of its definition of equivalence. So, I'm specifically putting that proposal on the table for both float and Decimal NaNs in Python: "Not a Number" is not a single floating point value. Instead each instance is a distinct value representing the precise conditions that created it. Thus, two "NaN" values x and y will compare equal iff they are the exact same NaN object (i.e. "if isnan(x) then x == y iff x is y". As stated above, such a change would allow us to restore reflexivity (eliminating a bunch of weirdness) while still honouring the idea of NaN being a set of values rather than a single value. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Steven D'Aprano wrote: > I'd like to turn the question around ... what algorithms are there that > rely on NaN == NaN being True? Absolutely anything that expects "x is y" to imply that "x == y". The builtin containers enforce this by checking identity before they check equality, but there are plenty of algorithms that won't. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
> From that spec, a straightforward API falls out: > > def format_mapping(self, kwds): > # Method body actually written in C, so it can > # easily invoke the internal formatting operation > return do_string_format(self, NULL, kwds) Thanks a lot for the advice, I'll provide according patch hopefully in the few days. -- Filip Gruszczyński ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 11:22 AM, Nick Coghlan wrote: > Mark Dickinson wrote: >> Here's an interesting recent blog post on this subject, from the >> creator of Eiffel: >> >> http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-civilization/ > > Interesting. So the natural tweak that would arise from that perspective > is for us to restore reflexivity by declaring that any given NaN is > equal to *itself* but not to any other NaN (even one with the same payload). > > With NaN (in general) not being interned, that would actually fit the > idea of a NaN implicitly carrying the operation that created the NaN as > part of its definition of equivalence. > > So, I'm specifically putting that proposal on the table for both float > and Decimal NaNs in Python: > > "Not a Number" is not a single floating point value. Instead each > instance is a distinct value representing the precise conditions that > created it. Thus, two "NaN" values x and y will compare equal iff they > are the exact same NaN object (i.e. "if isnan(x) then x == y iff > x is y". In other words, this would make explicit, at the level of ==, what Python's already doing under the hood (e.g. in PyObjectRichCompareBool) for membership testing---at least for nans. > As stated above, such a change would allow us to restore reflexivity > (eliminating a bunch of weirdness) while still honouring the idea of NaN > being a set of values rather than a single value. +0.2 from me. I could happily live with this change; but could also equally live with the existing weirdness. It's still a little odd for an immutable type to care about object identity, but I guess oddness comes with the floating-point territory. :) Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
Filip Gruszczyński wrote: From that spec, a straightforward API falls out: def format_mapping(self, kwds): # Method body actually written in C, so it can # easily invoke the internal formatting operation return do_string_format(self, NULL, kwds) Thanks a lot for the advice, I'll provide according patch hopefully in the few days. I think this is basically my patch, with the method renamed, and needing tests. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r79397 - in python/trunk: Doc/c-api/capsule.rst Doc/c-api/cobject.rst Doc/c-api/concrete.rst Doc/data/refcounts.dat Doc/extending/extending.rst Include/Python.h Incl
larry.hastings wrote: > Author: larry.hastings > Date: Thu Mar 25 01:54:54 2010 > New Revision: 79397 > > Log: > Backported PyCapsule from 3.1, and converted most uses of > CObject to PyCapsule. Backporting PyCapsule is fine, but the changes you made to all those PyCObject uses does not look backwards compatible. The C APIs exposed by the modules (e.g. the datetime module) are used in lots of 3rd party extension modules and changing them from PyCObject to PyCapsule is a major change in the module API. > Added: >python/trunk/Doc/c-api/capsule.rst >python/trunk/Include/pycapsule.h >python/trunk/Objects/capsule.c > Modified: >python/trunk/Doc/c-api/cobject.rst >python/trunk/Doc/c-api/concrete.rst >python/trunk/Doc/data/refcounts.dat >python/trunk/Doc/extending/extending.rst >python/trunk/Include/Python.h >python/trunk/Include/cStringIO.h >python/trunk/Include/cobject.h >python/trunk/Include/datetime.h >python/trunk/Include/py_curses.h >python/trunk/Include/pyexpat.h >python/trunk/Include/ucnhash.h >python/trunk/Lib/test/test_sys.py >python/trunk/Makefile.pre.in >python/trunk/Misc/NEWS >python/trunk/Modules/_ctypes/callproc.c >python/trunk/Modules/_ctypes/cfield.c >python/trunk/Modules/_ctypes/ctypes.h >python/trunk/Modules/_cursesmodule.c >python/trunk/Modules/_elementtree.c >python/trunk/Modules/_testcapimodule.c >python/trunk/Modules/cStringIO.c >python/trunk/Modules/cjkcodecs/cjkcodecs.h >python/trunk/Modules/cjkcodecs/multibytecodec.c >python/trunk/Modules/cjkcodecs/multibytecodec.h >python/trunk/Modules/datetimemodule.c >python/trunk/Modules/pyexpat.c >python/trunk/Modules/socketmodule.c >python/trunk/Modules/socketmodule.h >python/trunk/Modules/unicodedata.c >python/trunk/Objects/object.c >python/trunk/Objects/unicodeobject.c >python/trunk/PC/VS7.1/pythoncore.vcproj >python/trunk/PC/VS8.0/pythoncore.vcproj >python/trunk/PC/os2emx/python27.def >python/trunk/PC/os2vacpp/python.def >python/trunk/Python/compile.c >python/trunk/Python/getargs.c -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 25 2010) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Mark Dickinson wrote: > +0.2 from me. I could happily live with this change; but could also > equally live with the existing weirdness. > > It's still a little odd for an immutable type to care about object > identity, but I guess oddness comes with the floating-point territory. > :) The trick for me came in thinking of NaN as a set of values rather than a single value - at that point, the different id values just reflect the multitude of members of that set. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 11:22 AM, Nick Coghlan wrote:
> So, I'm specifically putting that proposal on the table for both float
> and Decimal NaNs in Python:
>
> "Not a Number" is not a single floating point value. Instead each
> instance is a distinct value representing the precise conditions that
> created it. Thus, two "NaN" values x and y will compare equal iff they
> are the exact same NaN object (i.e. "if isnan(x) then x == y iff
> x is y".
I'd also suggest that the language make no guarantees about whether
two distinct calls to float('nan') or Decimal('nan') (or any other
function call returning a nan) return identical values or not, but
leave implementations free to do what's convenient or efficient.
For example, with the current decimal module: Decimal('nan') returns
a new nan each time, but Decimal(-1).sqrt() always returns the same
nan object (assuming that InvalidOperation isn't trapped). I think
it's fine to regard this as an implementation detail.
Python 2.6.2 (r262:71600, Aug 26 2009, 09:40:44)
[GCC 4.2.1 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import *
>>> getcontext().traps[InvalidOperation] = 0
>>> x, y = Decimal('nan'), Decimal('nan')
>>> id(x), id(y)
(47309953516000, 47309930620880)
>>> x, y = Decimal(-1).sqrt(), Decimal(-1).sqrt()
>>> id(x), id(y)
(9922272, 9922272)
Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 03/25/2010 07:54 AM, Georg Brandl wrote:
>> float('nan') in [float('nan')]
>>> False
>>
>> Sure, but just think of it as having two different nans there. (You
>> could imagine thinking of the id of the nan as part of the payload.)
>
> That's interesting. Thinking of each value created by float('nan') as
> a different nan makes sense to my naive mind, and it also explains
> nicely the behavior present right now. Each nan comes from a different
> operation and therefore is a "different" non-number.
Infinites are "not equal" for a good reason, for example.
1/0 and 2/0 are both infinites, but one is "greater" than the other. Or
(1/0)^(1/0), an infinite infinitelly "bigger".
- --
Jesus Cea Avion _/_/ _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/ _/_/_/_/ _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/ _/_/_/_/_/
. _/_/ _/_/_/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/
"My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBS6tYzJlgi5GaxT1NAQL7kgP+LqjzNKOkSOZ+gTvgKfUTrF1poNP1VMC4
1LTkCcpFQYUoc4d8kk6lmzN7RdBesidbnVC2SApKTdNTAfbKMB3hjkTIzoxbx9wf
sLb5IUSqhtc+xJ+JQFepQwA7YLa64AVI23/wZcJCkqCBIg6S5DuGxhWErr3TXVgF
GqcZjhvD2lA=
=AVOO
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/25/2010 12:22 PM, Nick Coghlan wrote: > "Not a Number" is not a single floating point value. Instead each > instance is a distinct value representing the precise conditions that > created it. Thus, two "NaN" values x and y will compare equal iff they > are the exact same NaN object (i.e. "if isnan(x) then x == y iff > x is y". > > As stated above, such a change would allow us to restore reflexivity > (eliminating a bunch of weirdness) while still honouring the idea of NaN > being a set of values rather than a single value. Sounds good. But IEEE 754 was created by pretty clever guys and sure they had a reason for define things in the way they are. Probably we are missing something. - -- Jesus Cea Avion _/_/ _/_/_/_/_/_/ j...@jcea.es - http://www.jcea.es/ _/_/_/_/ _/_/_/_/ _/_/ jabber / xmpp:j...@jabber.org _/_/_/_/ _/_/_/_/_/ . _/_/ _/_/_/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/ "My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBS6tZfZlgi5GaxT1NAQJZtQP/bxi6l5TaiiOzv+no6cLaVXbkWXb9v6OL jmejzrlAosXzzd/4CuinN2mdFs7bd9Y3O9gHoQ2nUfbfWQc4SwxpxjK67j10PODJ MMz7wXgz075A8S7gUlpwWznByU2VfAys6ZVxZCv/uogW9SXIHqEBC/sXwWN5Hwvn uHImzIL4bfs= =RI3T -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
Nick Coghlan wrote:
Filip Gruszczyński wrote:
I would appreciate any advice on this topic, even if this ticket would
be dismissed altogether, as I would like to learn as much as possible
on practices on developing Python.
Raymond's use case is valid, but the currently proposed method name is
way too long to be usable and the use of *args is a definite misfeature.
I suggest approaching the issue with the exact spec needed to cover the
problem Raymond described:
Add a method to strings that allows a format string to
be applied to an existing mapping without implicitly
converting the mapping to a dict first.
It occurs to me that Raymond's use case could be satisfied using
existing Python, by slightly changing the format string. After all,
str.format() supports mapping lookup already:
$ ./python.exe
Python 2.6.5+ (release26-maint:79421, Mar 25 2010, 08:51:39)
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class Default(dict):
... def __missing__(self, key):
... return key
...
>>> s = '{m[name]} was born in {m[country]}'
>>> s.format(m=Default(name='Guido'))
'Guido was born in country'
>>>
Considering that, maybe the best thing is to do nothing. I'll update the
issue with this note.
--
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 12:36 PM, Jesus Cea wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 03/25/2010 07:54 AM, Georg Brandl wrote:
>>> float('nan') in [float('nan')]
False
>>>
>>> Sure, but just think of it as having two different nans there. (You
>>> could imagine thinking of the id of the nan as part of the payload.)
>>
>> That's interesting. Thinking of each value created by float('nan') as
>> a different nan makes sense to my naive mind, and it also explains
>> nicely the behavior present right now. Each nan comes from a different
>> operation and therefore is a "different" non-number.
>
> Infinites are "not equal" for a good reason, for example.
Well, that depends on your mathematical model. The underlying
mathematical model for IEEE 754 floating-point is the doubly extended
real line: that is, the set of all real numbers augmented with two
extra elements "infinity" and "-infinity", with the obvious total
order. This is made explicit in section 3.2 of the standard:
"The mathematical structure underpinning the arithmetic in this
standard is the extended reals, that is, the set
of real numbers together with positive and negative infinity."
This is the same model that one typically uses in a first course in
calculus when studying limits of functions; it's an appropriate model
for dealing with computer approximations to real numbers and
continuous functions. So the model has precisely two infinities, and
1/0, 2/0 and (1/0)**(1/0) all give the same infinity. The decision
to make 1/0 "infinity" rather than "-infinity" is admittedly a little
arbitrary. For floating-point (but not for calculus!), it makes sense
in the light of the decision to have both positive and negative
floating-point zeros; 1/(-0) is -infinity, of course.
Other models of "reals + one or more infinities" are possible, of
course, but they're not relevant to IEEE 754 floating point. There's
a case for using a floating-point model with a single infinity,
especially for those who care more about algebraic functions
(polynomials, rational functions) than transcendental ones; however,
IEEE 754 doesn't make provisions for this.
Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 12:39 PM, Jesus Cea wrote: > > But IEEE 754 was created by pretty clever guys and sure they had a > reason for define things in the way they are. Probably we are missing > something. Well, if we are, then nobody seems to know what! See the Bertrand Meyer blog post that was linked to up-thread. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Jesus Cea wrote: > But IEEE 754 was created by pretty clever guys and sure they had a > reason for define things in the way they are. Probably we are missing > something. Yes, this is where their "implementable in a hardware circuit" focus comes in. They were primarily thinking of a floating point representation where the 32/64 bits are *it* - you can't have "multiple NaNs" because you don't have the bits available to describe them. We don't have that limitation - by bring id() into play for NaN equality tests we have a lot more bits to play with (effectively adding an extra 32 or 64 bits to the floating point value just to say "which NaN is this one?"). Doubling the size of your storage just to have multiple kinds of NaN would be insane, but in our case the extra storage is already in use - we would just be applying an additional interpretation to it. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
Eric Smith wrote: > Considering that, maybe the best thing is to do nothing. I'll update the > issue with this note. I agree this slightly weakens the case for change, but it's not really the same thing. Adding a "format_mapping" method allows an arbitrary mapping to be used with any keyword-based format string. Crafting a special format string in order to use an arbitrary mapping means that the same format string can no longer be used with an ordinary keyword based call - you have to do "format(m=dict(name=y, country=z))" instead. Moving the decision of "how am I going to be called" to the time of writing the format string is a bit odd. On the other hand, the specially crafted format string does have the virtue of travelling far more easily through any APIs that wrap the basic format() method (since it doesn't need special treatment to make its way through the wrapping code). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, 25 Mar 2010 11:36:28 pm Jesus Cea wrote:
> Infinites are "not equal" for a good reason, for example.
>
> 1/0 and 2/0 are both infinites, but one is "greater" than the other.
> Or (1/0)^(1/0), an infinite infinitelly "bigger".
I think you're mistaken. In Python 3.1:
>>> x = float('inf')
>>> y = float('inf') + 1
>>> x == y
True
>>> x is y
False
In cardinal arithmetic, there are an infinity of different sized
infinities, but in ordinal arithmetic there are only two: +inf
and -inf, corresponding to the infinities on the real number line. (I
hope that I'm not over-simplifying -- it's been more than a decade
since I've needed to care about this.)
But in any case, the IEEE standard doesn't deal with cardinals: it only
uses two signed infinities.
--
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 7:08 AM, Nick Coghlan wrote: > Jesus Cea wrote: > > But IEEE 754 was created by pretty clever guys and sure they had a > > reason for define things in the way they are. Probably we are missing > > something. > > Yes, this is where their "implementable in a hardware circuit" focus > comes in. They were primarily thinking of a floating point > representation where the 32/64 bits are *it* - you can't have "multiple > NaNs" because you don't have the bits available to describe them. > Wait, what? I haven't been paying much attention, but this is backwards. There are multiple representations of NaN in the IEEE encoding; that's actually part of the problem with saying that NaN = NaN or NaN != NaN. If you want to ignore the "payload" in the NaN, then you're not just comparing bits any more. -- Curt Hagenlocher c...@hagenlocher.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 2:08 PM, Nick Coghlan wrote: > Jesus Cea wrote: >> But IEEE 754 was created by pretty clever guys and sure they had a >> reason for define things in the way they are. Probably we are missing >> something. > > Yes, this is where their "implementable in a hardware circuit" focus > comes in. They were primarily thinking of a floating point > representation where the 32/64 bits are *it* - you can't have "multiple > NaNs" because you don't have the bits available to describe them. I'm not so sure about this: standard 64-bit binary IEEE 754 doubles allow for 2**53-2 different nans (2**52-2 signaling nans, 2**52 quiet nans): anything with bit pattern (msb to lsb) x111 is an infinity or a nan, and there are only 2 infinities. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Le Thu, 25 Mar 2010 07:19:24 -0700, Curt Hagenlocher a écrit : > On Thu, Mar 25, 2010 at 7:08 AM, Nick Coghlan > wrote: > >> Jesus Cea wrote: >> > But IEEE 754 was created by pretty clever guys and sure they had a >> > reason for define things in the way they are. Probably we are missing >> > something. >> >> Yes, this is where their "implementable in a hardware circuit" focus >> comes in. They were primarily thinking of a floating point >> representation where the 32/64 bits are *it* - you can't have "multiple >> NaNs" because you don't have the bits available to describe them. >> > Wait, what? I haven't been paying much attention, but this is backwards. > There are multiple representations of NaN in the IEEE encoding; that's > actually part of the problem with saying that NaN = NaN or NaN != NaN. > If you want to ignore the "payload" in the NaN, then you're not just > comparing bits any more. This sounds a bit sophistic, if the (Python) user doesn't have access to the payload anyway. Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 2:26 PM, Antoine Pitrou wrote: > Le Thu, 25 Mar 2010 07:19:24 -0700, Curt Hagenlocher a écrit : >> Wait, what? I haven't been paying much attention, but this is backwards. >> There are multiple representations of NaN in the IEEE encoding; that's >> actually part of the problem with saying that NaN = NaN or NaN != NaN. >> If you want to ignore the "payload" in the NaN, then you're not just >> comparing bits any more. > > This sounds a bit sophistic, if the (Python) user doesn't have access to > the payload anyway. Well, you can get at the payload using the struct module, if you care enough. But yes, it's true that Python doesn't take much care with the payload: e.g., ideally, an operation on a nan (3.0 + nan, sqrt(nan), ...) should return exactly the same nan, to make sure that information in the payload is preserved. Python doesn't bother, for floats (though it does for decimal). Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Mark Dickinson gmail.com> writes: > > On Thu, Mar 25, 2010 at 12:39 PM, Jesus Cea jcea.es> wrote: > > > > But IEEE 754 was created by pretty clever guys and sure they had a > > reason for define things in the way they are. Probably we are missing > > something. > > Well, if we are, then nobody seems to know what! See the Bertrand > Meyer blog post that was linked to up-thread. The missing part, IMO, is that allowing a given NaN value to compare equal to itself only pushes the problem up one level. Any single operation yielding a NaN will still be unequal to itself. That is, under what is being proposed, with a function func() returning the same result of some calculation: x = func() s1 = (x) print x in s1 print func() in s1 This would print True and False, even though func() is perfoming the same calculation and thus logically returning the same NaN. I think the IEEE NaN represent the fact that you have a number of an undefined set, but it doesn't specify which. The only way out, IMO, is to make *all* NaN comparisons yield False, but identity yield true. No interning necessary. At most, you could make the identity function return False for the different types of NaN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 2:42 PM, Mark Dickinson wrote:
> On Thu, Mar 25, 2010 at 2:26 PM, Antoine Pitrou wrote:
>> This sounds a bit sophistic, if the (Python) user doesn't have access to
>> the payload anyway.
>
> Well, you can get at the payload using the struct module, if you care
> enough. But yes, it's true that Python doesn't take much care with
> the payload: e.g., ideally, an operation on a nan (3.0 + nan,
> sqrt(nan), ...) should return exactly the same nan, to make sure that
> information in the payload is preserved. Python doesn't bother, for
> floats (though it does for decimal).
Hmm. I take it back. I was being confused by the fact that sqrt(nan)
returns a nan with a new identity; but it does apparently preserve
the payload. An example:
>>> from struct import pack, unpack
>>> from math import sqrt
>>> x = unpack('>> y = sqrt(x)
>>> bin(unpack('>> bin(unpack('http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
Nick Coghlan wrote: Moving the decision of "how am I going to be called" to the time of writing the format string is a bit odd. On the other hand, the specially crafted format string does have the virtue of travelling far more easily through any APIs that wrap the basic format() method (since it doesn't need special treatment to make its way through the wrapping code). Completely agree on all points. Now we're just left with "is it worth expanding the str api for this?". I don't feel strongly either way. -- Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 7:54 AM, Mark Dickinson wrote: > > Hmm. I take it back. I was being confused by the fact that sqrt(nan) > returns a nan with a new identity; but it does apparently preserve > the payload. An example: I played with this some a few months ago, and both the FPU and the C libraries I tested will preserve the payload. I imagine Python just inherits their behavior. -- Curt Hagenlocher c...@hagenlocher.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 3:01 PM, Curt Hagenlocher wrote: > On Thu, Mar 25, 2010 at 7:54 AM, Mark Dickinson wrote: >> >> Hmm. I take it back. I was being confused by the fact that sqrt(nan) >> returns a nan with a new identity; but it does apparently preserve >> the payload. An example: > > I played with this some a few months ago, and both the FPU and the C > libraries I tested will preserve the payload. I imagine Python just > inherits their behavior. Pretty much, yes. I think we've also taken care to preserve payloads in functions that have been added to the math library as well (e.g., the gamma function). Not that that's particularly hard: it's just a matter of making sure to do "if (isnan(x)) return x;" rather than "if (isnan(x)) return standard_python_nan;". If that's not true, then there's a minor bug to be corrected. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Mark Dickinson wrote: > On Thu, Mar 25, 2010 at 2:08 PM, Nick Coghlan wrote: >> Jesus Cea wrote: >>> But IEEE 754 was created by pretty clever guys and sure they had a >>> reason for define things in the way they are. Probably we are missing >>> something. >> Yes, this is where their "implementable in a hardware circuit" focus >> comes in. They were primarily thinking of a floating point >> representation where the 32/64 bits are *it* - you can't have "multiple >> NaNs" because you don't have the bits available to describe them. > > I'm not so sure about this: standard 64-bit binary IEEE 754 doubles > allow for 2**53-2 different nans (2**52-2 signaling nans, 2**52 quiet > nans): anything with bit pattern (msb to lsb) > > x111 > > is an infinity or a nan, and there are only 2 infinities. I stand corrected :) It still seems to me that the problems mostly arise when we're trying to get floats and Decimals to behave like Python *objects* (i.e. with reflexive equality) rather than like IEEE defined numbers. It's an extra element that isn't part of the problem the numeric standards are trying to solve. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 9:39 PM, Jesus Cea wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 03/25/2010 12:22 PM, Nick Coghlan wrote: >> "Not a Number" is not a single floating point value. Instead each >> instance is a distinct value representing the precise conditions that >> created it. Thus, two "NaN" values x and y will compare equal iff they >> are the exact same NaN object (i.e. "if isnan(x) then x == y iff >> x is y". >> >> As stated above, such a change would allow us to restore reflexivity >> (eliminating a bunch of weirdness) while still honouring the idea of NaN >> being a set of values rather than a single value. > > Sounds good. > > But IEEE 754 was created by pretty clever guys and sure they had a > reason for define things in the way they are. Probably we are missing > something. Yes, indeed. I don't claim having a deep understanding myself, but up to now, everytime I thought something in IEE 754 was weird, it ended up being for good reasons. I think the fundamental missing point in this discussion about Nan is exception handling: a lot of NaN quircky behavior becomes much more natural once you take into account which operations are invalid under which condition. Unless I am mistaken, python itself does not support for FPU exception handling. For example, the reason why x != x for x Nan is because != (and ==) are about the only operations where you can have NaN as operands without risking raising an exception, and support for creating and detecting NaN in languages have been coming only quite lately (e.g. C99). Concerning the lack of rationale: a relatively short reference concerned about FPU exception and NaN handling is from Kahan himself http://www.eecs.berkeley.edu/~wkahan/ieee754status/ieee754.ps David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
Eric Smith wrote: > Nick Coghlan wrote: >> Moving the decision of "how am I going to be called" to the time of >> writing the format string is a bit odd. >> >> On the other hand, the specially crafted format string does have the >> virtue of travelling far more easily through any APIs that wrap the >> basic format() method (since it doesn't need special treatment to make >> its way through the wrapping code). > > Completely agree on all points. Now we're just left with "is it worth > expanding the str api for this?". I don't feel strongly either way. For something as core as the string API, we better feel darn strongly about it before we mess with it :) I'm inclined to leave it alone unless/until Raymond or somebody else steps up to really champion it. Cheers, Nick. P.S. There's also the language moratorium to consider - since this idea affect the methods of a builtin type, I believe the moratorium applies. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 3:05 PM, Nick Coghlan wrote: > Mark Dickinson wrote: >> On Thu, Mar 25, 2010 at 2:08 PM, Nick Coghlan wrote: >>> Jesus Cea wrote: But IEEE 754 was created by pretty clever guys and sure they had a reason for define things in the way they are. Probably we are missing something. >>> Yes, this is where their "implementable in a hardware circuit" focus >>> comes in. They were primarily thinking of a floating point >>> representation where the 32/64 bits are *it* - you can't have "multiple >>> NaNs" because you don't have the bits available to describe them. >> >> I'm not so sure about this: standard 64-bit binary IEEE 754 doubles >> allow for 2**53-2 different nans (2**52-2 signaling nans, 2**52 quiet >> nans): anything with bit pattern (msb to lsb) >> >> x111 >> >> is an infinity or a nan, and there are only 2 infinities. > > I stand corrected :) > > It still seems to me that the problems mostly arise when we're trying to > get floats and Decimals to behave like Python *objects* (i.e. with > reflexive equality) rather than like IEEE defined numbers. > > It's an extra element that isn't part of the problem the numeric > standards are trying to solve. Agreed. We don't have to be "missing something"; rather, the IEEE folks (quite understandably) almost certainly didn't anticipate this kind of usage. So I'll concede that it's reasonable to consider deviating from the standard in the light of this. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
Nick Coghlan gmail.com> writes: > > For something as core as the string API, we better feel darn strongly > about it before we mess with it :) Having two very similar methods with different names and subtly different APIs sounds bad IMHO. Also, the use case doesn't look very strong to me. > P.S. There's also the language moratorium to consider - since this idea > affect the methods of a builtin type, I believe the moratorium applies. So do I. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
Nick Coghlan wrote: Completely agree on all points. Now we're just left with "is it worth expanding the str api for this?". I don't feel strongly either way. For something as core as the string API, we better feel darn strongly about it before we mess with it :) I'm inclined to leave it alone unless/until Raymond or somebody else steps up to really champion it. I'm okay with that. P.S. There's also the language moratorium to consider - since this idea affect the methods of a builtin type, I believe the moratorium applies. I don't think there's any effort to get this in before the moratorium expires. Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Mar 25, 2010, at 4:22 AM, Nick Coghlan wrote: > Mark Dickinson wrote: >> Here's an interesting recent blog post on this subject, from the >> creator of Eiffel: >> >> http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-civilization/ > > Interesting. So the natural tweak that would arise from that perspective > is for us to restore reflexivity by declaring that any given NaN is > equal to *itself* but not to any other NaN (even one with the same payload). > > With NaN (in general) not being interned, that would actually fit the > idea of a NaN implicitly carrying the operation that created the NaN as > part of its definition of equivalence. > > So, I'm specifically putting that proposal on the table for both float > and Decimal NaNs in Python: > > "Not a Number" is not a single floating point value. Instead each > instance is a distinct value representing the precise conditions that > created it. Thus, two "NaN" values x and y will compare equal iff they > are the exact same NaN object (i.e. "if isnan(x) then x == y iff > x is y". > > As stated above, such a change would allow us to restore reflexivity > (eliminating a bunch of weirdness) while still honouring the idea of NaN > being a set of values rather than a single value. > +1 Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r79397 - in python/trunk: Doc/c-api/capsule.rst Doc/c-api/cobject.rst Doc/c-api/concrete.rst Doc/data/refcounts.dat Doc/extending/extending.rst Include/Python.h Incl
M.-A. Lemburg wrote:
Backporting PyCapsule is fine, but the changes you made to all
those PyCObject uses does not look backwards compatible.
The C APIs exposed by the modules (e.g. the datetime module)
are used in lots of 3rd party extension modules and changing
them from PyCObject to PyCapsule is a major change in the
module API.
You're right, my changes aren't backwards compatible. I thought it was
reasonable for four reasons:
1. The CObject API isn't safe. It's easy to crash Python 2.6 in just a
few lines by mixing and matching CObjects. Switching Python to capsules
prevents a class of exploits. I've included a script at the bottom of
this message that demonstrates three such crashes. The script runs in
Python 2 and 3, but 3.1 doesn't crash because it's using capsules.
2. As I just mentioned, Python 3.1 already uses capsules everywhere
instead of CObjects. Since part of the purpose of Python 2.7 is to
prepare developers for the to upgrade to 3.1, getting them to switch to
capsules now is just one more way they are prepared.
3. Because CObject is unsafe, I want to deprecate it in 2.7, and if we
ever made a 2.8 I want to remove it completely.
4. When Python publishes an API using a CObject, it describes the thing
the CObject points to in a header file. In nearly all cases that header
file also provides a macro or inline function that does the importing
work for you. I changed those to use capsules too. So if the
third-party code uses the macro or inline function, all you need do is
recompile it against 2.7 and it works fine. Sadly I know of one
exception: pyexpat.expat_CAPI. The header file just describes the
struct pointed to by the CObject, but callers
I can suggest four ways to ameliorate the problem.
First, we could do as Antoine Pitrou suggests on the bug (issue 7992):
wherever the CObject used to be published as a module attribute to
expose an API, we could provide both a CObject and a capsule; internally
Python would only use the capsules. This would allow third-party
libraries to run against 2.7 unchanged. The major problem with this is
that third-party libraries would still be vulnerable to the
mix-and-match CObject crash. A secondary, minor concern: obviously we'd
store the CObject attribute with the existing name, and the capsule
attribute would have to get some new name. But in Python 3.1, these
attributes already expose a capsule. Therefore, people who convert to
using the capsules now would have to convert again when moving to 3.1.
Second, we could make CObject internally support unpacking capsules. If
you gave a capsule to PyCObject_AsVoidPtr() it would unpack it and
return the pointer within. (We could probably also map the capsule
"context" to the CObject "desc", if any of the Python use cases needed
it.) I wouldn't change anything else about CObjects; creating and using
them would continue to work as normal. This would also allow
third-party libraries to run against Python 2.7 unchanged. The only
problem is that it's unsafe, as indeed allowing any use of
PyCObject_AsVoidPtr() is unsafe.
Third, I've been pondering writing a set of preprocessor macros, shipped
in their own header file distributed independently of Python and
released to the public domain, that would make it easy to use either
CObjects or capsules depending on what version of Python you were
compiling against. Obviously, using these macros would require a source
code change in the third-party library. But these macros would make it
a five-minute change. This could compliment the first or second approaches.
Fourth, we could back out of the changes to published APIs and convert
them back to CObjects. -1.
Your thoughts?
/larry/
-
import sys
def log(message):
print(message)
sys.stdout.flush()
def crash1():
log("Running crash1...")
try:
import datetime
import cStringIO
cStringIO.cStringIO_CAPI = datetime.datetime_CAPI
import cPickle
s = cPickle.dumps([1, 2, 3])
except ImportError:
# This test isn't translatable to Python 3.
pass
log("Survived crash1!")
def crash2():
log("Running crash2...")
try:
import unicodedata
import _socket
_socket.CAPI = unicodedata.ucnhash_CAPI
import ssl
except AttributeError:
# Congratulations, you didn't crash.
pass
log("Survived crash2!")
def crash3():
log("Running crash3...")
try:
import unicodedata
import _multibytecodec
_multibytecodec.__create_codec(unicodedata.ucnhash_CAPI)
except ValueError:
# Congratulations, you didn't crash.
pass
log("Survived crash3!")
import sys
if len(sys.argv) > 1:
if sys.argv[1] == '1':
crash1()
sys.exit(0)
elif sys.argv[1] == '2':
crash2()
sys.exit(0)
elif sys.argv[1] == '3':
crash3()
sys.exit(0)
crash1()
crash2()
crash3()
Re: [Python-Dev] Why is nan != nan?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/25/2010 03:19 PM, Steven D'Aprano wrote: > On Thu, 25 Mar 2010 11:36:28 pm Jesus Cea wrote: > >> Infinites are "not equal" for a good reason, for example. >> >> 1/0 and 2/0 are both infinites, but one is "greater" than the other. >> Or (1/0)^(1/0), an infinite infinitelly "bigger". > > > I think you're mistaken. In Python 3.1: I was refering to mathematical infinites and why inf!=inf is sensible and a natural consequence calculus and limits (mathematically). In any case, I am an engineer, not a mathematical or a language designer :). IANAL. - -- Jesus Cea Avion _/_/ _/_/_/_/_/_/ j...@jcea.es - http://www.jcea.es/ _/_/_/_/ _/_/_/_/ _/_/ jabber / xmpp:j...@jabber.org _/_/_/_/ _/_/_/_/_/ . _/_/ _/_/_/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/ "My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBS6ufvJlgi5GaxT1NAQKrpQP+PbbNP6tNtPi480/ovki89DEDNqvKH4RU A/R6jLXwm7byF0RQ+3B9gUh8SfANQaTOeCOYufUJHXPDM3BHsmt2S1kcPoW5sUYe a38nUuD0sTyCV23h2QtzZpNGG7qNa6iHTMEc6vYgY3CdfCw+301xdqH0ZkmXV1B9 OIrj7ec7m+s= =rTUf -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/25/2010 04:07 PM, David Cournapeau wrote: > Yes, indeed. I don't claim having a deep understanding myself, but up > to now, everytime I thought something in IEE 754 was weird, it ended > up being for good reasons. I was wondering if we could bring the question to news:comp.arch newsgroup. They have the knowledge, and I know there are people from the IEEE 754 group lurking there. I only have read-only access, nevertheless. Another relevant group could be news:comp.arch.arithmetic, but I am not familiar with it. - -- Jesus Cea Avion _/_/ _/_/_/_/_/_/ j...@jcea.es - http://www.jcea.es/ _/_/_/_/ _/_/_/_/ _/_/ jabber / xmpp:j...@jabber.org _/_/_/_/ _/_/_/_/_/ . _/_/ _/_/_/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/ "My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBS6uhyZlgi5GaxT1NAQIZJwP/bAx5vXBLMLI8f724Hf0OtfATpV4SFQ84 RaLHNBPPkE5+cdFxWIv6VmFThtYfKnjutmLNU1TJYFoDwgOvqigYO8hOTFnWlfML Sx5B3LFdtGSZAfSsd+rMF23wKpbpAy/TicE+B6zg+Qy1LFv1V+OVn/Y3xBPGxVW5 m4yAKWT5T4U= =mwXx -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r79397 - in python/trunk: Doc/c-api/capsule.rst Doc/c-api/cobject.rst Doc/c-api/concrete.rst Doc/data/refcounts.dat Doc/extending/extending.rst Include/Python.h Incl
Larry Hastings wrote: > > M.-A. Lemburg wrote: >> Backporting PyCapsule is fine, but the changes you made to all >> those PyCObject uses does not look backwards compatible. >> >> The C APIs exposed by the modules (e.g. the datetime module) >> are used in lots of 3rd party extension modules and changing >> them from PyCObject to PyCapsule is a major change in the >> module API. > > You're right, my changes aren't backwards compatible. I thought it was > reasonable for four reasons: > > 1. The CObject API isn't safe. It's easy to crash Python 2.6 in just a > few lines by mixing and matching CObjects. Switching Python to capsules > prevents a class of exploits. I've included a script at the bottom of > this message that demonstrates three such crashes. The script runs in > Python 2 and 3, but 3.1 doesn't crash because it's using capsules. > > 2. As I just mentioned, Python 3.1 already uses capsules everywhere > instead of CObjects. Since part of the purpose of Python 2.7 is to > prepare developers for the to upgrade to 3.1, getting them to switch to > capsules now is just one more way they are prepared. > > 3. Because CObject is unsafe, I want to deprecate it in 2.7, and if we > ever made a 2.8 I want to remove it completely. > > 4. When Python publishes an API using a CObject, it describes the thing > the CObject points to in a header file. In nearly all cases that header > file also provides a macro or inline function that does the importing > work for you. I changed those to use capsules too. So if the > third-party code uses the macro or inline function, all you need do is > recompile it against 2.7 and it works fine. Sadly I know of one > exception: pyexpat.expat_CAPI. The header file just describes the > struct pointed to by the CObject, but callers > > > I can suggest four ways to ameliorate the problem. > > First, we could do as Antoine Pitrou suggests on the bug (issue 7992): > wherever the CObject used to be published as a module attribute to > expose an API, we could provide both a CObject and a capsule; internally > Python would only use the capsules. This would allow third-party > libraries to run against 2.7 unchanged. The major problem with this is > that third-party libraries would still be vulnerable to the > mix-and-match CObject crash. A secondary, minor concern: obviously we'd > store the CObject attribute with the existing name, and the capsule > attribute would have to get some new name. But in Python 3.1, these > attributes already expose a capsule. Therefore, people who convert to > using the capsules now would have to convert again when moving to 3.1. > > Second, we could make CObject internally support unpacking capsules. If > you gave a capsule to PyCObject_AsVoidPtr() it would unpack it and > return the pointer within. (We could probably also map the capsule > "context" to the CObject "desc", if any of the Python use cases needed > it.) I wouldn't change anything else about CObjects; creating and using > them would continue to work as normal. This would also allow > third-party libraries to run against Python 2.7 unchanged. The only > problem is that it's unsafe, as indeed allowing any use of > PyCObject_AsVoidPtr() is unsafe. > > Third, I've been pondering writing a set of preprocessor macros, shipped > in their own header file distributed independently of Python and > released to the public domain, that would make it easy to use either > CObjects or capsules depending on what version of Python you were > compiling against. Obviously, using these macros would require a source > code change in the third-party library. But these macros would make it > a five-minute change. This could compliment the first or second > approaches. > > Fourth, we could back out of the changes to published APIs and convert > them back to CObjects. -1. > > > Your thoughts? More later, have to run. For now, I'm with Antoine on this one: * adding support for PyCapsules to 2.7 is fine. * exposing the various module C APIs as PyCapsules in addition to the existing PyCObject C APIS is fines as well and indeed a good idea, since it makes porting 2.7 applications to 3.1 easier. * adding more C APIs using PyCapsules in Python 2.7 is fine as well, e.g. for the decimal module. * removing the PyCObject C APIs is no option in Python 2.7. Please remember that you're dealing with Python 2.7. Major backwards incompatible changes are not allowed in that branch and you're pretty much breaking all 3rd party modules that have just started using e.g the datetime module C API. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 25 2010) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ___
Re: [Python-Dev] [Python-checkins] PyCapsule backport
Hello Larry, > You're right, my changes aren't backwards compatible. I thought it was > reasonable for four reasons: > > 1. The CObject API isn't safe. It's easy to crash Python 2.6 in just a > few lines by mixing and matching CObjects. Switching Python to capsules > prevents a class of exploits. I've included a script at the bottom of > this message that demonstrates three such crashes. The script runs in > Python 2 and 3, but 3.1 doesn't crash because it's using capsules. Ok, but this is IMHO not a good enough reason to break the API. No prior announcement (prior to 2.7) has been made that PyCObject APIs would be replaced by their PyCapsule equivalents. You must leave people time to adapt. > 2. As I just mentioned, Python 3.1 already uses capsules everywhere > instead of CObjects. Since part of the purpose of Python 2.7 is to > prepare developers for the to upgrade to 3.1, getting them to switch to > capsules now is just one more way they are prepared. Not /forcing them/ to switch, though. If you agree to put a bit more work into it, and make the PyCObject APIs coexist with the corresponding PyCapsule APIs in 2.7, then I think it would be fine. The point where we force people to switch their APIs is the 2.x -> 3.x transition. That's the whole point of breaking compatibility at one defined point. If we force them to switch when migrating to 2.7 instead, it becomes messy and quite unfriendly to them. > 3. Because CObject is unsafe, I want to deprecate it in 2.7, and if we > ever made a 2.8 I want to remove it completely. I don't think there is a point in removing it in 2.8. If 2.8 ever exists, its purpose will be to serve users who don't /want/ to switch to 3.x yet, so breaking compatibility for them kind of ruins the purpose. > First, we could do as Antoine Pitrou suggests on the bug (issue 7992): > wherever the CObject used to be published as a module attribute to > expose an API, we could provide both a CObject and a capsule; internally > Python would only use the capsules. [...] > > Second, we could make CObject internally support unpacking capsules. If > you gave a capsule to PyCObject_AsVoidPtr() it would unpack it and > return the pointer within. [...] > > Third, I've been pondering writing a set of preprocessor macros, shipped > in their own header file distributed independently of Python and > released to the public domain, that would make it easy to use either > CObjects or capsules depending on what version of Python you were > compiling against. Obviously, using these macros would require a source > code change in the third-party library. But these macros would make it > a five-minute change. This could compliment the first or second approaches. > > Fourth, we could back out of the changes to published APIs and convert > them back to CObjects. -1. I think solution #1 would be the best one. #2 looks too complicated. #3 is not much friendlier than the current compatibility breakage. #4 is the necessary fallback if you don't want to implement #1. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r79397 - in python/trunk: Doc/c-api/capsule.rst Doc/c-api/cobject.rst Doc/c-api/concrete.rst Doc/data/refcounts.dat Doc/extending/extending.rst Include/Python.h Incl
M.-A. Lemburg wrote: [Y]ou're pretty much breaking all 3rd party modules that have just started using e.g the datetime module C API. Not if they're using PyDateTime_IMPORT. My understanding is that requiring a recompile is okay, and if you use PyDateTime_IMPORT and recompile against 2.7 it works fine. As I already mentioned, the exception to this is pyexpat, which provides no such importer macro / function. I shall await your more fully-fleshed out reply, /larry/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r79397 - in python/trunk: Doc/c-api/capsule.rst Doc/c-api/cobject.rst Doc/c-api/concrete.rst Doc/data/refcounts.dat Doc/extending/extending.rst Include/Python.h Incl
On Thu, Mar 25, 2010 at 2:16 PM, Larry Hastings wrote: > My understanding is that requiring a recompile is okay This has always been a point of contention. I'm not even sure what the current official position is. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] PyCapsule backport
Just a reply to one part of your message, as I really need to get back
to work for now.
Antoine Pitrou wrote:
I think solution #1 would be the best one.
#2 looks too complicated.
#2 is a seven-line diff, below.
/larry/
Index: cobject.c
===
--- cobject.c (revision 79396)
+++ cobject.c (working copy)
@@ -50,7 +50,12 @@
PyCObject_AsVoidPtr(PyObject *self)
{
if (self) {
-if (self->ob_type == &PyCObject_Type)
+if (PyCapsule_CheckExact(self)) {
+const char *name = PyCapsule_GetName(self);
+const void *value = PyCapsule_GetPointer(self, name);
+return value;
+}
+else if (self->ob_type == &PyCObject_Type)
return ((PyCObject *)self)->cobject;
PyErr_SetString(PyExc_TypeError,
"PyCObject_AsVoidPtr with non-C-object");
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r79397 - in python/trunk: Doc/c-api/capsule.rst Doc/c-api/cobject.rst Doc/c-api/concrete.rst Doc/data/refcounts.dat Doc/extending/extending.rst Include/Python.h Incl
M.-A. Lemburg, 25.03.2010 10:10: larry.hastings wrote: Author: larry.hastings Date: Thu Mar 25 01:54:54 2010 New Revision: 79397 Log: Backported PyCapsule from 3.1, and converted most uses of CObject to PyCapsule. Backporting PyCapsule is fine, but the changes you made to all those PyCObject uses does not look backwards compatible. I got this in Cython's continuous integration server after this change: Traceback (most recent call last): File "runtests.py", line 19, in import cPickle as pickle TypeError: PyCObject_AsVoidPtr with non-C-object Although this looks like an internal problem in CPython, I'm certainly against any backwards incompatible changes of this size in the 2.x series. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On 3/25/2010 8:13 AM, Mark Dickinson wrote: On Thu, Mar 25, 2010 at 3:05 PM, Nick Coghlan wrote: Mark Dickinson wrote: On Thu, Mar 25, 2010 at 2:08 PM, Nick Coghlan wrote: Jesus Cea wrote: But IEEE 754 was created by pretty clever guys and sure they had a reason for define things in the way they are. Probably we are missing something. Yes, this is where their "implementable in a hardware circuit" focus comes in. They were primarily thinking of a floating point representation where the 32/64 bits are *it* - you can't have "multiple NaNs" because you don't have the bits available to describe them. I'm not so sure about this: standard 64-bit binary IEEE 754 doubles allow for 2**53-2 different nans (2**52-2 signaling nans, 2**52 quiet nans): anything with bit pattern (msb to lsb) x111 is an infinity or a nan, and there are only 2 infinities. I stand corrected :) It still seems to me that the problems mostly arise when we're trying to get floats and Decimals to behave like Python *objects* (i.e. with reflexive equality) rather than like IEEE defined numbers. It's an extra element that isn't part of the problem the numeric standards are trying to solve. Agreed. We don't have to be "missing something"; rather, the IEEE folks (quite understandably) almost certainly didn't anticipate this kind of usage. So I'll concede that it's reasonable to consider deviating from the standard in the light of this. It is my understand that even bit-for-bit identical NaN values will compare unequal according to IEEE 754 rules. I would have no problem with Python interning each encountered NaN value, to avoid having bit-for-bit identical NaN values with different Python IDs, but having them compare equal seems inappropriate. Glenn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 6081: format string using mapping rather than kwargs
>> I'm inclined to leave it alone unless/until Raymond or somebody else >> steps up to really champion it. > > I'm okay with that. And I am looking for another bug, that I could get some python-fu from ;-) -- Filip Gruszczyński ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Thu, 25 Mar 2010 10:25:35 pm Nick Coghlan wrote:
> Steven D'Aprano wrote:
> > I'd like to turn the question around ... what algorithms are there
> > that rely on NaN == NaN being True?
>
> Absolutely anything that expects "x is y" to imply that "x == y". The
> builtin containers enforce this by checking identity before they
> check equality, but there are plenty of algorithms that won't.
Fair point, but I was actually thinking about mathematical algorithms.
Builtin containers may also fail with any object that violates the
expectation that identity implies equality, e.g.:
class AlwaysDifferent:
def __eq__(self, other):
return False
def __ne__(self, other):
return True
I don't see this as a problem -- if you choose to use such objects,
you're responsible for understanding what is going on. If you choose to
use floats, then you need to understand that NANs are weird.
Personally, I'm less concerned about sets of floats ending up with
strange combinations of NANs than I am about the possibility of
disastrous maths errors caused by allowing NANs to test as equal.
Here's a simplistic example:
def func(a, b):
if a == b:
return 1.0
return math.sin(a*b)**(a-b)
(I say "simplistic" because it currently fails for a=b=INF.) Currently
this function will do the right thing for a, b both NANs:
>>> func(float('nan'), float('nan'))
nan
but making NANs test as equal will cause it to give the wrong answer. I
fear that kind of error far more than the current funny behaviour of
builtin containers with NANs.
--
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Georg Brandl wrote:
Thinking of each value created by float('nan') as
a different nan makes sense to my naive mind, and it also explains
nicely the behavior present right now.
Not entirely:
x = float('NaN')
y = x
if x == y:
...
There it's hard to argue that the NaNs being compared
result from different operations.
It does suggest a potential compromise, though: a single
NaN object compares equal to itself, but different NaN
objects are never equal (more or less what dict membership
testing does now, but extended to all == comparisons).
Whether that's a *sane* compromise I'm not sure.
--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Steven D'Aprano pearwood.info> writes: > > Personally, I'm less concerned about sets of floats ending up with > strange combinations of NANs than I am about the possibility of > disastrous maths errors caused by allowing NANs to test as equal. > Here's a simplistic example: You just said "if you choose to use floats, then you need to understand that NANs are weird". I wonder why this saying shouldn't apply to your "simplistic example" of NAN usage. (is your example even from real life?) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Thu, Mar 25, 2010 at 04:18, Steven D'Aprano wrote: > def myfunc(x, y): > if x == y: > return 1.0 > else: > return something_complicated**(x-y) > > > Optimising floating point code is fraught with dangers (the above fails > for x=y=INF as well as NAN) but anything that make Not A Numbers > pretend to be numbers is a bad thing. What about this: def myfunc(x): if x >= THRESHOLD: return 1.0 else: return something_complicated(x) If one behaves right it's more likely a fluke, not a designed in feature. It's certainly not obvious without covering every comparison with comments. Maybe that's the solution. Signal by default on comparison, but add a collection of naneq/naneg/etc functions (math module, methods, whatever) that use a particular quiet mapping, making the whole thing explicit? -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 12:31 PM, Glenn Linderman wrote: > It is my understand that even bit-for-bit identical NaN values will compare > unequal according to IEEE 754 rules. > > I would have no problem with Python interning each encountered NaN value, to > avoid having bit-for-bit identical NaN values with different Python IDs, but > having them compare equal seems inappropriate. Let's please not intern NaNs. Interning is for performance. If you have enough NaNs to affect your performance I think you have bigger worries! -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Am 25.03.2010 22:45, schrieb Greg Ewing:
> Georg Brandl wrote:
>> Thinking of each value created by float('nan') as
>> a different nan makes sense to my naive mind, and it also explains
>> nicely the behavior present right now.
>
> Not entirely:
>
>x = float('NaN')
>y = x
>if x == y:
> ...
>
> There it's hard to argue that the NaNs being compared
> result from different operations.
>
> It does suggest a potential compromise, though: a single
> NaN object compares equal to itself, but different NaN
> objects are never equal (more or less what dict membership
> testing does now, but extended to all == comparisons).
>
> Whether that's a *sane* compromise I'm not sure.
FWIW, I like it.
Georg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Mar 25, 2010, at 4:21 PM, Georg Brandl wrote:
> Am 25.03.2010 22:45, schrieb Greg Ewing:
>> Georg Brandl wrote:
>>> Thinking of each value created by float('nan') as
>>> a different nan makes sense to my naive mind, and it also explains
>>> nicely the behavior present right now.
>>
>> Not entirely:
>>
>> x = float('NaN')
>> y = x
>> if x == y:
>> ...
>>
>> There it's hard to argue that the NaNs being compared
>> result from different operations.
>>
>> It does suggest a potential compromise, though: a single
>> NaN object compares equal to itself, but different NaN
>> objects are never equal (more or less what dict membership
>> testing does now, but extended to all == comparisons).
>>
>> Whether that's a *sane* compromise I'm not sure.
>
> FWIW, I like it.
>
> Georg
>
+1
Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, 25 Mar 2010 06:26:11 am Mark Dickinson wrote:
> Here's an interesting recent blog post on this subject, from the
> creator of Eiffel:
>
> http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-
>civilization/
Sorry, but he lost me right at the beginning when he quoted someone
else:
"there is no reason to believe that the result of one
calculation with unclear value should match that of
another calculation with unclear value"
and then argued:
"The exact same argument can be used to assert that the
result should not be False:
… there is no reason to believe that the result of one
calculation with unclear value should not match that of
another calculation with unclear value.
Just as convincing! Both arguments complement each other:
there is no compelling reason for demanding that the
values be equal; and there is no compelling argument either
to demand that they be different. If you ignore one of the
two sides, you are biased."
This whole argument is invalid on at least three levels. I'll get the
first two out the way briefly
#1: Bertrand starts by treating NANs as "unclear values", and concludes
that we shouldn't prefer "two unclear values are different" as more
compelling than "two unclear values are the same". But this is
ridiculous -- if you ask me a pair of questions, and I answer "I'm not
sure" to both of them, why would you assume that the right answer to
both questions is actually the same?
#2: But in fact NANs aren't "unclear values", they are not values at
all. The answer to "what is the non-complex logarithm of -1?" is
not "I'm not sure" but "there is no such value". Bertrand spends an
awful lot of time trying to demonstrate why the reflexivity of equality
(every x is equal to itself) should apply to NANs as well as the other
floats, but RoE is a property of equivalence relations, which does not
(and should not) hold for "there is no such value".
By analogy: the Lizard King of Russia does not exist; the Vampire Queen
of New Orleans also does not exist. We don't therefore conclude that
the Lizard King and the Vampire Queen are therefore the same person.
#3: We could, if we wish, violate the IEEE standard and treat equality
of NANs as an equivalence relation. It's our language, we're free to
follow whatever standards we like, and reflexivity of equality is a
very useful axiom to have. Since it applies to all non-NAN floats (and
virtually every object in Python, other than those with funny __eq__
methods), perhaps we should extend it to NANs as well?
I hope to convince you that the cost of doing so is worse than the
disease. Since NANs are usually found in mathematical contexts, we
should follow the IEEE standard even at the cost of rare anomalies in
non-mathematical code containing NANs.
Simply put: we should treat "two unclear values are different" as more
compelling than "two unclear values are the same" as it leads to fewer,
smaller, errors. Consider:
log(-1) = NAN # maths equality, not assignment
log(-2) = NAN
If we allow NAN = NAN, then we permit the error:
log(-1) = NAN = log(-2)
therefore log(-1) = log(-2)
and 1 = 2
But if make NAN != NAN, then we get:
log(-1) != log(-2)
and all of mathematics does not collapse into a pile of rubble. I think
that is a fairly compelling reason to prefer inequality over equality.
One objection might be that while log(-1) and log(-2) should be
considered different NANs, surely NANs should be equal to themselves?
-1 = -1
implies log(-1) = log(-1)
But consider the practicalities: there are far more floats than
available NAN payloads. We simply can't map every invalid calculation
to a unique NAN, and therefore there *must* be cases like:
log(-123.456789e-8) = log(-9.876e47)
implies 123.456789e-8 = 9.876e47
So we mustn't consider NANs equal just because their payloads are equal.
What about identity? Even if we don't dare allow this:
x = log(-1) # assignment
y = log(-1) # another NAN with the same payload
assert x is not y
assert x == y
surely we can allow this?
assert x == x
But this is dangerous. Don't be fooled by the simplicity of the above
example. Just because you have two references to the same (as in
identity) NAN, doesn't mean they represent "the same thing" or came
from the same place:
data = [1, 2, float('nan'), float('nan'), 3]
x = harmonic_mean(data)
y = 1 - geometric_mean(data)
It is an accident of implementation whether x and y happen to be the
same object or not. Why should their inequality depend on such a
fragile thing?
In fact, identity of NANs is itself an implementation quirk of
programming languages like Python: logically, NANs don't have identity
at all.
To put it another way: all ONEs are the same ONE, even if they come from
different sources, are in different memory locations, or have different
identities; but all NANs are different, even if they come from the same
source, are in the
Re: [Python-Dev] Why is nan != nan?
On Fri, 26 Mar 2010 09:45:51 am Greg Ewing wrote:
> Georg Brandl wrote:
> > Thinking of each value created by float('nan') as
> > a different nan makes sense to my naive mind, and it also explains
> > nicely the behavior present right now.
>
> Not entirely:
>
>x = float('NaN')
>y = x
>if x == y:
> ...
>
> There it's hard to argue that the NaNs being compared
> result from different operations.
But unlike us, the equality operator only has a pinhole view of the
operands. It can't distinguish between your example and this:
x = float('nan')
y = some_complex_calculation(x)
if x == y:
...
where y merely happens to end up with the same object as x by some quirk
of implementation.
> It does suggest a potential compromise, though: a single
> NaN object compares equal to itself, but different NaN
> objects are never equal (more or less what dict membership
> testing does now, but extended to all == comparisons).
>
> Whether that's a *sane* compromise I'm not sure.
What do we do with Decimal? Aren't we committed to matching the Decimal
standard, in which case aren't we committed to this?
x = Decimal('nan')
assert x != x
If that's the case, then float NANs and Decimal NANs will behave
differently. I think that's a mistake.
For what it's worth, I'm -1 on allowing NANs to test equal to any NAN,
including itself. However, I would be -0 on the following compromise:
Allow NANs to test equal to themselves (identity testing).
math module to grow a function ieee_equals(x, y) that keeps the current
behaviour.
--
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Steven D'Aprano wrote: I'd like to turn the question around ... what algorithms are there that rely on NaN == NaN being True? That seems to be a straw question, since AFAIK nobody has suggested that there are any such algorithms. On the other hand, it has been claimed that some algorithms exist that benefit from Nan == NaN being false, so it's fair to ask for examples to back that up. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
At 11:57 AM 3/26/2010 +1100, Steven D'Aprano wrote: But they're not -- they're *signals* for "your calculation has gone screwy and the result you get is garbage", so to speak. You shouldn't even think of a specific NAN as a piece of specific garbage, but merely a label on the *kind* of garbage you've got (the payload): INF-INF is, in some sense, a different kind of error to log(-1). In the same way you might say "INF-INF could be any number at all, therefore we return NAN", you might say "since INF-INF could be anything, there's no reason to think that INF-INF == INF-INF." So, are you suggesting that maybe the Pythonic thing to do in that case would be to cause any operation on a NAN (including perhaps comparison) to fail, rather than allowing garbage to silently propagate? In other words, if NAN is only a signal that you have garbage, is there really any reason to keep it as an *object*, instead of simply raising an exception? Then, you could at least identify what calculation created the garbage, instead of it percolating up through other calculations. In low-level languages like C or Fortran, it obviously makes sense to represent NAN as a value, because there's no other way to represent it. But in a language with exceptions, is there a use case for it existing as a value? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
> -1 = -1 > implies log(-1) = log(-1) Mathematically speaking this is incorrect. x = y implies log(x) = log(y) for x > 0 and y > 0 Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/26/2010 01:57 AM, Steven D'Aprano wrote: > In fact, identity of NANs is itself an implementation quirk of > programming languages like Python: logically, NANs don't have identity > at all. > > To put it another way: all ONEs are the same ONE, even if they come from > different sources, are in different memory locations, or have different > identities; but all NANs are different, even if they come from the same > source, are in the same memory location, or have the same identity. +inf. Bravo!. :-) - -- Jesus Cea Avion _/_/ _/_/_/_/_/_/ j...@jcea.es - http://www.jcea.es/ _/_/_/_/ _/_/_/_/ _/_/ jabber / xmpp:j...@jabber.org _/_/_/_/ _/_/_/_/_/ . _/_/ _/_/_/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/ "My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBS6wP7Jlgi5GaxT1NAQJXFgP+KHshsGGOM1GO/f1lEazul1+vipw6tUuF Hf8Pm39srX3pKV7OUabWJWBaRURnJWIHEymjwILBwVo9X4604klbTRi3MrqbOeZv G8NFeMzagXLwEOlAnPbVcfDb3KcW4C/Zm0A5TOPY4X5T/8FRmhfbrC4Ip2klyyMh 24fGUWTr5DI= =d1RL -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On Thu, Mar 25, 2010 at 18:57, Steven D'Aprano wrote: > Simply put: we should treat "two unclear values are different" as more > compelling than "two unclear values are the same" as it leads to fewer, > smaller, errors. Consider: > > log(-1) = NAN # maths equality, not assignment > log(-2) = NAN > > If we allow NAN = NAN, then we permit the error: > > log(-1) = NAN = log(-2) > therefore log(-1) = log(-2) > and 1 = 2 > > But if make NAN != NAN, then we get: > > log(-1) != log(-2) > > and all of mathematics does not collapse into a pile of rubble. I think > that is a fairly compelling reason to prefer inequality over equality. > > One objection might be that while log(-1) and log(-2) should be > considered different NANs, surely NANs should be equal to themselves? > > -1 = -1 > implies log(-1) = log(-1) IMO, this just shows how ludicrous it is to compare NaNs. No matter what we do it will imply some insane mathematical consequence implied and code that will break. They are, after all, an error passed silently. Why is it complex can raise an exception when sorted, forcing you to use a sane (and explicit) method, but for NaN it's okay to silently fail? -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On 3/25/2010 4:14 PM, Guido van Rossum wrote: On Thu, Mar 25, 2010 at 12:31 PM, Glenn Linderman wrote: It is my understand that even bit-for-bit identical NaN values will compare unequal according to IEEE 754 rules. I would have no problem with Python interning each encountered NaN value, to avoid having bit-for-bit identical NaN values with different Python IDs, but having them compare equal seems inappropriate. Let's please not intern NaNs. Interning is for performance. If you have enough NaNs to affect your performance I think you have bigger worries! I'm OK with interning NaNs, and I'm OK with not interning NaNs. But I would much prefer to see Python conform to the IEEE 754 for float, and the Decimal standard for Decimal, where at all possible, including keeping NaNs not comparing equal to anything, themselves included. And retaining a mode where Decimal is standoffish, and doesn't accept non-Decimal comparisons or arithmetic seems like a big win for program correctness to me. Glenn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Curt Hagenlocher wrote: Wait, what? I haven't been paying much attention, but this is backwards. There are multiple representations of NaN in the IEEE encoding; I think Nick's point is that there aren't enough bits to give the result of every operation its own unique NaN. The payload of a NaN in typical hardware implementations is quite small, because it has to fit into the exponent field. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Steven D'Aprano wrote: By analogy: the Lizard King of Russia does not exist; the Vampire Queen of New Orleans also does not exist. We don't therefore conclude that the Lizard King and the Vampire Queen are therefore the same person. But it's equally invalid to say that they're *not* the same person, because it's meaningless to say *anything* about the properties of nonexistent people. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Steven D'Aprano wrote: What do we do with Decimal? Aren't we committed to matching the Decimal standard, It's been pointed out that the Decimal standard only defines some abstract operations, and doesn't mandate that they be mapped onto any particular language syntax. That gives us enough flexibility to make == do what we want and still claim compliance with the standard. BTW, does IEEE754 give us the same flexibility? If so, we may not have much of a problem in the first place. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
I impulsively wrote: The payload of a NaN in typical hardware implementations is quite small, because it has to fit into the exponent field. ...which turns out to be precisely wrong. Some day I'll learn to wait until somebody else in the thread has checked the facts for me before posting. :-) -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
Steven D'Aprano writes:
> But unlike us, the equality operator only has a pinhole view of the
> operands. It can't distinguish between your example and this:
>
> x = float('nan')
> y = some_complex_calculation(x)
> if x == y:
> ...
>
> where y merely happens to end up with the same object as x by some quirk
> of implementation.
Note that Mark has already provided a related example of such a quirk
(Decimal(-1).sqrt(), I think it was).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why is nan != nan?
On 3/25/2010 9:35 PM, Greg Ewing wrote: Steven D'Aprano wrote: What do we do with Decimal? Aren't we committed to matching the Decimal standard, It's been pointed out that the Decimal standard only defines some abstract operations, and doesn't mandate that they be mapped onto any particular language syntax. That gives us enough flexibility to make == do what we want and still claim compliance with the standard. BTW, does IEEE754 give us the same flexibility? If so, we may not have much of a problem in the first place. I propose that the abstract Decimal operation for addition be mapped to the syntax of operator - and that the abstract Decimal operation for subtraction be mapped to the syntax of operator +. Then people will have to actually read the manual to learn how to use the Decimal type in Python, rather than assuming that things might work the way they expect. This will lead to more robust and correct programs, because people will have read the manual. Or at least it seems that it should work that way... ˚͜ ˚ (Hmm, that might not render consistently for everyone, so I'll throw in a couple :) :) also.) Glenn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
