Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Tue, Feb 18, 2014 at 5:10 PM, Mark Lawrence breamore...@yahoo.co.uk wrote: Sorry if this has already been suggested, but why not introduce a new singleton to make the database people happier if not happy? To avoid confusion call it dbnull? A reasonable compromise or complete cobblers? :) I think this is possible already, for the database people. The problem is that it will not pass the is None test, which at the very least is not backwards compatible with how they have used it before. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/19/2014 2:53 AM, Lennart Regebro wrote: On Tue, Feb 18, 2014 at 5:10 PM, Mark Lawrence breamore...@yahoo.co.uk wrote: Sorry if this has already been suggested, but why not introduce a new singleton to make the database people happier if not happy? To avoid confusion call it dbnull? A reasonable compromise or complete cobblers? :) I think this is possible already, for the database people. The problem is that it will not pass the is None test, which at the very least is not backwards compatible with how they have used it before. The new singleton will be called something else, likely with Null in the name, so that's what I'll call it here... Null. So when switching from None to Null, you must also switch from is None to is Null. Of course it is not backwards compatible... but once all the database related None usage is switched to Null usage it should work the same as before, but with proper (for some database's definition of proper) semantics. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 20/02/14 08:23, Glenn Linderman wrote: Of course it is not backwards compatible... but once all the database related None usage is switched to Null usage it should work the same as before, My problem with this is that there is no clear distinction between database-related None usage and None usage in general. Data retrieved from a database is often passed to other code that has nothing to do with databases, and data received from other code is inserted into databases. Somewhere in between someone is going to have to be careful to translate back and forth between Nones and Nulls. This is a recipe for chaos. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
Greg Ewing writes: Data retrieved from a database is often passed to other code that has nothing to do with databases, and data received from other code is inserted into databases. Somewhere in between someone is going to have to be careful to translate back and forth between Nones and Nulls. Sure. The suggestion is to assign responsibility for such careful translation to implementers of the DB API. This is a recipe for chaos. If it is, then chaos is already upon us because you can't be careful even if you want to. I don't know if fixing it is worth the work and confusion involved in the fixing process, but fixing it will conserve (and maybe reduce) chaos, not create it. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 18 February 2014 07:35, Greg Ewing greg.ew...@canterbury.ac.nz wrote:
If you don't want to touch comparison in general,
how about an option to sort()?
results = sorted(invoices, key=attrgetter('duedate'), none='first')
Or alternatively, a default on None function - Oracle SQL calls this
nvl, so I will too:
def nvl(x, dflt):
return dflt if x is None else x
results = sorted(invoices, key=lambda x: nvl(x.duedate, datetime(MINYEAR,1,1))
Admittedly the key function is starting to get complex, but I find
that key functions often do - they are the one big area where Python
uses a lot of functional-style constructs. The existence of
itemgetter/attrgetter are a testament to this fact.
PS isn't this python-ideas territory by now?
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Tue, Feb 18, 2014 at 8:35 AM, Greg Ewing greg.ew...@canterbury.ac.nz wrote:
If you don't want to touch comparison in general,
how about an option to sort()?
results = sorted(invoices, key=attrgetter('duedate'), none='first')
I think this is a much better option.
//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
Am 18.02.2014 08:35, schrieb Greg Ewing: Tim Peters wrote: [Greg Ewing] often one wants to sort a collection of objects having keys that can take on null values. Perhaps that's often true of your code, but it's never been true of mine. It's fairly common in accounting circles. I have a collection of invoices, each of which can have a due date specified, but doesn't have to. I want to sort the invoices by due date. It's not particularly important whether the missing dates sort first or last, but it *is* important that the sort *not blow up*. Dates seem to be a particularly irksome case. Here's one suggestion from StackOverflow for dealing with the problem: import datetime mindate = datetime.date(datetime.MINYEAR, 1, 1) def getaccountingdate(x): return x['accountingdate'] or mindate results = sorted(data, key=getaccountingdate) That's a ridiculous amount of boilerplate for doing something that ought to be straightforward. Seeing how you need a key function in any case for this sort to work, it's only the or mindate added, which I can't recognize as ridiculous amount of boilerplate. Much more so since you can put the key function in a shared module and import it from there anywhere you need it. Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 18.02.2014 05:25, Tim Peters wrote: [M.-A. Lemburg] Now, the choice to have None compare less than all other objects may have been arbitrary, but IMO it was a good, consistent and useful choice. Possibly useful for some apps, sure. Not for my apps. For example, when I initialize an object attribute to None in Python 3, I _expect_ I'll get an exception if I try to use that attribute in most contexts. It makes no more sense to ask whether that attribute is, say, less than 3, than it does to add 3 to it. The exception is most useful then. More often than not, it was annoying to me that Python 2 _didn't_ whine about trying to compare None. Yes, I see your point, but please also consider that None is used in database applications as natural mapping for SQL NULL and in other data processing applications to mean no value. This is not garbage data, it's just missing information for a particular field and you can still happily process such data without having the information for these fields as values (indeed in some cases, the information whether a field has a value or not is important, e.g. for data validations and to cross check entries). You do still want to be able to sort such data, but as it stands, Python 3 doesn't allow you to, without adding an extra layer of protection against None values deep inside the structures you're sorting. So why not bring it back A huge barrier is (or should be) that Python 3 is over 5 years old now. Fiddling with the semantics of basic builtin types was possible - and even welcome - 6 years ago. Now they really shouldn't be touched in the absence of a critical bug, or a wholly backward-compatible (with Python 3) addition. and perhaps this time in a way that actually does work consistently for all Python objects by implementing the tp_richcompare slot on PyNoneType objects and documenting it ?! Something to suggest for Python 4, in which case I'll only be -0 ;-) Well, like I said: we'd be making something work again that has worked before and only remove a failure case. The barrier for entry should be lower in such a case. This is similar to bringing back the u literals. Those used to cause a SyntaxError and now they no longer do. Even better: we have a chance to properly document the behavior this time around and make it consistent all the way. The alternative would be adding a new singleton to mean mostly the same thing as None, but having the property of comparing less than all other objects and then recommend its use in the DB-API for Python 3 applications... which would break a lot of existing DB-API based apps of course... which is the reason why I'm advocating for changing None instead :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 18 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
18.02.14 10:10, Paul Moore написав(ла): Or alternatively, a default on None function - Oracle SQL calls this nvl, so I will too: def nvl(x, dflt): return dflt if x is None else x results = sorted(invoices, key=lambda x: nvl(x.duedate, datetime(MINYEAR,1,1)) Or, as was proposed above: results = sorted(invoices, key=lambda x: (x.duedate is not None, x.duedate)) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2014-02-18 13:48, Serhiy Storchaka wrote: 18.02.14 10:10, Paul Moore написав(ла): Or alternatively, a default on None function - Oracle SQL calls this nvl, so I will too: def nvl(x, dflt): return dflt if x is None else x results = sorted(invoices, key=lambda x: nvl(x.duedate, datetime(MINYEAR,1,1)) Or, as was proposed above: results = sorted(invoices, key=lambda x: (x.duedate is not None, x.duedate)) That makes me wonder. Why is: None None unorderable and not False but: (None, ) (None, ) orderable? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2014-02-18 14:11, MRAB wrote: On 2014-02-18 13:48, Serhiy Storchaka wrote: 18.02.14 10:10, Paul Moore написав(ла): Or alternatively, a default on None function - Oracle SQL calls this nvl, so I will too: def nvl(x, dflt): return dflt if x is None else x results = sorted(invoices, key=lambda x: nvl(x.duedate, datetime(MINYEAR,1,1)) Or, as was proposed above: results = sorted(invoices, key=lambda x: (x.duedate is not None, x.duedate)) That makes me wonder. Why is: None None unorderable and not False but: (None, ) (None, ) orderable? tuple's rich comparison uses PyObject_RichCompareBool(x, y, Py_EQ) to find the first pair of items that is unequal. Then it will test the order of any remaining elements. http://hg.python.org/cpython/file/79e5bb0d9b8e/Objects/tupleobject.c#l591 PyObject_RichCompareBool(x, y, Py_EQ) treats identical objects as equal. http://hg.python.org/cpython/file/79e5bb0d9b8e/Objects/object.c#l716 -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/18/2014 12:11 AM, Greg Ewing wrote: Nobody is asking for a return to the arbitrary-but- [in]consistent mess of Python 2, only to bring back *one* special case, i.e. None comparing less than everything else. For a None, that is only the fallback rule if a does not handle the comparison. The result is a mess, including a possible inconsistency between direct comparison and cmp. See my previous posts. 'Bringing back' what was or an improved version would be a semantic change that could break code and would require a two-version deprecation period. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/18/2014 2:35 AM, Greg Ewing wrote:
results = sorted(invoices, key=attrgetter('duedate'), none='first')
I think this is the best idea on the thread. As a pure enhancement, it
could be added in 3.5. The only tricky part of the implementation is
maintaining stability of the sort. The obvious implementation of
swapping Nones with objects at one end would break that. Instead, a scan
listing the positions of Nones should be followed by a series of block
moves of objects (pointers) between the Nones. It would still be O(n).
--
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
I'll reply inline, but tl;dr: I'm with Tim on this. On Tue, Feb 18, 2014 at 5:08 AM, M.-A. Lemburg m...@egenix.com wrote: On 18.02.2014 05:25, Tim Peters wrote: [M.-A. Lemburg] Now, the choice to have None compare less than all other objects may have been arbitrary, but IMO it was a good, consistent and useful choice. Possibly useful for some apps, sure. Not for my apps. For example, when I initialize an object attribute to None in Python 3, I _expect_ I'll get an exception if I try to use that attribute in most contexts. It makes no more sense to ask whether that attribute is, say, less than 3, than it does to add 3 to it. The exception is most useful then. More often than not, it was annoying to me that Python 2 _didn't_ whine about trying to compare None. Yes, I see your point, but please also consider that None is used in database applications as natural mapping for SQL NULL and in other data processing applications to mean no value. Fine, but not everything uses a database. =) This is not garbage data, it's just missing information for a particular field and you can still happily process such data without having the information for these fields as values (indeed in some cases, the information whether a field has a value or not is important, e.g. for data validations and to cross check entries). You do still want to be able to sort such data, but as it stands, Python 3 doesn't allow you to, without adding an extra layer of protection against None values deep inside the structures you're sorting. Which in my opinion is fine as the boilerplate is typically minimal. No one has said any solution is going to take even 10 lines of code to work around this. So why not bring it back A huge barrier is (or should be) that Python 3 is over 5 years old now. Fiddling with the semantics of basic builtin types was possible - and even welcome - 6 years ago. Now they really shouldn't be touched in the absence of a critical bug, or a wholly backward-compatible (with Python 3) addition. and perhaps this time in a way that actually does work consistently for all Python objects by implementing the tp_richcompare slot on PyNoneType objects and documenting it ?! Something to suggest for Python 4, in which case I'll only be -0 ;-) Well, like I said: we'd be making something work again that has worked before and only remove a failure case. The barrier for entry should be lower in such a case. But this is still a semantic change. Python 3 users could very well be relying on the fact that comparing against None raises an exception as a way to detect when erroneous data has leaked into some data structure. What is being suggested is a semantic change to a core data type within a major release. We waited until Python 3 to make comparisons against disparate types illegal for a reason. This is similar to bringing back the u literals. Those used to cause a SyntaxError and now they no longer do. I don't think they are similar at all. Allowing the u prefix on strings basically made something that was a typo or accidental hold-over from Python 2 just not throw an error. There was no semantic shift in what u meant in Python 3.2 vs. 3.3 which could break some code. Basically we just made some syntactic fluff not upset the parser which in no way changed the semantic meaning of the string literal it was attached to. Making None sort a specific way is beyond just adding some syntactic fluff. Even better: we have a chance to properly document the behavior this time around and make it consistent all the way. The alternative would be adding a new singleton to mean mostly the same thing as None, but having the property of comparing less than all other objects and then recommend its use in the DB-API for Python 3 applications... which would break a lot of existing DB-API based apps of course... which is the reason why I'm advocating for changing None instead :-) Or DB-API stuff needs to be updated to use some None-that-sorts singleton if they want to use None to represent NULL but sort in a specific way. As Tim has said, 3.0 has been out for five years, so we are talking about breaking code for this over what can be viewed as a minor inconvenience for DB code. The current situation is not insurmountable in any way for those that want None to sort. I should also mention I view None as representing nothing, which includes not sharing a type with anything, which is why it can't be compared against any other type in Python 3. But this suggestion of having None sort as the lowest thing no matter what in way says None is implicitly every type when it comes to sorting which breaks that mental model of None representing the void of information by being void of value, but everything when it comes to its type for comparison purposes. ___ Python-Dev mailing list Python-Dev@python.org
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/18/2014 12:32 AM, Greg Ewing wrote: Terry Reedy wrote: To make None a true bottom object, the rich comparison methods would have to special-case None as either argument before looking at the __rc__ special methods of either. I don't think it would be necessary to go that far. It is if you want None to be 'a true bottom object'. If you think something less is sufficient, or even desirable, then yes, ... It would be sufficient to put the special case *after* giving both operations a chance to handle the operation via the type slots. That would more or less reproduce the 2.x situation, except that a conflict between and cmp would not be possible. Well-behaved objects generally wouldn't go out of their way to defeat that, but they could do so if necessary, e.g. to create a Bottom() that compares less than None. Although once None becomes usable as a bottom in most cases, there shouldn't be much need for people to introduce their own bottoms. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 18 February 2014 15:53, Terry Reedy tjre...@udel.edu wrote:
On 2/18/2014 2:35 AM, Greg Ewing wrote:
results = sorted(invoices, key=attrgetter('duedate'), none='first')
I think this is the best idea on the thread. As a pure enhancement, it could
be added in 3.5. The only tricky part of the implementation is maintaining
stability of the sort. The obvious implementation of swapping Nones with
objects at one end would break that. Instead, a scan listing the positions
of Nones should be followed by a series of block moves of objects (pointers)
between the Nones. It would still be O(n).
This only works if the list entry is a simple None. If the None is
embedded in e.g. a tuple then it would fail:
(1, 2, 3) (1, 2, None)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unorderable types: int() NoneType()
Oscar
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 18/02/2014 15:45, Terry Reedy wrote: On 2/18/2014 12:11 AM, Greg Ewing wrote: Nobody is asking for a return to the arbitrary-but- [in]consistent mess of Python 2, only to bring back *one* special case, i.e. None comparing less than everything else. For a None, that is only the fallback rule if a does not handle the comparison. The result is a mess, including a possible inconsistency between direct comparison and cmp. See my previous posts. 'Bringing back' what was or an improved version would be a semantic change that could break code and would require a two-version deprecation period. Sorry if this has already been suggested, but why not introduce a new singleton to make the database people happier if not happy? To avoid confusion call it dbnull? A reasonable compromise or complete cobblers? :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 02/18/2014 02:08 AM, M.-A. Lemburg wrote: This is not garbage data, it's just missing information for a particular field [...] I think the basic problem is that None was over-used (or even mis-used?) to the point where 3.0 had to make a choice to bring consistency back to the language. It seems to me what we really need is either a Null singleton to represent a data point with no known value but that can still be sorted, or have every data type able to represent a no-value state (datetime, I'm looking at you!). -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
18.02.14 19:20, Ethan Furman написав(ла): It seems to me what we really need is either a Null singleton to represent a data point with no known value but that can still be sorted, or have every data type able to represent a no-value state (datetime, I'm looking at you!). A Never singleton? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/18/2014 7:57 AM, Brett Cannon wrote: Yes, I see your point, but please also consider that None is used in database applications as natural mapping for SQL NULL and in other data processing applications to mean no value. Fine, but not everything uses a database. =) Python None and SQL NULL share some properties, which makes it seem like it is a natural mapping. However, they also have some differing properties, which means that really it isn't a natural mapping. A big mistake in most Python/SQL marriages is the failure to implement an SQL NULL class that fully matches SQL semantics. Of course it is easier to map SQL NULL to Python None and then complain about the semantic differences. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
Brett Cannon br...@python.org writes: Yes, I see your point, but please also consider that None is used in database applications as natural mapping for SQL NULL and in other data processing applications to mean no value. Fine, but not everything uses a database. =) Right, and even if everything did, as already said not all databases share the same default ordering rules, or have the same syntax to impose one specific direction. But more to the point, no database I know let you say WHERE somecolumn 1 and expect that when somecolumn IS NULL the condition is true... Paradoxically, if Python goes back to give a meaning to None 1, it would have an even more different semantic than most database engines out there. ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Wed, Feb 19, 2014 at 3:10 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: Sorry if this has already been suggested, but why not introduce a new singleton to make the database people happier if not happy? To avoid confusion call it dbnull? A reasonable compromise or complete cobblers? :) That would be a major change to the DB API. Possibly the best solution, though. Start off by having the default be to return None (as now) and have a flag that can be set please return sys.dbnull instead (does dbnull belong in sys?), and let that settle for a few years. Recommend that all applications explicitly set the flag, either to True or to False. Then eventually, with the full deprecation warnings, change the default to True. (Or maybe make it an error to not set it.) Then, after another long round of deprecations, drop the None behaviour from the spec altogether. But even in its early steps, that could solve the problem. Anyone who wants to sort a list of tuples that came from the database can simply set the flag (assuming their back-end supports that flag) and happily use dbnull. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
Georg Brandl wrote: Seeing how you need a key function in any case for this sort to work, it's only the or mindate added, which I can't recognize as ridiculous amount of boilerplate. Well, I don't much like having to construct a key function in the first place for something as common as sorting on an attribute. Also, in the particular case of dates, there's the annoying fact that the datetime module doesn't provide any kind of null object that can be compared with datetimes, so you have to make a fake one yourself. All of these small annoyances add up to what is, for me, a fairly big annoyance. What I'd *really* like to be able to write is: sort(invoices, keyattr = 'duedate', none = 'first') -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 18 February 2014 21:17, Greg Ewing greg.ew...@canterbury.ac.nz wrote: What I'd *really* like to be able to write is: sort(invoices, keyattr = 'duedate', none = 'first') Which is of course a pretty trivial utility function to write. But I understand your point - these trivial helpers add up over time into a bit of a burden. But fixing that (if anyone feels it's worth doing so) can be handled many ways, and changing the semantics of None is far from the simplest or most obvious. Pul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
M.-A. Lemburg wrote: The alternative would be adding a new singleton to mean mostly the same thing as None, but having the property of comparing less than all other objects and then recommend its use in the DB-API for Python 3 applications... Which I think would be a *really bad* idea, because there would then no longer be One Obvious Way to represent and process null values. E.g. you would no longer be able to write 'value is None' and trust it to work on all kinds of null value you're likely to get. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 19 Feb 2014 07:26, Paul Moore p.f.mo...@gmail.com wrote: On 18 February 2014 21:17, Greg Ewing greg.ew...@canterbury.ac.nz wrote: What I'd *really* like to be able to write is: sort(invoices, keyattr = 'duedate', none = 'first') Which is of course a pretty trivial utility function to write. But I understand your point - these trivial helpers add up over time into a bit of a burden. But fixing that (if anyone feels it's worth doing so) can be handled many ways, and changing the semantics of None is far from the simplest or most obvious. So perhaps the real answer is for someone to write a sorting helper module and put it on PyPI. Things like sort_byattr (implicitly uses attrgetter), sort_byitem (implicitly uses itemgetter), a flag argument defaulting to none_first=True, SortsLow and SortsHigh singletons, etc. Those don't need to be builtins, but collating them into a clean helper API would not only allow that module to be used directly, but also act as a reference for anyone wanting to implement the behaviour themselves. (I know, I know, we're way into python-ideas territory - it's just that so much if the thread *was* on topic for python-dev, it seems relatively pointless to split this iteration of the discussion. The *next* thread about it should definitely be on python-ideas, though) Cheers, Nick. Pul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/18/2014 1:27 PM, Greg Ewing wrote: M.-A. Lemburg wrote: The alternative would be adding a new singleton to mean mostly the same thing as None, but having the property of comparing less than all other objects and then recommend its use in the DB-API for Python 3 applications... Which I think would be a *really bad* idea, because there would then no longer be One Obvious Way to represent and process null values. E.g. you would no longer be able to write 'value is None' and trust it to work on all kinds of null value you're likely to get. Of course you couldn't... value is None would never work full Null values, only for None, like it should. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 15.02.2014 07:03, Stephen J. Turnbull wrote: M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. Python 3 breaks this notion by always raising an exception when using None in an ordered comparison, making it pretty much useless for the above purpose. Yes, there are ways around this, but none of them are intuitive. Here's a particularly nasty case: l = [(1, None), (2, None)] l.sort() l [(1, None), (2, None)] l = [(1, None), (2, None), (3, 4)] l.sort() l [(1, None), (2, None), (3, 4)] l = [(1, None), (2, None), (3, 4), (2, 3)] l.sort() Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() NoneType() -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17 February 2014 11:14, M.-A. Lemburg m...@egenix.com wrote: On 15.02.2014 07:03, Stephen J. Turnbull wrote: M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. Python 3 breaks this notion by always raising an exception when using None in an ordered comparison, making it pretty much useless for the above purpose. Yes, there are ways around this, but none of them are intuitive. Here's a particularly nasty case: l = [(1, None), (2, None)] l.sort() l [(1, None), (2, None)] l = [(1, None), (2, None), (3, 4)] l.sort() l [(1, None), (2, None), (3, 4)] l = [(1, None), (2, None), (3, 4), (2, 3)] l.sort() Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() NoneType() Maybe Python 3 should have a couple of None-like objects that compare the way you want: AlwaysComparesLess and AlwaysComparesGreater, but with better names (maybe 'PlusInfinity' and 'MinusInfinity'?). Just leave None alone, please. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com -- Gustavo J. A. M. Carneiro Gambit Research LLC The universe is always one step beyond logic. -- Frank Herbert ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17.02.2014 12:23, Gustavo Carneiro wrote: On 17 February 2014 11:14, M.-A. Lemburg m...@egenix.com wrote: On 15.02.2014 07:03, Stephen J. Turnbull wrote: M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. Python 3 breaks this notion by always raising an exception when using None in an ordered comparison, making it pretty much useless for the above purpose. Yes, there are ways around this, but none of them are intuitive. Here's a particularly nasty case: l = [(1, None), (2, None)] l.sort() l [(1, None), (2, None)] l = [(1, None), (2, None), (3, 4)] l.sort() l [(1, None), (2, None), (3, 4)] l = [(1, None), (2, None), (3, 4), (2, 3)] l.sort() Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() NoneType() Maybe Python 3 should have a couple of None-like objects that compare the way you want: AlwaysComparesLess and AlwaysComparesGreater, but with better names (maybe 'PlusInfinity' and 'MinusInfinity'?). Just leave None alone, please. This doesn't only apply to numeric comparisons. In Python 2 you can compare None with any kind of object and it always sorts first, based on the intuition that nothing is less than anything :-) FWIW, I don't think we need to invent a new name for it, just add an appropriate tp_richcompare slot to the PyNoneType or readd the special case to Object/object.c. This would also aid in porting existing Python 2 code to Python 3. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
17.02.14 13:14, M.-A. Lemburg написав(ла): Here's a particularly nasty case: l = [(1, None), (2, None)] l.sort() l [(1, None), (2, None)] l = [(1, None), (2, None), (3, 4)] l.sort() l [(1, None), (2, None), (3, 4)] l = [(1, None), (2, None), (3, 4), (2, 3)] l.sort() Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() NoneType() If you replace None to another value that cannot be compared with int (e.g. string), you will got the same nasty case. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17 February 2014 11:43, M.-A. Lemburg m...@egenix.com wrote: On 17.02.2014 12:23, Gustavo Carneiro wrote: On 17 February 2014 11:14, M.-A. Lemburg m...@egenix.com wrote: On 15.02.2014 07:03, Stephen J. Turnbull wrote: M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. Python 3 breaks this notion by always raising an exception when using None in an ordered comparison, making it pretty much useless for the above purpose. Yes, there are ways around this, but none of them are intuitive. Here's a particularly nasty case: l = [(1, None), (2, None)] l.sort() l [(1, None), (2, None)] l = [(1, None), (2, None), (3, 4)] l.sort() l [(1, None), (2, None), (3, 4)] l = [(1, None), (2, None), (3, 4), (2, 3)] l.sort() Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() NoneType() Maybe Python 3 should have a couple of None-like objects that compare the way you want: AlwaysComparesLess and AlwaysComparesGreater, but with better names (maybe 'PlusInfinity' and 'MinusInfinity'?). Just leave None alone, please. This doesn't only apply to numeric comparisons. In Python 2 you can compare None with any kind of object and it always sorts first, based on the intuition that nothing is less than anything :-) I still think that relying on your intuition is not the right way for Python. Refuse the temptation to guess. FWIW, I don't think we need to invent a new name for it, just add an appropriate tp_richcompare slot to the PyNoneType or readd the special case to Object/object.c. This would also aid in porting existing Python 2 code to Python 3. Based on your comment, SortsFirst and SortsLast sound like good names ;-) These would be universal sortable objects, that could be compared to any other type. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ -- Gustavo J. A. M. Carneiro Gambit Research LLC The universe is always one step beyond logic. -- Frank Herbert ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17 February 2014 11:49, Gustavo Carneiro gjcarne...@gmail.com wrote: FWIW, I don't think we need to invent a new name for it, just add an appropriate tp_richcompare slot to the PyNoneType or readd the special case to Object/object.c. This would also aid in porting existing Python 2 code to Python 3. Based on your comment, SortsFirst and SortsLast sound like good names ;-) These would be universal sortable objects, that could be compared to any other type. I think that having both is over-engineered. For the use cases I know of, all that is wanted is *one* object that doesn't raise a TypeError when compared with any other object, and compares predictably (always before or always after, doesn't matter which). Whether that object is None, or whether it should be a new singleton, is not obvious. The advantage of None is that it's Python 2 compatible (which aids porting as noted). The advantage of a new object is that not all uses of None need universal sortability, and indeed in some cases universal sortability will hide bugs. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17.02.2014 12:47, Serhiy Storchaka wrote: 17.02.14 13:14, M.-A. Lemburg написав(ла): Here's a particularly nasty case: l = [(1, None), (2, None)] l.sort() l [(1, None), (2, None)] l = [(1, None), (2, None), (3, 4)] l.sort() l [(1, None), (2, None), (3, 4)] l = [(1, None), (2, None), (3, 4), (2, 3)] l.sort() Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() NoneType() If you replace None to another value that cannot be compared with int (e.g. string), you will got the same nasty case. Yes, but that's not the point. Unlike strings or other mixed types that you cannot compare, None is used as placeholder in data processing as special value to mean no value available. You intentionally use such values in programming. It's not a bug to have None in a data list or as value of a variable. -- Marc-Andre Lemburg Director Python Software Foundation http://www.python.org/psf/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17.02.2014 12:49, Gustavo Carneiro wrote: On 17 February 2014 11:43, M.-A. Lemburg m...@egenix.com wrote: On 17.02.2014 12:23, Gustavo Carneiro wrote: On 17 February 2014 11:14, M.-A. Lemburg m...@egenix.com wrote: On 15.02.2014 07:03, Stephen J. Turnbull wrote: M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. Python 3 breaks this notion by always raising an exception when using None in an ordered comparison, making it pretty much useless for the above purpose. Yes, there are ways around this, but none of them are intuitive. Here's a particularly nasty case: l = [(1, None), (2, None)] l.sort() l [(1, None), (2, None)] l = [(1, None), (2, None), (3, 4)] l.sort() l [(1, None), (2, None), (3, 4)] l = [(1, None), (2, None), (3, 4), (2, 3)] l.sort() Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() NoneType() Maybe Python 3 should have a couple of None-like objects that compare the way you want: AlwaysComparesLess and AlwaysComparesGreater, but with better names (maybe 'PlusInfinity' and 'MinusInfinity'?). Just leave None alone, please. This doesn't only apply to numeric comparisons. In Python 2 you can compare None with any kind of object and it always sorts first, based on the intuition that nothing is less than anything :-) I still think that relying on your intuition is not the right way for Python. Refuse the temptation to guess. Hmm, are you sure ? There should be one-- and preferably only one --obvious way to do it. and then Although that way may not be obvious at first unless you're Dutch. which directly relates to: changeset: 16123:f997ded4e219 branch: legacy-trunk user:Guido van Rossum gu...@python.org date:Mon Jan 22 19:28:09 2001 + summary: New special case in comparisons: None is smaller than any other object :-) FWIW, I don't think we need to invent a new name for it, just add an appropriate tp_richcompare slot to the PyNoneType or readd the special case to Object/object.c. This would also aid in porting existing Python 2 code to Python 3. Based on your comment, SortsFirst and SortsLast sound like good names ;-) These would be universal sortable objects, that could be compared to any other type. Of course, it's easy to add a new type for this, but a lot of Python 2 code relies on None behaving this way, esp. code that reads data from databases, since None is the Python mapping for SQL NULL. -- Marc-Andre Lemburg Director Python Software Foundation http://www.python.org/psf/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17 Feb 2014 21:15, M.-A. Lemburg m...@egenix.com wrote: On 15.02.2014 07:03, Stephen J. Turnbull wrote: M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. This is the first I've ever heard of that sorting behaviour being an intentional feature, rather than just an artefact of Python 2 allowing arbitrarily ordered comparisons between different types. Can you point me to the relevant entry in the Python 2 language reference? Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
17.02.14 13:56, M.-A. Lemburg написав(ла):
Yes, but that's not the point. Unlike strings or other mixed types that
you cannot compare, None is used as placeholder in data processing as
special value to mean no value available.
Isn't float('nan') such placeholder?
You intentionally use such values in programming. It's not a bug to
have None in a data list or as value of a variable.
You can't have None in array('f'), you can't add or multiply by None.
Relation operators don't looks an exception here. Applying sorted() to a
list which contains numbers and Nones makes as much sense as applying
sum() to it.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17.02.2014 13:12, Nick Coghlan wrote: On 17 Feb 2014 21:15, M.-A. Lemburg m...@egenix.com wrote: On 15.02.2014 07:03, Stephen J. Turnbull wrote: M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. This is the first I've ever heard of that sorting behaviour being an intentional feature, rather than just an artefact of Python 2 allowing arbitrarily ordered comparisons between different types. Can you point me to the relevant entry in the Python 2 language reference? This is not documented anywhere in the language spec, AFAIK. It is documented in the code (Python 2.7; Object/object.c): default_3way_compare(PyObject *v, PyObject *w) ... /* None is smaller than anything */ if (v == Py_None) return -1; if (w == Py_None) return 1; Note that it's not important whether None is smaller or greater than any other object. The important aspect is that it's sorting order is consistent and doesn't raise a TypeError when doing an ordered comparison with other objects. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17.02.2014 13:19, Serhiy Storchaka wrote:
17.02.14 13:56, M.-A. Lemburg написав(ла):
Yes, but that's not the point. Unlike strings or other mixed types that
you cannot compare, None is used as placeholder in data processing as
special value to mean no value available.
Isn't float('nan') such placeholder?
You can easily construct other such placeholders, but None was intended
for this purpose:
http://docs.python.org/2.7/c-api/none.html?highlight=none#Py_None
The Python None object, denoting lack of value. ...
You intentionally use such values in programming. It's not a bug to
have None in a data list or as value of a variable.
You can't have None in array('f'), you can't add or multiply by None.
Relation operators don't looks
an exception here. Applying sorted() to a list which contains numbers and
Nones makes as much sense
as applying sum() to it.
Of course, you cannot apply any operations with None - it doesn't
have a value -, but you can compare it to other objects and it provides
a consistent behavior in Python 2. Python 3 is missing such an object.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Feb 17 2014)
Python Projects, Consulting and Support ... http://www.egenix.com/
mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53
: Try our mxODBC.Connect Python Database Interface for free ! ::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17 February 2014 12:30, M.-A. Lemburg m...@egenix.com wrote:
On 17.02.2014 13:19, Serhiy Storchaka wrote:
17.02.14 13:56, M.-A. Lemburg написав(ла):
Yes, but that's not the point. Unlike strings or other mixed types that
you cannot compare, None is used as placeholder in data processing as
special value to mean no value available.
Isn't float('nan') such placeholder?
You can easily construct other such placeholders, but None was intended
for this purpose:
http://docs.python.org/2.7/c-api/none.html?highlight=none#Py_None
The Python None object, denoting lack of value. ...
You intentionally use such values in programming. It's not a bug to
have None in a data list or as value of a variable.
You can't have None in array('f'), you can't add or multiply by None.
Relation operators don't looks
an exception here. Applying sorted() to a list which contains numbers
and Nones makes as much sense
as applying sum() to it.
Of course, you cannot apply any operations with None - it doesn't
have a value -, but you can compare it to other objects and it provides
a consistent behavior in Python 2. Python 3 is missing such an object.
I agree with you that Python 3 could use such an object. Just don't make
it the default. Leave None as it is.
Also I agree that my previous naming suggestions are bad. What we need is
only one (or two) additional object whose main semantic meaning is still
no value, but which also adds a meaning of comparable. Then it's a
matter of choosing a good name for it, with lots of bikeshedding involved.
Just lets not change None only because we're too lazy to discuss a proper
alternative name. Also this use case is not _that_ common, so it's ok if
it has a slightly longer name than None.
Also think of the implications of changing None at this point. It would
allow us to write programs that work Python = 3.5 and Python = 2.7, but
fail mysteriously in all other versions in between. What a mess that would
be...
--
Gustavo J. A. M. Carneiro
Gambit Research LLC
The universe is always one step beyond logic. -- Frank Herbert
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17.02.2014 14:29, Gustavo Carneiro wrote:
On 17 February 2014 12:30, M.-A. Lemburg m...@egenix.com wrote:
On 17.02.2014 13:19, Serhiy Storchaka wrote:
17.02.14 13:56, M.-A. Lemburg написав(ла):
Yes, but that's not the point. Unlike strings or other mixed types that
you cannot compare, None is used as placeholder in data processing as
special value to mean no value available.
Isn't float('nan') such placeholder?
You can easily construct other such placeholders, but None was intended
for this purpose:
http://docs.python.org/2.7/c-api/none.html?highlight=none#Py_None
The Python None object, denoting lack of value. ...
You intentionally use such values in programming. It's not a bug to
have None in a data list or as value of a variable.
You can't have None in array('f'), you can't add or multiply by None.
Relation operators don't looks
an exception here. Applying sorted() to a list which contains numbers
and Nones makes as much sense
as applying sum() to it.
Of course, you cannot apply any operations with None - it doesn't
have a value -, but you can compare it to other objects and it provides
a consistent behavior in Python 2. Python 3 is missing such an object.
I agree with you that Python 3 could use such an object. Just don't make
it the default. Leave None as it is.
Also I agree that my previous naming suggestions are bad. What we need is
only one (or two) additional object whose main semantic meaning is still
no value, but which also adds a meaning of comparable. Then it's a
matter of choosing a good name for it, with lots of bikeshedding involved.
Just lets not change None only because we're too lazy to discuss a proper
alternative name. Also this use case is not _that_ common, so it's ok if
it has a slightly longer name than None.
Also think of the implications of changing None at this point. It would
allow us to write programs that work Python = 3.5 and Python = 2.7, but
fail mysteriously in all other versions in between. What a mess that would
be...
Yes, that's unfortunately true.
If you regard this as bug in the Python 3 series, we could still fix
it for 3.3 and 3.4, though. After all, we'd just be removing an unwanted
exception and not make already working Python 3 applications fail.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Feb 17 2014)
Python Projects, Consulting and Support ... http://www.egenix.com/
mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53
: Try our mxODBC.Connect Python Database Interface for free ! ::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Tue, Feb 18, 2014 at 1:30 AM, M.-A. Lemburg m...@egenix.com wrote: Also think of the implications of changing None at this point. It would allow us to write programs that work Python = 3.5 and Python = 2.7, but fail mysteriously in all other versions in between. What a mess that would be... Yes, that's unfortunately true. I don't know that that's in itself that much of a problem. (BTW, I wouldn't call it = 2.7; it'd be all 2.x, it's not like some of the 2.7.y versions aren't included.) There are already barriers to supporting 2.7 and 3.1/3.2, like the uasdf notation for Unicode literals. Making it easier to support 2.x and 3.x from the same codebase is an improvement that has been done and will be done more. It's not such a mysterious failure; it's just that you support Python 2.5+ and 3.4+ (or whatever the specific versions are). ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Mon, Feb 17, 2014 at 12:43:25PM +0100, M.-A. Lemburg wrote: This doesn't only apply to numeric comparisons. In Python 2 you can compare None with any kind of object and it always sorts first, No you can't. See http://bugs.python.org/issue1673405 . According to Tim Peters, the None is less than everything rule never existed. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17.02.2014 15:38, Jon Ribbens wrote: On Mon, Feb 17, 2014 at 12:43:25PM +0100, M.-A. Lemburg wrote: This doesn't only apply to numeric comparisons. In Python 2 you can compare None with any kind of object and it always sorts first, No you can't. See http://bugs.python.org/issue1673405 . According to Tim Peters, the None is less than everything rule never existed. Well, then Tim probably didn't read the code in object.c :-) Seriously, the datetime module types were the first types to experiment with the new mixed type operation mechanism added at the time: http://www.python.org/dev/peps/pep-0207/ http://www.python.org/dev/peps/pep-0208/ Objects implementing the rich comparison slot can indeed override these defaults and also raise exceptions in comparison operations, so you're right: None comparisons can still be made to raise exceptions, even in Python 2.7. Still, None comparisons work just fine for most types in Python 2.x and people have written code assuming that it works for many years. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/17/2014 7:25 AM, M.-A. Lemburg wrote: On 17.02.2014 13:12, Nick Coghlan wrote: On 17 Feb 2014 21:15, M.-A. Lemburg m...@egenix.com wrote: On 15.02.2014 07:03, Stephen J. Turnbull wrote: M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. This is the first I've ever heard of that sorting behaviour being an intentional feature, rather than just an artefact of Python 2 allowing arbitrarily ordered comparisons between different types. Can you point me to the relevant entry in the Python 2 language reference? This is not documented anywhere in the language spec, AFAIK. http://docs.python.org/2/reference/expressions.html#not-in The operators , , ==, =, =, and != compare the values of two objects. The objects need not have the same type. If both are numbers, they are converted to a common type. Otherwise, objects of different types always compare unequal, and are ordered consistently but arbitrarily. http://docs.python.org/2/library/stdtypes.html#comparisons Objects of different types, except different numeric types and different string types, never compare equal; such objects are ordered consistently but arbitrarily It goes on to note the exception for complex numbers, but none for None. It continues CPython implementation detail: Objects of different types except numbers are ordered by their type names; Again, there is no exception noted for None, although, since the type name of None is 'NoneType', its behavior does not match that doc note. I believe that CPython implementation detail was some of the arbitrary orderings changed in CPython between 1.3 and 2.7. I do not know whether other implementations all mimicked CPython or not, but the reference manual does not require that. It is documented in the code (Python 2.7; Object/object.c): default_3way_compare(PyObject *v, PyObject *w) This is the 'final fallback' for comparisons, not the first thing tried. ... /* None is smaller than anything */ Reading CPython C code is not supposed to be a requirement for programming in Python. If that comment were true, I would regard it as only documenting a version of CPython, not the language standard. But it is not even true in 2.7.6. But it is not even true. class Bottom(object): # get same results below without 'object' def __lt__(self, other): return True # the following two results are consistent and # contradict the claim that 'None is smaller than anything' Bottom() None True cmp(Bottom(), None) -1 # the following two results are not consistent with the # definition of cmp, so 1 of the 2 is buggy None Bottom() True cmp(None, Bottom()) 1 It appears that 'anything' needs to be qualified as something like 'anything that does not itself handle comparison with None or is not given a chance to do so in a particular comparison expression'. if (v == Py_None) return -1; if (w == Py_None) return 1; Note that it's not important whether None is smaller or greater than any other object. The important aspect is that it's sorting order is consistent and doesn't raise a TypeError when doing an ordered comparison with other objects. I can agree that it might have been nice if None had been defined as a universal bottom object, but it has not been so defined and it did not act as such. To make it (or anything else) a true bottom object would require a change in the way comparisons are evaluated. Comparisons with 'bottom' would have to be detected and special-cased at the beginning of the code, not at the end as a fallback. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/17/2014 10:22 AM, M.-A. Lemburg wrote: On 17.02.2014 15:38, Jon Ribbens wrote: On Mon, Feb 17, 2014 at 12:43:25PM +0100, M.-A. Lemburg wrote: This doesn't only apply to numeric comparisons. In Python 2 you can compare None with any kind of object and it always sorts first, No you can't. See http://bugs.python.org/issue1673405 . According to Tim Peters, the None is less than everything rule never existed. Tim is correct. Copying from my other response (posted after you wrote this) class Bottom(object): # get same results below without 'object' def __lt__(self, other): return True # the following two results are consistent and # contradict the claim that 'None is smaller than anything' Bottom() None True cmp(Bottom(), None) -1 # the following two results are not consistent with the # definition of cmp, so 1 of the 2 is buggy None Bottom() True cmp(None, Bottom()) 1 Well, then Tim probably didn't read the code in object.c :-) I did, as I suspect Time has also. Function default_3way_compare is a 'final fallback'. The comment within, besides being a code comment and not the doc, is wrong unless 'anything' is qualified. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
17.02.14 14:11, M.-A. Lemburg написав(ла): Of course, it's easy to add a new type for this, but a lot of Python 2 code relies on None behaving this way, esp. code that reads data from databases, since None is the Python mapping for SQL NULL. At the same time a lot of Python 2 relies on the assumption that any two values are comparable. I suppose that I see a code that try to sort a heterogeneous list more often than a code which want to sort a list containing only numbers and Nones. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
17.02.14 20:18, Serhiy Storchaka написав(ла): 17.02.14 14:11, M.-A. Lemburg написав(ла): Of course, it's easy to add a new type for this, but a lot of Python 2 code relies on None behaving this way, esp. code that reads data from databases, since None is the Python mapping for SQL NULL. At the same time a lot of Python 2 relies on the assumption that any two values are comparable. I suppose that I see a code that try to sort a heterogeneous list more often than a code which want to sort a list containing only numbers and Nones. Sorry, I press Send too early. If we consider the idea to make None universally comparable, we should consider the idea to make all other values universally comparable too. And consider again the reasons which caused these changes in Python 3. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/17/2014 12:59 PM, Terry Reedy wrote: On 2/17/2014 10:22 AM, M.-A. Lemburg wrote: On 17.02.2014 15:38, Jon Ribbens wrote: On Mon, Feb 17, 2014 at 12:43:25PM +0100, M.-A. Lemburg wrote: This doesn't only apply to numeric comparisons. In Python 2 you can compare None with any kind of object and it always sorts first, No you can't. See http://bugs.python.org/issue1673405 . This issue was about the fact that datetimes do not compare with None, which means that even in 2.x, putting None in a list of datetimes (to mean no datetime) means that the list cannot be sorted. According to Tim Peters, the None is less than everything rule never existed. Tim is correct. Copying from my other response (posted after you wrote this) class Bottom(object): # get same results below without 'object' def __lt__(self, other): return True # the following two results are consistent and # contradict the claim that 'None is smaller than anything' Bottom() None True cmp(Bottom(), None) -1 # the following two results are not consistent with the # definition of cmp, so 1 of the 2 is buggy None Bottom() True cmp(None, Bottom()) 1 Well, then Tim probably didn't read the code in object.c :-) I did, as I suspect Time has also. Function default_3way_compare is a 'final fallback'. The comment within, besides being a code comment and not the doc, is wrong unless 'anything' is qualified. FWIW: do_cmp first calls try_rich_to_3way_compare and if that fails it calls try_3way_compare and if that fails it calls default_3way_compare. Somewhat confusingly, try_rich_to_3way_compare first calls try_3way_compare and if that fails, it calls default_3way_compare. So the backup calls in do_cmp are redundant, as they will have already failed in try_rich_to_3way_compare. The special casing of None in default_3way_compare was added by Guido in rev. 16123 on 2001 Jan 22. Before that, None was compared by the typename, as still specified in the docs. Tim was correct when he wrote For older types, the result of inequality comparison with None isn't defined by the language, and the outcome does vary across CPython releases. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/17/2014 1:18 PM, Serhiy Storchaka wrote: 17.02.14 14:11, M.-A. Lemburg написав(ла): Of course, it's easy to add a new type for this, but a lot of Python 2 code relies on None behaving this way, esp. code that reads data from databases, since None is the Python mapping for SQL NULL. At the same time a lot of Python 2 relies on the assumption that any two values are comparable. That assumption was very intentionally broken and abandoned by Guido when complex numbers were added over a decade ago. It was further abandoned, intentionally, when datetimes were added. I think it was decided then to finish the job in 3.0. I suppose that I see a code that try to sort a heterogeneous list more often than a code which want to sort a list containing only numbers and Nones. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
The behavior of None in comparisons is intentional in Python 3. You can agitate to change it, but it will probably die when Guido gets wind of it ;-) The catch-all mixed-type comparison rules in Pythons 1 and 2 were only intended to be arbitrary but consistent. Of course each specific release picked some specific scheme, but these schemes weren't documented, and intentionally not. For a long time CPython's default for mixed-typed comparisons was to compare the names of the types (as strings). And for just as long, nobody had noticed that this _wasn't_ always consistent. For example, 3L 4 must be true. But [] 3L was also true (because list long), and 4 [] was true too (because int list). So, in all, [] 3L 4 [] was true, implying [] []. Part of fixing that was removing some cases of compare objects of different types by comparing the type name strings. Guido I were both in the office at the time, and one said to the other: what about None? Comparing that to other types via comparing the string 'None' doesn't make much sense. Ya, OK ... how about changing None to - by default - comparing 'less than' objects of other types? Don't see why not - sure. OK, done!. No more than 2 minutes of thought went into it. There was no intent to cater to any real use case here - the only intent was to pick some arbitrary-but-consistent rule that didn't suck _quite_ as badly as pretending None was the string None ;-) Guido wanted to drop all the arbitrary but consistent mixed-type comparison crud for Python 3. Nothing special about None in that. As already noted, the various `datetime` types were the first to experiment with implementing full blown Python3-ish mixed-type comparison rules. After the first two times that caught actual bugs in code using the new types, there was no turning back. It's not so much that Python 3 finished the job as that Python 2 started it ;-) much-ado-about-nonething-ly y'rs - tim ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 17.02.2014 21:12, Tim Peters wrote: [...] Guido wanted to drop all the arbitrary but consistent mixed-type comparison crud for Python 3. Nothing special about None in that. As already noted, the various `datetime` types were the first to experiment with implementing full blown Python3-ish mixed-type comparison rules. After the first two times that caught actual bugs in code using the new types, there was no turning back. It's not so much that Python 3 finished the job as that Python 2 started it ;-) Well, I guess it depends on how you look at it. None worked as compares less than all other objects simply due to the fact that None is a singleton and doesn't implement the comparison slots (which for all objects not implementing rich comparisons, meant that the fallback code triggered in Python 2). In Python 3 the special casing was dropped and because None still doesn't implement the comparison slot (tp_richcompare this time), it doesn't support ordering comparisons anymore. Now, the choice to have None compare less than all other objects may have been arbitrary, but IMO it was a good, consistent and useful choice. So why not bring it back and perhaps this time in a way that actually does work consistently for all Python objects by implementing the tp_richcompare slot on PyNoneType objects and documenting it ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 17 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 18 Feb 2014 08:25, Nick Coghlan ncogh...@gmail.com wrote: On 17 Feb 2014 22:25, M.-A. Lemburg m...@egenix.com wrote: On 17.02.2014 13:12, Nick Coghlan wrote: On 17 Feb 2014 21:15, M.-A. Lemburg m...@egenix.com wrote: None is special in Python and has always (and intentionally) sorted before any other object. In data processing and elsewhere in Python programming, it's used to signal: no value available. This is the first I've ever heard of that sorting behaviour being an intentional feature, rather than just an artefact of Python 2 allowing arbitrarily ordered comparisons between different types. Can you point me to the relevant entry in the Python 2 language reference? This is not documented anywhere in the language spec, AFAIK. It is documented in the code (Python 2.7; Object/object.c): default_3way_compare(PyObject *v, PyObject *w) ... /* None is smaller than anything */ if (v == Py_None) return -1; if (w == Py_None) return 1; Note that it's not important whether None is smaller or greater than any other object. The important aspect is that it's sorting order is consistent and doesn't raise a TypeError when doing an ordered comparison with other objects. Thanks, that's enough to persuade me that it is a good idea to restore that behaviour (and make it an official part of the language spec) as part of the eliminate remaining barriers to migration from Python 2 effort in 3.5. It will mean that SQL sorting involving NULL values maps cleanly again, and it's easy enough to use any() to check that container doesn't contain None. Note that I sent this before reading Tim's reply regarding the origin of the Python 2 sorting behaviour, and Terry's point that the relevant Python 2 code lived in the legacy 3 way comparison fallback that was deliberately removed from 3.0. MAL's perspective does still at least mean I am only -0 rather than -1 on making None always sorts first rather than throwing an exception, but my preferred solution remains to provide backporting-friendly support for handling of None values in comparisons (and particularly sorting operations): http://bugs.python.org/issue20630 Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Feb 18, 2014, at 08:25 AM, Nick Coghlan wrote: Thanks, that's enough to persuade me that it is a good idea to restore that behaviour (and make it an official part of the language spec) as part of the eliminate remaining barriers to migration from Python 2 effort in 3.5. At the very least, it's an idea worth PEP-ifying. -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2/17/2014 5:25 PM, Nick Coghlan wrote: On 17 Feb 2014 22:25, M.-A. Lemburg m...@egenix.com mailto:m...@egenix.com wrote: default_3way_compare(PyObject *v, PyObject *w) ... /* None is smaller than anything */ Unless it is not, as with datetimes, perhaps other classes written similarly, and some user class instances. Note that it's not important whether None is smaller or greater than any other object. The important aspect is that it's sorting order is consistent and doesn't raise a TypeError when doing an ordered comparison with other objects. Thanks, that's enough to persuade me that it is a good idea to restore that behaviour Would you restore the actual sometimes inconsistent 2.x behavior or implement something new -- what M-A claims but never was? I doubt the former would be trivial since it was part of the now deleted cmp and 3way machinery. To make None a true bottom object, the rich comparison methods would have to special-case None as either argument before looking at the __rc__ special methods of either. Regardless of how implemented, such a change would break user code that defines a Bottom class and depends on Bottom() None == True. Given the python-ideas discussions about adding a top or bottom class, I expect that such classes are in use. So a deprecation period would be needed, pushing the change off to 3.7. It is also possible that someone took the ending of cross-type comparisons seriously and is depending on the TypeError. while True: x = f(args) try: if x 10: a(x) else: b(x) except TypeError: # f did not return a number break -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
[M.-A. Lemburg] ... None worked as compares less than all other objects simply due to the fact that None is a singleton and doesn't implement the comparison slots (which for all objects not implementing rich comparisons, meant that the fallback code triggered in Python 2). And the fallback code - which you've already found - went on to special-case the snot out of None to make less than work. It's not like it was a natural outcome: it was forced. In Python 3 the special casing was dropped and because None still doesn't implement the comparison slot (tp_richcompare this time), it doesn't support ordering comparisons anymore. Which makes mixed-type non-numeric default comparisons involving None work the same way all other mixed-type non-numeric default comparisons work in Python 3. This was the result of a deliberate design decision about how comparison _should_ work in Python 3, not an accidental consequence of the None type not defining tp_richcompare. IOW, this is about design, not about implementation. The differing implementations you see are consequences of the differing designs. Staring at the implementations is irrelevant to the design decisions. Now, the choice to have None compare less than all other objects may have been arbitrary, but IMO it was a good, consistent and useful choice. Possibly useful for some apps, sure. Not for my apps. For example, when I initialize an object attribute to None in Python 3, I _expect_ I'll get an exception if I try to use that attribute in most contexts. It makes no more sense to ask whether that attribute is, say, less than 3, than it does to add 3 to it. The exception is most useful then. More often than not, it was annoying to me that Python 2 _didn't_ whine about trying to compare None. So why not bring it back A huge barrier is (or should be) that Python 3 is over 5 years old now. Fiddling with the semantics of basic builtin types was possible - and even welcome - 6 years ago. Now they really shouldn't be touched in the absence of a critical bug, or a wholly backward-compatible (with Python 3) addition. and perhaps this time in a way that actually does work consistently for all Python objects by implementing the tp_richcompare slot on PyNoneType objects and documenting it ?! Something to suggest for Python 4, in which case I'll only be -0 ;-) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
Tim Peters wrote: Guido wanted to drop all the arbitrary but consistent mixed-type comparison crud for Python 3. Nobody is asking for a return to the arbitrary-but- [in]consistent mess of Python 2, only to bring back *one* special case, i.e. None comparing less than everything else. I think there is a reasonable argument to be made in favour of that. Like it or not, None does have a special place in Python as the one obvious way to represent a null or missing value, and often one wants to sort a collection of objects having keys that can take on null values. Refusing to make that easy seems like allowing purity to beat practicality. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
[Tim] Guido wanted to drop all the arbitrary but consistent mixed-type comparison crud for Python 3. [Greg Ewing] Nobody is asking for a return to the arbitrary-but- [in]consistent mess of Python 2, only to bring back *one* special case, i.e. None comparing less than everything else. Of course. My point was that dropping None comparisons wasn't specifically aimed at None comparisons. I think there is a reasonable argument to be made in favour of that. There may have been 6 years ago, but Python 3 isn't a thought experiment anymore. It was first released over 5 years ago, and has its own compatibility (with Python 3, not Python 2) constraints now. Like it or not, None does have a special place in Python as the one obvious way to represent a null or missing value, and often one wants to sort a collection of objects having keys that can take on null values. Perhaps that's often true of your code, but it's never been true of mine. Refusing to make that easy seems like allowing purity to beat practicality. Since I've never had a need for it, I can't say for sure whether or not it's easy - but the .sort() interface is rich enough that I almost always find even truly complex key transformations easy enough. Perhaps a real, concrete use case would help. I have a hard time imagining why I'd _want_ to sort a list of objects with null or missing keys, instead of filtering such garbage out of the list first and sorting what remains. But even with a compelling use case, I'd still think it's years too late to change this in Python 3. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
Terry Reedy wrote: To make None a true bottom object, the rich comparison methods would have to special-case None as either argument before looking at the __rc__ special methods of either. I don't think it would be necessary to go that far. It would be sufficient to put the special case *after* giving both operations a chance to handle the operation via the type slots. Well-behaved objects generally wouldn't go out of their way to defeat that, but they could do so if necessary, e.g. to create a Bottom() that compares less than None. Although once None becomes usable as a bottom in most cases, there shouldn't be much need for people to introduce their own bottoms. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
Tim Peters wrote:
[Greg Ewing]
often
one wants to sort a collection of objects having
keys that can take on null values.
Perhaps that's often true of your code, but it's never been true of mine.
It's fairly common in accounting circles. I have a
collection of invoices, each of which can have a due
date specified, but doesn't have to. I want to sort
the invoices by due date. It's not particularly
important whether the missing dates sort first or
last, but it *is* important that the sort *not blow
up*.
Dates seem to be a particularly irksome case. Here's
one suggestion from StackOverflow for dealing with
the problem:
import datetime
mindate = datetime.date(datetime.MINYEAR, 1, 1)
def getaccountingdate(x):
return x['accountingdate'] or mindate
results = sorted(data, key=getaccountingdate)
That's a ridiculous amount of boilerplate for doing
something that ought to be straightforward.
If you don't want to touch comparison in general,
how about an option to sort()?
results = sorted(invoices, key=attrgetter('duedate'), none='first')
I have a hard time
imagining why I'd _want_ to sort a list of objects with null or
missing keys, instead of filtering such garbage out of the list first
A piece of data is *not* garbage just because an optional
part of it is not specified.
--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] python 3 niggle: None 1 raises TypeError
Hi All, Sending this to python-dev as I'm wondering if this was considered when the choice to have objects of different types raise a TypeError when ordered... So, the concrete I case I have is implementing stable ordering for the python Range objects that psycopg2 uses. These have 3 attributes that can either be None or, for sake of argument, a numeric value. To implement __lt__ in Python 2, I could do: def __lt__(self, other): if not isinstance(other, Range): return True return ((self._lower, self._upper, self._bounds) (other._lower, other._upper, other._bounds)) Because None 1 raises a TypeError, in Python 3 I have to do: def __lt__(self, other): if not isinstance(other, Range): return NotImplemented for attr in '_lower', '_upper', '_bounds': self_value = getattr(self, attr) other_value = getattr(other, attr) if self_value == other_value: pass elif self_value is None: return True elif other_value is None: return False else: return self_value other_value return False Am I missing something? How can I get this method down to a sane size? cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 14 February 2014 18:04, Chris Withers ch...@simplistix.co.uk wrote: Am I missing something? How can I get this method down to a sane size? The easiest way is usually to normalise the attributes to a sensible numeric value depending on where you want None to sort (positive and negative infinity floating point values often work well in the case of numeric data, but a custom AlwaysLess or AlwaysGreater type works for arbitrary data). You can either do that dynamically, or else cache the normalised values when the attributes are set. Python 2 used to guess, Python 3 makes developers decide how they want None to be handled in the context of ordering operations. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 7:04 PM, Chris Withers ch...@simplistix.co.uk wrote: To implement __lt__ in Python 2, I could do: def __lt__(self, other): if not isinstance(other, Range): return True return ((self._lower, self._upper, self._bounds) (other._lower, other._upper, other._bounds)) Because None 1 raises a TypeError, in Python 3 I have to do: def __lt__(self, other): if not isinstance(other, Range): return NotImplemented for attr in '_lower', '_upper', '_bounds': self_value = getattr(self, attr) other_value = getattr(other, attr) if self_value == other_value: pass elif self_value is None: return True elif other_value is None: return False else: return self_value other_value return False Am I missing something? How can I get this method down to a sane size? Can you be certain that all your values are either None or positive integers? If so, try this: def __lt__(self, other): if not isinstance(other, Range): return True # or NotImplemented, not sure why this change return ((self._lower or 0, self._upper or 0, self._bounds or 0) (other._lower or 0, other._upper or 0, other._bounds or 0)) That'll treat all Nones as 0, and compare them accordingly. If you can't depend on them being positive (eg if you need None to be less than 0 rather than equal - even more so if you need it to be less than negative numbers), you'll need to more explicitly check. But this is nice and tidy, if it works. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 9:04 AM, Chris Withers ch...@simplistix.co.uk wrote:
Hi All,
Sending this to python-dev as I'm wondering if this was considered when the
choice to have objects of different types raise a TypeError when ordered...
So, the concrete I case I have is implementing stable ordering for the
python Range objects that psycopg2 uses. These have 3 attributes that can
either be None or, for sake of argument, a numeric value.
To implement __lt__ in Python 2, I could do:
def __lt__(self, other):
if not isinstance(other, Range):
return True
return ((self._lower, self._upper, self._bounds)
(other._lower, other._upper, other._bounds))
Because None 1 raises a TypeError, in Python 3 I have to do:
def __lt__(self, other):
if not isinstance(other, Range):
return NotImplemented
for attr in '_lower', '_upper', '_bounds':
self_value = getattr(self, attr)
other_value = getattr(other, attr)
if self_value == other_value:
pass
elif self_value is None:
return True
elif other_value is None:
return False
else:
return self_value other_value
return False
Am I missing something? How can I get this method down to a sane size?
It was considered. It's not obvious where you want None to appear in
your ordering, so you will have to implement this by yourself. I can't
come up with anything obviously shorter.
Or, well, this, but it's not exactly pretty:
def __lt__(self, other):
if not isinstance(other, Range):
return NotImplemented
ninf = float('-inf') # So None is lower than anything else.
return ((self._lower if self._lower is not None else ninf,
self._upper if self._upper is not None else ninf,
self._bounds if self._bounds is not None else ninf) (
other._lower if other._lower is not None else ninf,
other._upper if other._upper is not None else ninf,
other._bounds if other._bounds is not None else ninf))
Or maybe:
def __lt__(self, other):
if not isinstance(other, Range):
return NotImplemented
s = (self._lower, self._upper, self._bounds)
if None is s:
return True
o = (other._lower, other._upper, other._bounds)
if None in o:
return False
return s o
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, 14 Feb 2014 10:46:50 +0100 Lennart Regebro rege...@gmail.com wrote: Sending this to python-dev as I'm wondering if this was considered when the choice to have objects of different types raise a TypeError when ordered... So, the concrete I case I have is implementing stable ordering for the python Range objects that psycopg2 uses. These have 3 attributes that can either be None or, for sake of argument, a numeric value. [...] It was considered. It's not obvious where you want None to appear in your ordering, so you will have to implement this by yourself. I can't come up with anything obviously shorter. I have to agree with Lennart. The fact that SQL defines an order for NULL and other values doesn't mean it's obvious or right in any way (I never remember which way it goes). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 14 February 2014 20:02, Antoine Pitrou solip...@pitrou.net wrote: On Fri, 14 Feb 2014 10:46:50 +0100 Lennart Regebro rege...@gmail.com wrote: Sending this to python-dev as I'm wondering if this was considered when the choice to have objects of different types raise a TypeError when ordered... So, the concrete I case I have is implementing stable ordering for the python Range objects that psycopg2 uses. These have 3 attributes that can either be None or, for sake of argument, a numeric value. [...] It was considered. It's not obvious where you want None to appear in your ordering, so you will have to implement this by yourself. I can't come up with anything obviously shorter. I have to agree with Lennart. The fact that SQL defines an order for NULL and other values doesn't mean it's obvious or right in any way (I never remember which way it goes). SQL doesn't define an order for NULL, it's more like a quiet NaN - if either operand is NULL, the result is also NULL. (I ran into this recently, in the context of NULL == value and NULL != value both being effectively false). We could theoretically do something similar, but the NULL handling in SQL is a frequent source of bugs in filter definitions, so that really doesn't sound like a good idea. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, 14 Feb 2014 20:13:43 +1000 Nick Coghlan ncogh...@gmail.com wrote: On 14 February 2014 20:02, Antoine Pitrou solip...@pitrou.net wrote: On Fri, 14 Feb 2014 10:46:50 +0100 Lennart Regebro rege...@gmail.com wrote: Sending this to python-dev as I'm wondering if this was considered when the choice to have objects of different types raise a TypeError when ordered... So, the concrete I case I have is implementing stable ordering for the python Range objects that psycopg2 uses. These have 3 attributes that can either be None or, for sake of argument, a numeric value. [...] It was considered. It's not obvious where you want None to appear in your ordering, so you will have to implement this by yourself. I can't come up with anything obviously shorter. I have to agree with Lennart. The fact that SQL defines an order for NULL and other values doesn't mean it's obvious or right in any way (I never remember which way it goes). SQL doesn't define an order for NULL, it's more like a quiet NaN - if either operand is NULL, the result is also NULL. (I ran into this recently, in the context of NULL == value and NULL != value both being effectively false). Hmm, it seems you're right, but I'm quite sure some DBMSes have a consistent way of ordering NULLs when using ORDER BY on a nullable column. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 14 February 2014 10:20, Antoine Pitrou solip...@pitrou.net wrote: Hmm, it seems you're right, but I'm quite sure some DBMSes have a consistent way of ordering NULLs when using ORDER BY on a nullable column. ORDER BY xxx [NULLS FIRST|LAST] is the syntax in Oracle, with (IIRC) NULLS LAST as default. But I agree, this is not an argument in favour of doing the same in Python. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 9:20 PM, Antoine Pitrou solip...@pitrou.net wrote: Hmm, it seems you're right, but I'm quite sure some DBMSes have a consistent way of ordering NULLs when using ORDER BY on a nullable column. Yes, and I believe it's part of the SQL-92 spec. Certainly here's PostgreSQL's take on the matter: http://www.postgresql.org/docs/9.3/static/sql-select.html#SQL-ORDERBY In short, NULL is by default considered greater than anything else (last in an ascending sort, first in a descending sort), but this can be inverted. Oddly enough, the syntax is NULLS FIRST vs NULLS LAST, not NULLS LOW vs NULLS HIGH. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 14 February 2014 20:30, Paul Moore p.f.mo...@gmail.com wrote: On 14 February 2014 10:20, Antoine Pitrou solip...@pitrou.net wrote: Hmm, it seems you're right, but I'm quite sure some DBMSes have a consistent way of ordering NULLs when using ORDER BY on a nullable column. ORDER BY xxx [NULLS FIRST|LAST] is the syntax in Oracle, with (IIRC) NULLS LAST as default. But I agree, this is not an argument in favour of doing the same in Python. IIRC, MySQL and PostgreSQL sort them in the opposite order from each other (or it's possibly just Teiid's PostgreSQL emulation that is the opposite of MySQL). Either way, it nicely illustrates *why* we didn't grant None an exemption from the no implicit cross-type ordering operations in 3.x. It does make *adapting* to other models like SQL a bit more painful though. That's the main reason I still occasionally wonder if AlwaysMin and AlwaysMax singletons might make sense, although that approach has problems of its own (specifically, it's hard to actually use without breaking the clean NULL - None mapping and in many cases +/- infinity already work fine). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 9:42 PM, Nick Coghlan ncogh...@gmail.com wrote: IIRC, MySQL and PostgreSQL sort them in the opposite order from each other Ouch! You're right: http://dev.mysql.com/doc/refman/5.7/en/working-with-null.html When doing an ORDER BY, NULL values are presented first if you do ORDER BY ... ASC and last if you do ORDER BY ... DESC. Not configurable in MySQL. It's apparently not part of the standard, according to Wikipedia: http://en.wikipedia.org/wiki/Order_by So definitely SQL's handling of NULL should not be any sort of guide as regards Python's treatment of None. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 09:54:35PM +1100, Chris Angelico ros...@gmail.com wrote: So definitely SQL's handling of NULL should not be any sort of guide as regards Python's treatment of None. Why not? Just make the order different for CPython and PyPy, Cython and Jython. ;-) Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Feb 14, 2014 1:13 PM, Oleg Broytman p...@phdru.name wrote: On Fri, Feb 14, 2014 at 09:54:35PM +1100, Chris Angelico ros...@gmail.com wrote: So definitely SQL's handling of NULL should not be any sort of guide as regards Python's treatment of None. Why not? Just make the order different for CPython and PyPy, Cython and Jython. ;-) Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. Unicode + bytes explodes in order to get the error sooner.i.e Not waiting for the killer 8bit string. So None1 is the opposite ! sort(somelist) is a ticking bomb in py3 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 14.02.2014 11:20, Antoine Pitrou wrote: On Fri, 14 Feb 2014 20:13:43 +1000 Nick Coghlan ncogh...@gmail.com wrote: On 14 February 2014 20:02, Antoine Pitrou solip...@pitrou.net wrote: On Fri, 14 Feb 2014 10:46:50 +0100 Lennart Regebro rege...@gmail.com wrote: Sending this to python-dev as I'm wondering if this was considered when the choice to have objects of different types raise a TypeError when ordered... So, the concrete I case I have is implementing stable ordering for the python Range objects that psycopg2 uses. These have 3 attributes that can either be None or, for sake of argument, a numeric value. [...] It was considered. It's not obvious where you want None to appear in your ordering, so you will have to implement this by yourself. I can't come up with anything obviously shorter. I have to agree with Lennart. The fact that SQL defines an order for NULL and other values doesn't mean it's obvious or right in any way (I never remember which way it goes). SQL doesn't define an order for NULL, it's more like a quiet NaN - if either operand is NULL, the result is also NULL. (I ran into this recently, in the context of NULL == value and NULL != value both being effectively false). Hmm, it seems you're right, but I'm quite sure some DBMSes have a consistent way of ordering NULLs when using ORDER BY on a nullable column. They do, but it's not consistent: MS SQL Server: orders NULLs first (in table order; stable sort) Sybase ASE:orders NULLs first (in arbitrary order) Oracle:orders NULLs last (in arbitrary order) PostgreSQL:orders NULLs last (in arbitrary order) IBM DB2: orders NULLs last (in table order; stable sort) Reference: tests done with actual databases. A note about consistency: None is always ordered first in Python 2, so there is a precedent. And since Python's list.sort() is stable, Python 2 is in the same camp as MS SQL Server. From Python-2.7.6/Objects/object.c: /* None is smaller than anything */ if (v == Py_None) return -1; if (w == Py_None) return 1; IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. Python2: l = [1,2,None,4,5,None,6] l.sort() l [None, None, 1, 2, 4, 5, 6] Python3: l = [1,2,None,4,5,None,6] l.sort() Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: NoneType() int() -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 14 2014) Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2014-02-12: Released mxODBC.Connect 2.0.4 ... http://egenix.com/go53 : Try our mxODBC.Connect Python Database Interface for free ! :: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
What I think is the biggest nitpick here is that you can't make a NULL object so that NULL is None evaluates as True. If you could, you could then easily make this NULL object evaluate as less than everything except itself and None, and use that consistently. But since this NULL is not None, that breaks any backwards compatibility if your SQL library mapped NULL to None in earlier versions. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 2014-02-14 08:04, Chris Withers wrote: Hi All, Sending this to python-dev as I'm wondering if this was considered when the choice to have objects of different types raise a TypeError when ordered... So, the concrete I case I have is implementing stable ordering for the python Range objects that psycopg2 uses. These have 3 attributes that can either be None or, for sake of argument, a numeric value. To implement __lt__ in Python 2, I could do: def __lt__(self, other): if not isinstance(other, Range): return True return ((self._lower, self._upper, self._bounds) (other._lower, other._upper, other._bounds)) Because None 1 raises a TypeError, in Python 3 I have to do: def __lt__(self, other): if not isinstance(other, Range): return NotImplemented for attr in '_lower', '_upper', '_bounds': self_value = getattr(self, attr) other_value = getattr(other, attr) if self_value == other_value: pass elif self_value is None: return True elif other_value is None: return False else: return self_value other_value return False Am I missing something? How can I get this method down to a sane size? How about this: def make_key(item): return [(x is not None, x or 0) for i in [item._lower, item._upper, item._bounds]] def __lt__(self, other): if not isinstance(other, Range): return NotImplemented return make_key(self) make_key(other) It'll make None less than any number. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 1:29 AM, Nick Coghlan ncogh...@gmail.com wrote:
On 14 February 2014 18:04, Chris Withers ch...@simplistix.co.uk wrote:
Am I missing something? How can I get this method down to a sane size?
The easiest way is usually to normalise the attributes to a sensible
numeric value depending on where you want None to sort (positive and
negative infinity floating point values often work well in the case of
numeric data, but a custom AlwaysLess or AlwaysGreater type works for
arbitrary data).
This is actually a pretty common case -- the question here is: Is this
really None? or does None have a special meaning. It sounds like this is a
wrapper for a PostgreSQL range object, in which case None isn't really
right, there must be a +-infinity concept there -- how does postgress
handle it? (is this a generic range, or a datetime range? I'm not up on it).
If this means what I think it does, None is really a hack, both because of
al lteh specia case code required for sorting, comparing, etc, but also
because it's missing information -- the two ends of the range should really
have different meanings.
I've dealt with this for Python datetime via a pari fo custom InfDateTime
objects one for plu infiinity, one for minus. Then we can use these in
ranges and the reest of teh code can jsut do simple comparisons, sorting
etc.
Of course, floating point numbers have inf and -inf, which is really useful.
I hadn't though of Nick's generic AlwaysLess or AlwaysGreater type -- if
that means what I think it does -- i,e, it would compare appropriately to
ANY other object, that would actually be a nice addition to
the standard library. I had been thinking that the standard
library datetime could use these, but why not just use generic ones?
(though it could get a bit tricky -- what would AlwaysGreater
float('inf) evaluate to?
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/ORR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Sat, Feb 15, 2014 at 4:14 AM, Chris Barker chris.bar...@noaa.gov wrote:
(though it could get a bit tricky -- what would AlwaysGreater float('inf)
evaluate to?
It'd be true. AlwaysGreater is greater than infinity. It is greater
than float(nan). It is greater than the entire concept of floating
point. It is greater than everything imaginable... except for a nice
MLT - a mutton, lettuce, and tomato sandwich... oh.
Where things get tricky is when you compare two AlwaysGreater objects.
Or even compare one against itself. It has to be greater.
Great.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, 14 Feb 2014, Chris Barker wrote:
This is actually a pretty common case -- the question here is: Is this
really None? or does None have a special meaning. It sounds like this is a
wrapper for a PostgreSQL range object, in which case None isn't really
right, there must be a +-infinity concept there -- how does postgress handle
it? (is this a generic range, or a datetime range? I'm not up on it).
If this means what I think it does, None is really a hack, both because of
al lteh specia case code required for sorting, comparing, etc, but also
because it's missing information -- the two ends of the range should really
have different meanings.
Postgres range types are a separate kind of type which can be created
using CREATE TYPE:
http://www.postgresql.org/docs/current/interactive/sql-createtype.html
Several handy range types are built in, although not as many as one might
expect.
Postgres ranges can be made infinite by specifying NULL as a bound.
There isn't a separate special value for this. This has somewhat strange
effects when the base type of the range has its own infinity value:
= select upper_inf (daterange (current_date, 'infinity'));
upper_inf
---
f
(1 row)
= select isfinite ('infinity'::date);
isfinite
--
f
(1 row)
= select upper_inf (daterange (current_date, null));
upper_inf
---
t
(1 row)
In other words, a range with non-NULL bounds is finite according to the
range functions even if the bound is the infinity value of the base type.
Isaac Morland CSCF Web Guru
DC 2619, x36650 WWW Software Specialist___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
The idea of top and bottom objects, by whatever name, has be proposed,
discussed, and rejected on python-ideas list (which is where this
discussion really belongs if continued).
On 2/14/2014 4:41 PM, Chris Angelico wrote:
(though it could get a bit tricky -- what would AlwaysGreater float('inf)
evaluate to?
It'd be true. AlwaysGreater is greater than infinity. It is greater
than float(nan). It is greater than the entire concept of floating
point. It is greater than everything imaginable... except for a nice
MLT - a mutton, lettuce, and tomato sandwich... oh.
Where things get tricky is when you compare two AlwaysGreater objects.
Or even compare one against itself. It has to be greater.
equal, and less than itself. Or not, depending on exactly how the
methods are defined and coded. Which, as I remember, is part of why the
idea of a *generic* class was rejected. Real cases are better served by
custom classes that meet the custom need.
--
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On 15 February 2014 08:10, Terry Reedy tjre...@udel.edu wrote: The idea of top and bottom objects, by whatever name, has be proposed, discussed, and rejected on python-ideas list (which is where this discussion really belongs if continued). A fair point, but my recollection of those discussions is that we completely missed the use case of being able to sensibly map SQL NULL handling in ORDER BY clauses to the sorting of Python 3 containers. Having a concrete use case with widespread implications makes a big difference :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
M.-A. Lemburg writes: IMO, it was a mistake to have None return a TypeError in comparisons, since it makes many typical data operations fail, e.g. I don't understand this statement. The theory is that they *should* fail. The example of sort is a good one. Sometimes you want missing values to be collected at the beginning of a list, sometimes at the end. Sometimes you want them treated as top elements, sometimes as bottom. And sometimes it is a real error for missing values to be present. Not to mention that sometimes the programmer simply hasn't thought about the appropriate policy. I don't think Python should silently impose a policy in that case, especially given that the programmer may have experience with any of the above treatments in other contexts. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
