[issue27561] Warn against subclassing builtins, and overriding their methods

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[Raymond Hettinger]

Raymond Hettinger added the comment:

Will do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[Chris Rebert]

Changes by Chris Rebert :


--
nosy: +cvrebert

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[Kirk Hansen]

Kirk Hansen added the comment:

Raymond: Thanks for essentially answering my stackoverflow question (if you are 
a member, and add most of your response, I'll accept it).

I agree enough with your documentation statement. An entry in an FAQ would have 
been just as helpful to me as a note in the documentation, and the FAQ is 
probably more accessible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[Steven D'Aprano]

Steven D'Aprano added the comment:

Raymond, that was a fantastic explanation. Would you be willing to turn it into 
a FAQ? Or if you don't have the time, to allow somebody to steal your 
description and use it?

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[R. David Murray]

R. David Murray added the comment:

Raymond: yes, I was dubious about the benefit of the doc note.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[Raymond Hettinger]

Raymond Hettinger added the comment:

This is an example of the open-closed-principle (the class is open for 
extension but closed for modification).  It is good thing to have because it 
allows subclassers to extend or override a method without unintentionally 
triggering behavior changes in others and without breaking the classes's 
invariants.

We even do this in pure python code as well; for example, inside the pure 
python ordered dict code, the class local call from __init__ to update() is 
done using name mangling.  This allows a subclasser to override update() 
without accidentally breaking __init__.

Sometimes, this is inconvenient.  It means that a subclasser has to override 
every method whose behavior they want to change including get(), update(), and 
others.  However, there are offsetting benefits (protection of internal 
invariants, preventing implementation details from leaking from the 
abstraction, and allowing users to assume the methods are independent of one 
another).

This style (chosen by Guido from the outset) is the default for the builtin 
types (otherwise we would forever be fighting segfaulting invariant violations) 
and for some pure python classes.

We do document when there is a departure from the default.  For example, the 
cmd module uses the framework design pattern, letting the user define various 
do_action() methods.  Also, some of the http modules do the same, specifically 
documentation that the user's do_get method gets called and that some 
specifically named user handler method gets called.

In the absence of specifically documented method hooks (i.e. those listed above 
or methods like __missing__), a subclasser should presume method independence.  
Otherwise, how are you to know whether __getitem__ calls get() under the hood 
or vice-versa?

FWIW, this isn't unique to Python.  It comes up quite a bit in object oriented 
programming.  Correctly designed classes either document root methods that 
affect the behavior of other methods or they are presumed to be independent.

There may need to be a FAQ for this, but nothing is broken or wrong here (other 
than Python having way to many dict variants to chose from).  Also, R David 
Murray almost no one would be helped by a note in some random place in the 
docs.  If someone assumes or believes that __getitem__ must be called by the 
other accessor methods, they find out very quickly that assumption is wrong 
(that is if they run even minimal tests on the code).

--
assignee: docs@python -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
priority: normal -> low

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[R. David Murray]

R. David Murray added the comment:

It doesn't just apply to dicts, it applies to all the built in collection types 
(and possibly to other built in types).  But not to all the methods, which will 
make the documentation somewhat vague ("you may not be able to successfully 
override base class methods such as __setitem__...")  Oh, and I suppose it 
needs to be marked as a CPython implementation detail.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[Kirk Hansen]

Kirk Hansen added the comment:

I think it could make sense at the top of the intro. It could also make sense 
to have it sit 
https://docs.python.org/2.7/library/stdtypes.html?highlight=dict#mapping-types-dict
 and have the intro link to it, or vice-versa. Thoughts?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[R. David Murray]

R. David Murray added the comment:

Where in the docs do you think such a note should go?  I suppose it could go in 
the intro to the chapter you link to.  I doubt many people will find it there, 
though :(

--
nosy: +r.david.murray
versions:  -Python 3.2, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27561] Warn against subclassing builtins, and overriding their methods

[Kirk Hansen]

New submission from Kirk Hansen:

I tried subclassing dict, and overriding its __setitem__ and __getitem__ and 
got some interesting results. See 
http://stackoverflow.com/questions/38362420/subclassing-dict-dict-update-returns-incorrrect-value-python-bug?noredirect=1#comment64142710_38362420
 for an example.

I tried sub-classing UserDict.UserDict and experienced some of the same 
problems. Eventually, I discovered that subclassing MutableMapping was my best 
bet.

After an hour or two of searching and reading I discovered that CPython will 
not call overridden built-in methods of the same object 
(https://pypy.readthedocs.io/en/latest/cpython_differences.html#subclasses-of-built-in-types).
 This behaviour could (and did) cause some hard to track down bugs in code. 

I briefly looked at the later versions of python documentation and didn't see a 
mention of this (sorry if it's there), but python2.7 definitely does not 
mention this. In fact, python2.7 seems to __encourage__ users to subclass the 
built-ins (see the note 
https://docs.python.org/2.7/library/stdtypes.html?highlight=dict#built-in-types).
 Subclassing dict to __extend__ functionality is great, but there should be a 
big bad warning when trying to __override__ built-ins like __setitem__ and 
__getitem__.

--
assignee: docs@python
components: Documentation
messages: 270754
nosy: Kirk Hansen, docs@python
priority: normal
severity: normal
status: open
title: Warn against subclassing builtins, and overriding their methods
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com