On Wed, Jul 13, 2016 at 7:15 AM, Martin Teichmann <lkb.teichm...@gmail.com> wrote: > Hi list, > > another round for PEP 487, is there any chance it still makes it into > Python 3.6?
Sure, feature freeze isn't until September (https://www.python.org/dev/peps/pep-0494/). > The PEP should be effectively done, I updated the examples in it, > given that I implemented the PEP I could actually test the examples, > so now they work. I am +1 on the idea of the PEP; below I am just asking for some clarifications and pointing out a typo or two. Please submit the next version to the github peps project as a PR! Re-review should be much quicker then. > The implementation is at http://bugs.python.org/issue27366, including > documentation and tests. Unfortunately nobody has reviewed the patch > yet. Sorry, I don't have time for that part, but I'm sure once the PEP is approved the review will follow. > The new version of the PEP is attached. > > Greetings > > Martin > > PEP: 487 > Title: Simpler customisation of class creation > Version: $Revision$ > Last-Modified: $Date$ > Author: Martin Teichmann <lkb.teichm...@gmail.com>, > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 27-Feb-2015 > Python-Version: 3.6 > Post-History: 27-Feb-2015, 5-Feb-2016, 24-Jun-2016, 2-Jul-2016, 13-Jul-2016 > Replaces: 422 > > > Abstract > ======== > > Currently, customising class creation requires the use of a custom metaclass. > This custom metaclass then persists for the entire lifecycle of the class, > creating the potential for spurious metaclass conflicts. > > This PEP proposes to instead support a wide range of customisation > scenarios through a new ``__init_subclass__`` hook in the class body, > and a hook to initialize attributes. > > The new mechanism should be easier to understand and use than > implementing a custom metaclass, and thus should provide a gentler > introduction to the full power of Python's metaclass machinery. > > > Background > ========== > > Metaclasses are a powerful tool to customize class creation. They have, > however, the problem that there is no automatic way to combine metaclasses. > If one wants to use two metaclasses for a class, a new metaclass combining > those two needs to be created, typically manually. > > This need often occurs as a surprise to a user: inheriting from two base > classes coming from two different libraries suddenly raises the necessity > to manually create a combined metaclass, where typically one is not > interested in those details about the libraries at all. This becomes > even worse if one library starts to make use of a metaclass which it > has not done before. While the library itself continues to work perfectly, > suddenly every code combining those classes with classes from another library > fails. > > Proposal > ======== > > While there are many possible ways to use a metaclass, the vast majority > of use cases falls into just three categories: some initialization code > running after class creation, the initalization of descriptors and initialization > keeping the order in which class attributes were defined. > > The first two categories can easily be achieved by having simple hooks > into the class creation: > > 1. An ``__init_subclass__`` hook that initializes > all subclasses of a given class. > 2. upon class creation, a ``__set_owner__`` hook is called on all the > attribute (descriptors) defined in the class, and > > The third category is the topic of another PEP 520. PEP, PEP 520. > As an example, the first use case looks as follows:: > > >>> class QuestBase: > ... # this is implicitly a @classmethod maybe add "(see below for motivation)" ? > ... def __init_subclass__(cls, swallow, **kwargs): > ... cls.swallow = swallow > ... super().__init_subclass__(**kwargs) > > >>> class Quest(QuestBase, swallow="african"): > ... pass > > >>> Quest.swallow > 'african' > > The base class ``object`` contains an empty ``__init_subclass__`` > method which serves as an endpoint for cooperative multiple inheritance. > Note that this method has no keyword arguments, meaning that all > methods which are more specialized have to process all keyword > arguments. > > This general proposal is not a new idea (it was first suggested for > inclusion in the language definition `more than 10 years ago`_, and a > similar mechanism has long been supported by `Zope's ExtensionClass`_), > but the situation has changed sufficiently in recent years that > the idea is worth reconsidering for inclusion. > > The second part of the proposal adds an ``__set_owner__`` > initializer for class attributes, especially if they are descriptors. > Descriptors are defined in the body of a > class, but they do not know anything about that class, they do not > even know the name they are accessed with. They do get to know their > owner once ``__get__`` is called, but still they do not know their > name. This is unfortunate, for example they cannot put their > associated value into their object's ``__dict__`` under their name, > since they do not know that name. This problem has been solved many > times, and is one of the most important reasons to have a metaclass in > a library. While it would be easy to implement such a mechanism using > the first part of the proposal, it makes sense to have one solution > for this problem for everyone. > > To give an example of its usage, imagine a descriptor representing weak > referenced values:: > > import weakref > > class WeakAttribute: > def __get__(self, instance, owner): > return instance.__dict__[self.name]() > > def __set__(self, instance, value): > instance.__dict__[self.name] = weakref.ref(value) > > # this is the new initializer: > def __set_owner__(self, owner, name): > self.name = name This example is missing something -- an example of how the WeakAttribute class would be *used*. I suppose something like # We wish we could write class C: foo = WeakAttribute() x = C() x.foo = ... print(x.foo) > While this example looks very trivial, it should be noted that until > now such an attribute cannot be defined without the use of a metaclass. > And given that such a metaclass can make life very hard, this kind of > attribute does not exist yet. > > Initializing descriptors could simply be done in the > ``__init_subclass__`` hook. But this would mean that descriptors can > only be used in classes that have the proper hook, the generic version > like in the example would not work generally. One could also call > ``__set_owner__`` from whithin the base implementation of > ``object.__init_subclass__``. But given that it is a common mistake > to forget to call ``super()``, it would happen too often that suddenly > descriptors are not initialized. > > > Key Benefits > ============ > > > Easier inheritance of definition time behaviour > ----------------------------------------------- > > Understanding Python's metaclasses requires a deep understanding of > the type system and the class construction process. This is legitimately > seen as challenging, due to the need to keep multiple moving parts (the code, > the metaclass hint, the actual metaclass, the class object, instances of the > class object) clearly distinct in your mind. Even when you know the rules, > it's still easy to make a mistake if you're not being extremely careful. > > Understanding the proposed implicit class initialization hook only requires > ordinary method inheritance, which isn't quite as daunting a task. The new > hook provides a more gradual path towards understanding all of the phases > involved in the class definition process. > > > Reduced chance of metaclass conflicts > ------------------------------------- > > One of the big issues that makes library authors reluctant to use metaclasses > (even when they would be appropriate) is the risk of metaclass conflicts. > These occur whenever two unrelated metaclasses are used by the desired > parents of a class definition. This risk also makes it very difficult to > *add* a metaclass to a class that has previously been published without one. > > By contrast, adding an ``__init_subclass__`` method to an existing type poses > a similar level of risk to adding an ``__init__`` method: technically, there > is a risk of breaking poorly implemented subclasses, but when that occurs, > it is recognised as a bug in the subclass rather than the library author > breaching backwards compatibility guarantees. > > > New Ways of Using Classes > ========================= > > Subclass registration > --------------------- > > Especially when writing a plugin system, one likes to register new > subclasses of a plugin baseclass. This can be done as follows:: > > class PluginBase(Object): What is "Object"? I presume just a typo for "object"? > subclasses = [] > > def __init_subclass__(cls, **kwargs): > super().__init_subclass__(**kwargs) > cls.subclasses.append(cls) > > In this example, ``PluginBase.subclasses`` will contain a plain list of all > subclasses in the entire inheritance tree. One should note that this also > works nicely as a mixin class. > > Trait descriptors > ----------------- > > There are many designs of Python descriptors in the wild which, for > example, check boundaries of values. Often those "traits" need some support > of a metaclass to work. This is how this would look like with this > PEP:: > > class Trait: > def __init__(self, minimum, maximum): > self.minimum = minimum > self.maximum = maximum > > def __get__(self, instance, owner): > return instance.__dict__[self.key] > > def __set__(self, instance, value): > if self.minimum < value < self.maximum: > instance.__dict__[self.key] = value > else: > raise ValueError("value not in range") > > def __set_owner__(self, owner, name): I wonder if this should be renamed to __set_name__ or something else that clarifies we're passing it the name of the attribute? The method name __set_owner__ made me assume this is about the owning object (which is often a useful term in other discussions about objects), whereas it is really about telling the descriptor the name of the attribute for which it applies. > self.key = name > > Implementation Details > ====================== > > For those who prefer reading Python over english, the following is a Python > equivalent of the C API changes proposed in this PEP, where the new ``object`` > and ``type`` defined here inherit from the usual ones:: That (inheriting type from type, and object from object) is very confusing. Why not just define new classes e.g. NewType and NewObject here, since it's just pseudo code anyway? > import types > > class type(type): > def __new__(cls, *args, **kwargs): > if len(args) == 1: > return super().__new__(cls, args[0]) > name, bases, ns = args > init = ns.get('__init_subclass__') > if isinstance(init, types.FunctionType): > ns['__init_subclass__'] = classmethod(init) > self = super().__new__(cls, name, bases, ns) > for k, v in self.__dict__.items(): > func = getattr(v, '__set_owner__', None) > if func is not None: > func(self, k) > super(self, self).__init_subclass__(**kwargs) > return self > > def __init__(self, name, bases, ns, **kwargs): > super().__init__(name, bases, ns) What does this definition of __init__ add? > class object: > @classmethod > def __init_subclass__(cls): > pass > > class object(object, metaclass=type): > pass Eek! Too many things named object. > In this code, first the ``__set_owner__`` are called on the descriptors, and > then the ``__init_subclass__``. This means that subclass initializers already > see the fully initialized descriptors. This way, ``__init_subclass__`` users > can fix all descriptors again if this is needed. > > Another option would have been to call ``__set_owner__`` in the base > implementation of ``object.__init_subclass__``. This way it would be possible > event to prevent ``__set_owner__`` from being called. Most of the times, event to prevent??? > however, such a prevention would be accidental, as it often happens that a > call > to ``super()`` is forgotten. > > Another small change should be noted here: in the current implementation of > CPython, ``type.__init__`` explicitly forbids the use of keyword arguments, > while ``type.__new__`` allows for its attributes to be shipped as keyword > arguments. This is weirdly incoherent, and thus the above code forbids that. > While it would be possible to retain the current behavior, it would be better > if this was fixed, as it is probably not used at all: the only use case would > be that at metaclass calls its ``super().__new__`` with *name*, *bases* and > *dict* (yes, *dict*, not *namespace* or *ns* as mostly used with modern > metaclasses) as keyword arguments. This should not be done. > > As a second change, the new ``type.__init__`` just ignores keyword > arguments. Currently, it insists that no keyword arguments are given. This > leads to a (wanted) error if one gives keyword arguments to a class > declaration > if the metaclass does not process them. Metaclass authors that do want to > accept keyword arguments must filter them out by overriding ``__init___``. > > In the new code, it is not ``__init__`` that complains about keyword > arguments, > but ``__init_subclass__``, whose default implementation takes no arguments. In > a classical inheritance scheme using the method resolution order, each > ``__init_subclass__`` may take out it's keyword arguments until none are left, > which is checked by the default implementation of ``__init_subclass__``. I called this out previously, and I am still a bit uncomfortable with the backwards incompatibility here. But I believe what you describe here is the compromise proposed by Nick, and if that's the case I have peace with it. > Rejected Design Options > ======================= > > > Calling the hook on the class itself > ------------------------------------ > > Adding an ``__autodecorate__`` hook that would be called on the class > itself was the proposed idea of PEP 422. Most examples work the same > way or even better if the hook is called on the subclass. In general, > it is much easier to explicitly call the hook on the class in which it > is defined (to opt-in to such a behavior) than to opt-out, meaning > that one does not want the hook to be called on the class it is > defined in. > > This becomes most evident if the class in question is designed as a > mixin: it is very unlikely that the code of the mixin is to be > executed for the mixin class itself, as it is not supposed to be a > complete class on its own. > > The original proposal also made major changes in the class > initialization process, rendering it impossible to back-port the > proposal to older Python versions. > > More importantly, having a pure Python implementation allows us to > take two preliminary steps before before we actually change the > interpreter, giving us the chance to iron out all possible wrinkles > in the API. > > > Other variants of calling the hook > ---------------------------------- > > Other names for the hook were presented, namely ``__decorate__`` or > ``__autodecorate__``. This proposal opts for ``__init_subclass__`` as > it is very close to the ``__init__`` method, just for the subclass, > while it is not very close to decorators, as it does not return the > class. > > > Requiring an explicit decorator on ``__init_subclass__`` > -------------------------------------------------------- > > One could require the explicit use of ``@classmethod`` on the > ``__init_subclass__`` decorator. It was made implicit since there's no > sensible interpretation for leaving it out, and that case would need > to be detected anyway in order to give a useful error message. > > This decision was reinforced after noticing that the user experience of > defining ``__prepare__`` and forgetting the ``@classmethod`` method > decorator is singularly incomprehensible (particularly since PEP 3115 > documents it as an ordinary method, and the current documentation doesn't > explicitly say anything one way or the other). > > A more ``__new__``-like hook > ---------------------------- > > In PEP 422 the hook worked more like the ``__new__`` method than the > ``__init__`` method, meaning that it returned a class instead of > modifying one. This allows a bit more flexibility, but at the cost > of much harder implementation and undesired side effects. > > Adding a class attribute with the attribute order > ------------------------------------------------- > > This got its own PEP 520. > > > History > ======= > > This used to be a competing proposal to PEP 422 by Nick Coghlan and Daniel > Urban. PEP 422 intended to achieve the same goals as this PEP, but with a > different way of implementation. In the meantime, PEP 422 has been withdrawn > favouring this approach. > > References > ========== > > .. _more than 10 years ago: > http://mail.python.org/pipermail/python-dev/2001-November/018651.html > > .. _Zope's ExtensionClass: > http://docs.zope.org/zope_secrets/extensionclass.html > > > Copyright > ========= > > This document has been placed in the public domain. > > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
