Re: [Python-Dev] What is the design purpose of metaclasses vs code generating decorators? (was Re: PEP 557: Data Classes)
> Things that will not work if Enum does not have a metaclass: > > list(EnumClass) -> list of enum members > dir(EnumClass) -> custom list of "interesting" items > len(EnumClass) -> number of members > member in EnumClass -> True or False > > - protection from adding, deleting, and changing members > - guards against reusing the same name twice > - possible to have properties and members with the same name (i.e. "value" > and "name") In current Python this is true. But if we would go down the route of PEP 560 (which I just found, I wasn't involved in its discussion), then we could just add all the needed functionality to classes. I would do it slightly different than proposed in PEP 560: classmethods are very similar to methods on a metaclass. They are just not called by the special method machinery. I propose that the following is possible: >>> class Spam: ... @classmethod ... def __getitem__(self, item): ... return "Ham" >>> Spam[3] Ham this should solve most of your usecases. When thinking about how an automatic metaclass combiner would look like, I realized that it should ideally just reproduce the class mro, just with metaclasses. So if a class has an mro of [A, B, C, object], its metaclass should have an mro of unique_everseen([type(A), type(B), type(C), type]). But in this case, why add this layer at all? Just give the class the ability to do everything a metaclass could do, using mechanisms like @classmethod, and we're done. Greetings Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What is the design purpose of metaclasses vs code generating decorators? (was Re: PEP 557: Data Classes)
>> I do worry that things like your autoslots decorator example might be >> problematic because they create a new class, throwing away a lot of work >> that was already done. But perhaps the right way to address this would be >> to move the decision about the instance layout to a later phase? (Not sure >> if that makes sense though.) >> >> --Guido > > > Just FYI, recreating the class with slots runs into problems with regards to > PEP 3135 (New Super). In attrs we resort to black magic to update the > __class__ cell in existing methods. Being able to add slotness later would > be good, but not that useful for us since we have to support down to 2.7. You're both bringing up an important point here: while in function decorators it is normal to return a completely new function (albeit one that wraps the original), this is close to impossible for classes. You cannot just wrap a class in another one. You may inherit from it, but that's already often not what one wants. While I am not worried about the poor computers having to do a lot of work creating a throwaway class, I do see the problem with the new super. What I would like to see is something like the @wraps decorator for classes, such that you could write something like: def class_decorator(cls): @wraps_class class MyNewCoolClass: """my cool functionality here""" return MyNewCoolClass wraps_class would then copy over everything such that the new class gets it. Unfortunately, this won't work, because of the new super. The new super is about the only thing that cannot be dynamically changed in Python. While it is no problem to make a function a method of a random class (just use __get__), it is not possible to move a function from one class to another, because you cannot change its binding to __class__, which is used by super(). And even if we could, the method whose __class__ we want to change might hide in a wrapper. The current behavior of __class__ is weird, it is set to the class that type.__new__ creates. So even if another metaclasses __new__ or a decorator returns another class, the method's __class__ would still point to the original class, which might even not exist anymore. One might argue that this is due to the fact that it is not well-defined what __class__ should be set to. But this is not true, it is crystal clear: __class__ should be set to the class from whose __dict__ the method was drawn. I thought about a new three-parameter __get__(self, instance, owner, supplier), which would then set __class__ to the supplier. This is a beautiful concept, that is unfortunately not so simple when it comes to methods hidden in wrappers. Greetings Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What is the design purpose of metaclasses vs code generating decorators? (was Re: PEP 557: Data Classes)
On 15 October 2017 at 01:00, Martin Teichmann wrote: > While I am not worried about the poor computers having to do a lot of > work creating a throwaway class, I do see the problem with the new > super. What I would like to see is something like the @wraps decorator > for classes, such that you could write something like: > > def class_decorator(cls): > @wraps_class > class MyNewCoolClass: > """my cool functionality here""" > return MyNewCoolClass > > wraps_class would then copy over everything such that the new class gets > it. > > Unfortunately, this won't work, because of the new super. The new > super is about the only thing that cannot be dynamically changed in > Python. While it is no problem to make a function a method of a random > class (just use __get__), it is not possible to move a function from > one class to another, because you cannot change its binding to > __class__, which is used by super(). And even if we could, the method > whose __class__ we want to change might hide in a wrapper. > Up until 3.6, we made it so that the class cell for zero-arg super was an almost entirely hidden implementation detail at class creation time. To allow zero-arg super() class methods to work from __init_subclass__, we changed that to include __classcell__ in the execution namespace passed to the metaclass. So it seems to me that to enable the class *replacement* use case, we'd only need one new thing: access to that cell as an attribute on the class itself. That way, when creating the new class you could set "ns['__classcell__'] = old_cls.__classcell__" before calling the new metaclass, which would allow the new class to take over zero-arg super() resolution for any methods defined lexically inside the original. For the class *duplication* use case (where you want to leave the original class intact), the main thing this would let you do is to reliably detect that there is at least one method in the class body using the zero-arg super() form (when cls.__classcell__ is non-None), and either issue a warning or fail outright (since methods that rely on cooperative multiple inheritance need a specific defining class or else the runtime parent resolution gets inconsistent). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What is the design purpose of metaclasses vs code generating decorators? (was Re: PEP 557: Data Classes)
On 10/14/2017 07:37 AM, Martin Teichmann wrote: Things that will not work if Enum does not have a metaclass: list(EnumClass) -> list of enum members dir(EnumClass) -> custom list of "interesting" items len(EnumClass) -> number of members member in EnumClass -> True or False - protection from adding, deleting, and changing members - guards against reusing the same name twice - possible to have properties and members with the same name (i.e. "value" and "name") In current Python this is true. But if we would go down the route of PEP 560 (which I just found, I wasn't involved in its discussion), then we could just add all the needed functionality to classes. I would do it slightly different than proposed in PEP 560: classmethods are very similar to methods on a metaclass. They are just not called by the special method machinery. I propose that the following is possible: >>> class Spam: ... @classmethod ... def __getitem__(self, item): ... return "Ham" >>> Spam[3] Ham this should solve most of your usecases. The problem with your solution is you couldn't then have a __getitem__ for the instances -- it's an either/or situation. The problem with PEP 560 is that it doesn't allow the class definition protections that a metaclass does. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What is the design purpose of metaclasses vs code generating decorators? (was Re: PEP 557: Data Classes)
On 14 October 2017 at 17:49, Ethan Furman wrote: > The problem with PEP 560 is that it doesn't allow the class definition > protections that a metaclass does. > Since the discussion turned to PEP 560, I can say that I don't want this to be a general mechanism, the PEP rationale section gives several specific examples why we don't want metaclasses to implement generic class machinery/internals. Could you please elaborate more what is wrong with PEP 560 and what do you mean by "class definition protections" -- Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 560 vs metaclass' class definition protections [was Re: What is the design purpose of metaclasses ...]
On 10/14/2017 08:57 AM, Ivan Levkivskyi wrote: On 14 October 2017 at 17:49, Ethan Furman wrote: The problem with PEP 560 is that it doesn't allow the class definition >> protections that a metaclass does. Since the discussion turned to PEP 560, I can say that I don't want this > to be a general mechanism, the PEP rationale section gives several specific > examples why we don't want metaclasses to implement generic class > machinery/internals. Could you please elaborate more what is wrong with PEP 560 and what do you > mean by "class definition protections" Nothing is wrong with PEP 560. What I am referring to is: class MyEnum(Enum): red = 0 red = 1 The Enum metaclass machinery will raise an error at the "red = 1" line because it detects the redefinition of "red". This check can only happen during class definition, so only the metaclass can do it. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 560 vs metaclass' class definition protections [was Re: What is the design purpose of metaclasses ...]
On 15 October 2017 at 02:14, Ethan Furman wrote: > On 10/14/2017 08:57 AM, Ivan Levkivskyi wrote: > >> On 14 October 2017 at 17:49, Ethan Furman wrote: >> > > The problem with PEP 560 is that it doesn't allow the class definition >>> >> >> protections that a metaclass does. > >> >> Since the discussion turned to PEP 560, I can say that I don't want this >> > > to be a general mechanism, the PEP rationale section gives several > specific > > examples why we don't want metaclasses to implement generic class > > machinery/internals. > >> >> Could you please elaborate more what is wrong with PEP 560 and what do you >> > > mean by "class definition protections" > > Nothing is wrong with PEP 560. What I am referring to is: > > class MyEnum(Enum): >red = 0 >red = 1 > > The Enum metaclass machinery will raise an error at the "red = 1" line > because it detects the redefinition of "red". This check can only happen > during class definition, so only the metaclass can do it. > That's not necessarily an inherent restriction though - if we did decide to go even further in the direction of "How do we let base classes override semantics that currently require a custom metaclass?", then there's a fairly clear parallel between "mcl.__init__/bases.__init_subclass__" and "mcl.__prepare__/bases.__prepare_subclass__". OTOH, if you have multiple bases with competing __prepare__ methods you really *should* get a metaclass conflict, since the class body can only be executed in one namespace. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 560 vs metaclass' class definition protections [was Re: What is the design purpose of metaclasses ...]
On 14 October 2017 at 18:14, Ethan Furman wrote: > On 10/14/2017 08:57 AM, Ivan Levkivskyi wrote: > >> >> Could you please elaborate more what is wrong with PEP 560 and what do >> you mean by "class definition protections" >> > > Nothing is wrong with PEP 560. What I am referring to is: > [snip] > > OK thanks, then let us keep PEP 560 to its original scope. Its design is specific to generic classes, so it will probably not help with "wider" metaclass problems. As a side note, I don't think elimination of metaclasses should be a "goal by itself". This is a powerful and flexible mechanism, but there are specific situations where metaclasses don't work well because of e.g. frequent conflicts or performance penalties. -- Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 560 vs metaclass' class definition protections [was Re: What is the design purpose of metaclasses ...]
On 10/14/2017 11:30 AM, Ivan Levkivskyi wrote: As a side note, I don't think elimination of metaclasses should be a "goal by itself". This is a powerful and flexible mechanism, but there are specific situations where metaclasses don't work well because of e.g. frequent conflicts or performance penalties. +1 -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
