[Python-Dev] Re: __init_subclass__ and metaclasses

2020-12-24 Thread Joao S. O. Bueno
Actually, there are a few steps that `type.__new__`  perform that are not
customizable in metaclasses.

I had sometimes thought about mailing this here, or Python ideas, but could
not
come up with a "real world" use case where the customization of those
would be meaningful.

Let'me see if I recall all cases - two of them are the calls to
`__init_subclass__` and the descriptors `__set_name__` as you put it,
I think there is a third behavior that can't be separated from
`type.__new__` - but
I can't remember it now


Anyway, the "thing to do" that always occurred to me about it is to add
"soft" method slots
to `type` itself - so that `type.__new__` would call those  on the
corresponding initialization phases.

Since these are to be run only when classes are created, their impact
should be negligible.

In other words, have `type` implement methods like `__run_init_subclass__`,
`__run_descriptor_setname__`,
(and one for the other task I can't remember now). So, all metaclass code
written up to today remains valid,
and these behaviors become properly customizable.

Adding keyword parameters to `type.__new__`, IMHO, besides a little bit
fishy as we are talking of
arguments to change the behavior of the method, would themselves compete
and have to be
filtered out, or otherwise special-cased in the `__init_subclass__` method
itself.
I mean - let's suppose we add `__suppress_init_subclass__` as an named
parameter to
`type.__new__` - what would happen with this argument in
`__init_subclass__` ? Would it show
up in the kwargs? Otherwise it would be the _only_  kwarg popped out and
not passed
to __init_subclass__, being an inconvenient exception.

Having an overridable, separate, method in type to run __init_subclass__
and __set_name__
bypass these downsides.

In time, Happy holidays everyone!

   js
 -><-

On Fri, 25 Dec 2020 at 00:38, Ethan Furman  wrote:

> PEP 487 introduced __init_subclass__ and __set_name__, and both of those
> were wins for the common cases of metaclass usage.
>
> Unfortunately, the implementation of PEP 487 with regards to
> __init_subclass__ has made the writing of correct
> metaclasses significantly harder, if not impossible.
>
> The cause is that when a metaclass calls type.__new__ to actually create
> the class, type.__new__ calls the
> __init_subclass__ methods of the new class' parents, passing it the newly
> created, but incomplete, class.  In code:
>
> ```
> class Meta(type):
>  #
>  def __new__(mcls, name, bases, namespace, **kwds):
>  # create new class, which will call __init_subclass__ and
> __set_name__
>  new_class = type.__new__(mcls, name, bases, namespace, **kwds)
>  # finish setting up class
>  new_class.some_attr = 9
> ```
>
> As you can deduce, when the parent __init_subclass__ is called with the
> new class, `some_attr` has not been added yet --
> the new class is incomplete.
>
> For Enum, this means that __init_subclass__ doesn't have access to the new
> Enum's members (they haven't beet added yet).
>
> For ABC, this means that __init_subclass__ doesn't have access to
> __abstract_methods__ (it hasn't been created yet).
>
> Because Enum is pure Python code I was able to work around it:
> - remove new __init_subclass__ (if it exists)
> - insert dummy class with a no-op __init_subclass__
> - call type.__new__
> - save any actual __init_subclass__
> - add back any new __init_subclass__
> - rewrite the new class' __bases__, removing the no-op dummy class
> - finish creating the class
> - call the parent __init_subclass__ with the now complete Enum class
>
> I have not been able to work around the problem for ABC.
>
> Two possible solutions I can think of:
>
> - pass a keyword argument to type.__new__ that suppresses the call to
> __init_subclass__; and
> - provide a way to invoke new class' parent's __init_subclass__ before
> returning it
>
> or
>
> - instead of type.__new__ doing that work, have type.__init__ do it.
>
> Thoughts?
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ZMRRNSFSLJZDGGZ66CFCYQBINU62CDNX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OCHSRMW44UUA5ESY2W54KTN64MZH2KFE/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] __init_subclass__ and metaclasses

2020-12-24 Thread Ethan Furman

PEP 487 introduced __init_subclass__ and __set_name__, and both of those were 
wins for the common cases of metaclass usage.

Unfortunately, the implementation of PEP 487 with regards to __init_subclass__ has made the writing of correct 
metaclasses significantly harder, if not impossible.


The cause is that when a metaclass calls type.__new__ to actually create the class, type.__new__ calls the 
__init_subclass__ methods of the new class' parents, passing it the newly created, but incomplete, class.  In code:


```
class Meta(type):
#
def __new__(mcls, name, bases, namespace, **kwds):
# create new class, which will call __init_subclass__ and __set_name__
new_class = type.__new__(mcls, name, bases, namespace, **kwds)
# finish setting up class
new_class.some_attr = 9
```

As you can deduce, when the parent __init_subclass__ is called with the new class, `some_attr` has not been added yet -- 
the new class is incomplete.


For Enum, this means that __init_subclass__ doesn't have access to the new 
Enum's members (they haven't beet added yet).

For ABC, this means that __init_subclass__ doesn't have access to 
__abstract_methods__ (it hasn't been created yet).

Because Enum is pure Python code I was able to work around it:
- remove new __init_subclass__ (if it exists)
- insert dummy class with a no-op __init_subclass__
- call type.__new__
- save any actual __init_subclass__
- add back any new __init_subclass__
- rewrite the new class' __bases__, removing the no-op dummy class
- finish creating the class
- call the parent __init_subclass__ with the now complete Enum class

I have not been able to work around the problem for ABC.

Two possible solutions I can think of:

- pass a keyword argument to type.__new__ that suppresses the call to 
__init_subclass__; and
- provide a way to invoke new class' parent's __init_subclass__ before 
returning it

or

- instead of type.__new__ doing that work, have type.__init__ do it.

Thoughts?

--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZMRRNSFSLJZDGGZ66CFCYQBINU62CDNX/
Code of Conduct: http://python.org/psf/codeofconduct/