Re: [Python-Dev] PEP 560: bases classes / confusion

[Ivan Levkivskyi]

On 16 November 2017 at 18:58, Ethan Furman  wrote:

> On 11/16/2017 04:22 AM, Ivan Levkivskyi wrote:
>
>> On 16 November 2017 at 07:56, Nick Coghlan wrote:
>>
>
> Jim also raised an important point that needs clarification at the spec
>>>
>> >> level: given multiple entries in "orig_bases" with __mro_entries__
> methods,
> >> do all such methods get passed the *same* orig_bases tuple? Or do they
>
>> receive partially resolved ones, such that bases listed before them have
>>>
>> >> already been resolved to their MRO entries by the time they run.
>
>>
>> Yes, they all get the same initial bases tuple as an argument. Passing
>>
> > updated ones will cost a bit more and I don't think it will be needed
> > (in the worst case a base can resolve another base by calling its
> > __mro_entries__ manually).  I will clarify this in the PEP.
>
> If the extra complexity is to:
>
> > - given orig_bases, a method could avoid injecting bases already listed
> >   if it wanted to
> > - allowing multiple items to be returned provides a way to
> programmatically
> >   combine mixins without having to define a new subclass for each
> combination
>
> And each method is passed the same original tuple (without other methods'
> updates) then don't we end up in a situation where we can have duplicates
> base classes?
>

Not that it is impossible now (in certain sense):

class MultiMeta(type):
def __new__(mcls, name, bases, ns):
return super().__new__(mcls, name, (), ns)

class MultiBase(metaclass=MultiMeta):
pass

class C(MultiBase, list, list, MultiBase, dict, dict, dict):  # OK
pass

What is probably confusing in the current PEP text, is that it doesn't say
clearly that
the substitution happens before any other steps in __build_class__.
Therefore all normal checks (like duplicate bases and MRO consistency)
happen and e.g.

class C(List[int], List[str]):
pass

will fail with:

TypeError: duplicate base class list

(by the way while playing with this I have found a bug in the reference
implementation)

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Ethan Furman]

On 11/16/2017 04:22 AM, Ivan Levkivskyi wrote:

On 16 November 2017 at 07:56, Nick Coghlan wrote:



Jim also raised an important point that needs clarification at the spec

>> level: given multiple entries in "orig_bases" with __mro_entries__ methods,
>> do all such methods get passed the *same* orig_bases tuple? Or do they

receive partially resolved ones, such that bases listed before them have

>> already been resolved to their MRO entries by the time they run.


Yes, they all get the same initial bases tuple as an argument. Passing

> updated ones will cost a bit more and I don't think it will be needed
> (in the worst case a base can resolve another base by calling its
> __mro_entries__ manually).  I will clarify this in the PEP.

If the extra complexity is to:

> - given orig_bases, a method could avoid injecting bases already listed
>   if it wanted to
> - allowing multiple items to be returned provides a way to programmatically
>   combine mixins without having to define a new subclass for each combination

And each method is passed the same original tuple (without other methods' updates) then don't we end up in a situation 
where we can have duplicates base classes?


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Koos Zevenhoven]

On Thu, Nov 16, 2017 at 6:28 PM, brent bejot  wrote:

> Hello all,
>
> Noticed that "MRO" is not actually defined in the PEP and it seems like it
> should be.  Probably in the Performance section where the abbreviation is
> first used outside of a function name.
>
>
​I don't think it will hurt if I suggest that __bases__, bases, "original
bases"​, mro, __orig_bases__, MRO, __mro__ and "concatenated mro entries"
are all defined as synonyms of each other, except with different meanings
:-)

––Koos



> -Brent
>
> On Thu, Nov 16, 2017 at 7:22 AM, Ivan Levkivskyi 
> wrote:
>
>> On 16 November 2017 at 07:56, Nick Coghlan  wrote:
>>
>>> On 16 November 2017 at 04:39, Ivan Levkivskyi 
>>> wrote:
>>>
 Nick is exactly right here. Jim, if you want to propose alternative
 wording, then we could consider it.

>>>
>>> Jim also raised an important point that needs clarification at the spec
>>> level: given multiple entries in "orig_bases" with __mro_entries__ methods,
>>> do all such methods get passed the *same* orig_bases tuple? Or do they
>>> receive partially resolved ones, such that bases listed before them have
>>> already been resolved to their MRO entries by the time they run.
>>>
>>>
>>>
>> Yes, they all get the same initial bases tuple as an argument. Passing
>> updated ones will cost a bit more and I don't think it will be needed (in
>> the worst case a base can resolve another base by calling its
>> __mro_entries__ manually).
>> I will clarify this in the PEP.
>>
>> --
>> Ivan
>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/brent.
>> bejot%40gmail.com
>>
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> k7hoven%40gmail.com
>
>


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[brent bejot]

Hello all,

Noticed that "MRO" is not actually defined in the PEP and it seems like it
should be.  Probably in the Performance section where the abbreviation is
first used outside of a function name.

-Brent

On Thu, Nov 16, 2017 at 7:22 AM, Ivan Levkivskyi 
wrote:

> On 16 November 2017 at 07:56, Nick Coghlan  wrote:
>
>> On 16 November 2017 at 04:39, Ivan Levkivskyi 
>> wrote:
>>
>>> Nick is exactly right here. Jim, if you want to propose alternative
>>> wording, then we could consider it.
>>>
>>
>> Jim also raised an important point that needs clarification at the spec
>> level: given multiple entries in "orig_bases" with __mro_entries__ methods,
>> do all such methods get passed the *same* orig_bases tuple? Or do they
>> receive partially resolved ones, such that bases listed before them have
>> already been resolved to their MRO entries by the time they run.
>>
>>
>>
> Yes, they all get the same initial bases tuple as an argument. Passing
> updated ones will cost a bit more and I don't think it will be needed (in
> the worst case a base can resolve another base by calling its
> __mro_entries__ manually).
> I will clarify this in the PEP.
>
> --
> Ivan
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brent.bejot%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Ivan Levkivskyi]

On 16 November 2017 at 07:56, Nick Coghlan  wrote:

> On 16 November 2017 at 04:39, Ivan Levkivskyi 
> wrote:
>
>> Nick is exactly right here. Jim, if you want to propose alternative
>> wording, then we could consider it.
>>
>
> Jim also raised an important point that needs clarification at the spec
> level: given multiple entries in "orig_bases" with __mro_entries__ methods,
> do all such methods get passed the *same* orig_bases tuple? Or do they
> receive partially resolved ones, such that bases listed before them have
> already been resolved to their MRO entries by the time they run.
>
>
>
Yes, they all get the same initial bases tuple as an argument. Passing
updated ones will cost a bit more and I don't think it will be needed (in
the worst case a base can resolve another base by calling its
__mro_entries__ manually).
I will clarify this in the PEP.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Nick Coghlan]

On 16 November 2017 at 04:39, Ivan Levkivskyi  wrote:

> Nick is exactly right here. Jim, if you want to propose alternative
> wording, then we could consider it.
>

Jim also raised an important point that needs clarification at the spec
level: given multiple entries in "orig_bases" with __mro_entries__ methods,
do all such methods get passed the *same* orig_bases tuple? Or do they
receive partially resolved ones, such that bases listed before them have
already been resolved to their MRO entries by the time they run.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560

[Terry Reedy]

On 11/14/2017 3:26 PM, Ivan Levkivskyi wrote:
After some discussion on python-ideas, see 
https://mail.python.org/pipermail/python-ideas/2017-September/047220.html, 
this PEP received positive comments. The updated version that takes into 
account the comments that appeared in the discussion so far is available 
at https://www.python.org/dev/peps/pep-0560/


Here I post the full text for convenience:

++

PEP: 560
Title: Core support for typing module and generic types
Author: Ivan Levkivskyi >
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 03-Sep-2017
Python-Version: 3.7
Post-History: 09-Sep-2017

...

Suggested wording improvements:


Performance
---

The ``typing`` module is one of the heaviest and slowest modules in
the standard library even with all the optimizations made.



Mainly this is
because of subscripted generic types (see PEP 484 for definition of terms
used in this PEP) are class objects (see also [1]_).


Delete 'of' after 'because' to make this a proper sentence.


The three main ways how


"There are three ..." reads better to me.

the performance can be improved with the help of the proposed special 
methods:


- Creation of generic classes is slow since the ``GenericMeta.__new__`` is
   very slow; we will not need it anymore.

- Very long MROs for generic classes will be twice shorter;


I  believe by 'twice shorter', which is meaningless by itself, you mean 
'half as long'.  If so, please say the latter.

 > they are present

   because we duplicate the ``collections.abc`` inheritance chain
   in ``typing``.

- Time of instantiation of generic classes will be improved


Instantiation of generic classes will be faster.


   (this is minor however).


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Koos Zevenhoven]

On Wed, Nov 15, 2017 at 5:37 PM, Nick Coghlan  wrote:

> On 16 November 2017 at 00:20, Jim J. Jewett  wrote:
>
>> I *think* the following will happen:
>>
>> "NewList[int]" will be evaluated, and __class_getitem__ called, so
>> that the bases tuple will be (A, GenericAlias(NewList, int), B)
>>
>> # (A)  I *think* __mro_entries__ gets called with the full tuple,
>> # instead of just the object it is found on.
>> # (B) I *think* it is called on the results of evaluating
>> # the terms within the tuple, instead of the original
>> # string representation.
>> _tmp = __mro_entries__(A, GenericAlias(NewList, int), B)
>>
>> # (C)  I *think* __mro_entries__ returns a replacement for
>> # just the single object, even though it was called on
>> # the whole tuple, without knowing which object it
>> # represents.
>> bases = (A, _tmp, B)
>>
>
> My understanding of the method signature:
>
> def __mro_entries__(self, orig_bases):
> ...
> return replacement_for_self
>
> My assumption as to the purpose of the extra complexity was:
>
> - given orig_bases, a method could avoid injecting bases already listed if
> it wanted to
> - allowing multiple items to be returned provides a way to
> programmatically combine mixins without having to define a new subclass for
> each combination
>
>

​Thanks, this might provide an answer to my question about multiple mro
entries here

https://mail.python.org/pipermail/python-ideas/2017-November/047897.html​

​––Koos​



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Ivan Levkivskyi]

Nick is exactly right here. Jim, if you want to propose alternative
wording, then we could consider it.

--
Ivan


On 15 November 2017 at 16:37, Nick Coghlan  wrote:

> On 16 November 2017 at 00:20, Jim J. Jewett  wrote:
>
>> I *think* the following will happen:
>>
>> "NewList[int]" will be evaluated, and __class_getitem__ called, so
>> that the bases tuple will be (A, GenericAlias(NewList, int), B)
>>
>> # (A)  I *think* __mro_entries__ gets called with the full tuple,
>> # instead of just the object it is found on.
>> # (B) I *think* it is called on the results of evaluating
>> # the terms within the tuple, instead of the original
>> # string representation.
>> _tmp = __mro_entries__(A, GenericAlias(NewList, int), B)
>>
>> # (C)  I *think* __mro_entries__ returns a replacement for
>> # just the single object, even though it was called on
>> # the whole tuple, without knowing which object it
>> # represents.
>> bases = (A, _tmp, B)
>>
>
> My understanding of the method signature:
>
> def __mro_entries__(self, orig_bases):
> ...
> return replacement_for_self
>
> My assumption as to the purpose of the extra complexity was:
>
> - given orig_bases, a method could avoid injecting bases already listed if
> it wanted to
> - allowing multiple items to be returned provides a way to
> programmatically combine mixins without having to define a new subclass for
> each combination
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Nick Coghlan]

On 16 November 2017 at 00:20, Jim J. Jewett  wrote:

> I *think* the following will happen:
>
> "NewList[int]" will be evaluated, and __class_getitem__ called, so
> that the bases tuple will be (A, GenericAlias(NewList, int), B)
>
> # (A)  I *think* __mro_entries__ gets called with the full tuple,
> # instead of just the object it is found on.
> # (B) I *think* it is called on the results of evaluating
> # the terms within the tuple, instead of the original
> # string representation.
> _tmp = __mro_entries__(A, GenericAlias(NewList, int), B)
>
> # (C)  I *think* __mro_entries__ returns a replacement for
> # just the single object, even though it was called on
> # the whole tuple, without knowing which object it
> # represents.
> bases = (A, _tmp, B)
>

My understanding of the method signature:

def __mro_entries__(self, orig_bases):
...
return replacement_for_self

My assumption as to the purpose of the extra complexity was:

- given orig_bases, a method could avoid injecting bases already listed if
it wanted to
- allowing multiple items to be returned provides a way to programmatically
combine mixins without having to define a new subclass for each combination

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Koos Zevenhoven]

For anyone confused about similar things, I expect you to be interested in
my post on python-ideas from today:

https://mail.python.org/pipermail/python-ideas/2017-November/047896.html

––Koos


On Wed, Nov 15, 2017 at 4:20 PM, Jim J. Jewett  wrote:

> (1)  I found the following (particularly "bases classes") very confusing:
>
> """
> If an object that is not a class object appears in the bases of a class
>
> definition, then ``__mro_entries__`` is searched on it. If found,
> it is called with the original tuple of bases as an argument. The result
> of the call must be a tuple, that is unpacked in the bases classes in place
> of this object. (If the tuple is empty, this means that the original bases
> is
> simply discarded.)
> """
>
> Based on the following GenericAlias/NewList/Tokens example, I think I
> now I understand what you mean, and would have had somewhat less
> difficulty if it were expressed as:
>
> """
> When an object that is not a class object appears in the (tuple of)
> bases of a class
> definition, then attribute ``__mro_entries__`` is searched on that
> non-class object.  If ``__mro_entries__`` found,
> it is called with the entire original tuple of bases as an argument. The
> result
> of the call must be a tuple, which is unpacked and replaces only the
> non-class object in the tuple of bases.  (If the tuple is empty, this
> means that the original bases
> is
> simply discarded.)
> """
>
> Note that this makes some assumptions about the __mro_entries__
> signature that I wasn't quite sure about from the example.  So
> building on that:
>
> class ABList(A, NewList[int], B):
>
> I *think* the following will happen:
>
> "NewList[int]" will be evaluated, and __class_getitem__ called, so
> that the bases tuple will be (A, GenericAlias(NewList, int), B)
>
> # (A)  I *think* __mro_entries__ gets called with the full tuple,
> # instead of just the object it is found on.
> # (B) I *think* it is called on the results of evaluating
> # the terms within the tuple, instead of the original
> # string representation.
> _tmp = __mro_entries__(A, GenericAlias(NewList, int), B)
>
> # (C)  I *think* __mro_entries__ returns a replacement for
> # just the single object, even though it was called on
> # the whole tuple, without knowing which object it
> # represents.
> bases = (A, _tmp, B)
>
> # (D) If there are two non-class objects, I *think* the
> # second one gets the same arguments as the first,
> # rather than an intermediate tuple with the first such
> # object already substituted out.
>
> -jJ
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> k7hoven%40gmail.com
>



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560

[Nick Coghlan]

On 15 November 2017 at 06:26, Ivan Levkivskyi  wrote:

> After some discussion on python-ideas, see https://mail.python.org/
> pipermail/python-ideas/2017-September/047220.html, this PEP received
> positive comments. The updated version that takes into account the comments
> that appeared in the discussion so far is available at
> https://www.python.org/dev/peps/pep-0560/
>

I don't have anything to add to the python-ideas comments you already
incorporated, so +1 for this version from me.

* ``importlib.reload(typing)`` is up to 7x faster
>

Nice! That's getting much closer to the "negligible" range, even for
command line apps.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560 vs metaclass' class definition protections [was Re: What is the design purpose of metaclasses ...]

[Ethan Furman]

On 10/14/2017 11:30 AM, Ivan Levkivskyi wrote:


As a side note, I don't think elimination of metaclasses should be a "goal by 
itself".
This is a powerful and flexible mechanism, but there are specific situations 
where
metaclasses don't work well because of e.g. frequent conflicts or performance 
penalties.


+1

--
~Ethan~

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560 vs metaclass' class definition protections [was Re: What is the design purpose of metaclasses ...]

[Ivan Levkivskyi]

On 14 October 2017 at 18:14, Ethan Furman  wrote:

> On 10/14/2017 08:57 AM, Ivan Levkivskyi wrote:
>
>>
>> Could you please elaborate more what is wrong with PEP 560 and what do
>> you mean by "class definition protections"
>>
>
> Nothing is wrong with PEP 560.  What I am referring to is:
>
[snip]
>
>
OK thanks, then let us keep PEP 560 to its original scope. Its design is
specific to generic classes,
so it will probably not help with "wider" metaclass problems.

As a side note, I don't think elimination of metaclasses should be a "goal
by itself".
This is a powerful and flexible mechanism, but there are specific
situations where
metaclasses don't work well because of e.g. frequent conflicts or
performance penalties.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560 vs metaclass' class definition protections [was Re: What is the design purpose of metaclasses ...]

[Nick Coghlan]

On 15 October 2017 at 02:14, Ethan Furman  wrote:

> On 10/14/2017 08:57 AM, Ivan Levkivskyi wrote:
>
>> On 14 October 2017 at 17:49, Ethan Furman wrote:
>>
>
> The problem with PEP 560 is that it doesn't allow the class definition
>>>
>> >> protections that a metaclass does.
>
>>
>> Since the discussion turned to PEP 560, I can say that I don't want this
>>
> > to be a general mechanism, the PEP rationale section gives several
> specific
> > examples why we don't want metaclasses to implement generic class
> > machinery/internals.
>
>>
>> Could you please elaborate more what is wrong with PEP 560 and what do you
>>
> > mean by "class definition protections"
>
> Nothing is wrong with PEP 560.  What I am referring to is:
>
> class MyEnum(Enum):
>red = 0
>red = 1
>
> The Enum metaclass machinery will raise an error at the "red = 1" line
> because it detects the redefinition of "red". This check can only happen
> during class definition, so only the metaclass can do it.
>

That's not necessarily an inherent restriction though - if we did decide to
go even further in the direction of "How do we let base classes override
semantics that currently require a custom metaclass?", then there's a
fairly clear parallel between "mcl.__init__/bases.__init_subclass__" and
"mcl.__prepare__/bases.__prepare_subclass__".

OTOH, if you have multiple bases with competing __prepare__  methods you
really *should* get a metaclass conflict, since the class body can only be
executed in one namespace.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com