Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-11 Thread Nick Coghlan
On 11 September 2016 at 13:05, Nick Coghlan  wrote:
> VOC & Batavia *should* be OK (worst case, they return
> collections.OrderedDict from __prepare__ and also use it for __dict__
> attributes), but I'm less certain about MicroPython (since I don't
> know enough about how its current dict implementation works to know
> whether or not they'll be able to make the same change PyPy and
> CPython did)

Micropython' s Damien George got back to me and indicated that once
they get around to working on Python 3.6 compatibility (they're
currently still working on 3.5), they'd likely also need to go down
the path of using collections.OrderedDict in the situations where the
3.6 language spec calls for it (MicroPython's default dict
implementation is less sparse than the CPython one, trading greater
memory usage efficiency for an increased risk of hash collisions, so
it's unlikely the new implementation would count as "compact" from
that perspective).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-11 Thread Ethan Furman

On 09/11/2016 01:55 AM, Victor Stinner wrote:

2016-09-10 3:49 GMT-04:00 Ethan Furman wrote:



With __definition_order__ Enum can display the actual creation order of enum
members and methods, while relying on Enum.__dict__.keys() presents a
jumbled mess with many attributes the user never wrote, the enum members either
appearing /after/ all the methods (even if actually written before), or
entirely absent.


Python 3.5 also returns methods in Enum.__dict__(). So it would be a
new feature, right?


__definition_order__ is (would be) a new feature, yes.


The use case seems to be specific to Enum. Can't you add a new method
which only returns members (ordered by insertion order)?


The use case is specific to any custom metaclass that does more than enhance 
the attributes and/or methods already in the class body.


list(myenum._member_maps.keys()) returns members, sorted by insertion
order. Is it what you want?


That only includes members, not other attributes nor methods.  What I want is 
to make sure the other points of PEP 520 are not forgotten about, and that Enum 
conforms to the accepted PEP.


Code:
---
import enum

class Color(enum.Enum):
 red = 1
 blue = red
 green = 2

print(Color.__dict__.keys())
print(list(Color._member_map_.keys()))
---

Python 3.5:
---
dict_keys(['__module__', '_member_names_', 'green', '_member_type_',
'blue', '_value2member_map_', '_member_map_', '__new__', 'red',
'__doc__'])
['red', 'blue', 'green']
---

Python 3.6:
---
dict_keys(['_generate_next_value_', '__module__', '__doc__',
'_member_names_', '_member_map_', '_member_type_',
'_value2member_map_', 'red', 'blue', 'green', '__new__'])
['red', 'blue', 'green']
---

Note: It seems like dir(myenum) ignores "aliases" like blue=red in my example.


That is intentional.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-11 Thread Victor Stinner
2016-09-10 3:49 GMT-04:00 Ethan Furman :
> With __definition_order__ Enum can display the actual creation order of enum
> members and methods, while relying on Enum.__dict__.keys() presents a
> jumbled mess with many attributes the user never wrote, the enum members 
> either
> appearing /after/ all the methods (even if actually written before), or
> entirely absent.

Python 3.5 also returns methods in Enum.__dict__(). So it would be a
new feature, right?

The use case seems to be specific to Enum. Can't you add a new method
which only returns members (ordered by insertion order)?

list(myenum._member_maps.keys()) returns members, sorted by insertion
order. Is it what you want?

Code:
---
import enum

class Color(enum.Enum):
red = 1
blue = red
green = 2

print(Color.__dict__.keys())
print(list(Color._member_map_.keys()))
---

Python 3.5:
---
dict_keys(['__module__', '_member_names_', 'green', '_member_type_',
'blue', '_value2member_map_', '_member_map_', '__new__', 'red',
'__doc__'])
['red', 'blue', 'green']
---

Python 3.6:
---
dict_keys(['_generate_next_value_', '__module__', '__doc__',
'_member_names_', '_member_map_', '_member_type_',
'_value2member_map_', 'red', 'blue', 'green', '__new__'])
['red', 'blue', 'green']
---

Note: It seems like dir(myenum) ignores "aliases" like blue=red in my example.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Russell Keith-Magee
On Sun, Sep 11, 2016 at 11:05 AM, Nick Coghlan  wrote:

> On 11 September 2016 at 07:26, Guido van Rossum  wrote:
> > On Sat, Sep 10, 2016 at 10:57 AM, Nick Coghlan 
> wrote:
> >> On 11 September 2016 at 03:08, Guido van Rossum 
> wrote:
> >>> So I'm happy to continue thinking about this, but I expect this is not
> >>> such a big deal as you fear. Anyway, let's see if someone comes up
> >>> with a more convincing argument by beta 2!
> >>
> >> For CPython specifically, I don't have anything more convincing than
> >> Ethan's Enum example (where the way the metaclass works means most of
> >> the interesting attributes don't live directly in the class dict, they
> >> live in private data structures stored in the class dict, making
> >> "list(MyEnum.__dict__)" inherently uninteresting, regardless of
> >> whether it's ordered or not).
> >
> > But that would only matter if we also defined a helper utility that
> > used __definition_order__. I expect that the implementation of Enum
> > could be simplified somewhat in Python 3.6 since it can trust that the
> > namespace passed into __new__ is ordered (so it doesn't have to switch
> > it to an OrderedDict in __prepare__, perhaps).
> >
> > In any case the most likely way to use __definition_order__ in general
> > was always to filter its contents through some other condition (e.g.
> > "isn't a method and doesn't start with underscore") -- you can do the
> > same with keys(). Classes that want to provide a custom list of
> > "interesting" attributes can provide that using whatever class method
> > or attribute they want -- it's just easier to keep those attributes
> > ordered because the namespace is always ordered.
>
> For example,it's already possible to expose order information via
> __dir__, consumers of the information just have to bypass the implicit
> sorting applied by the dir() builtin:
>
>   >>> class Example:
>   ... def __dir__(self):
>   ... return "first second third fourth".split()
>   ...
>   >>> dir(Example())
>   ['first', 'fourth', 'second', 'third']
>   >>> Example().__dir__()
>   ['first', 'second', 'third', 'fourth']
>
> You've persuaded me that omitting __definition_order__ is the right
> thing to do for now, so the last thing I'm going to do is to
> explicitly double check with the creators of a few interesting
> alternate implementations (MicroPython, VOC for JVM environments,
> Batavia for JavaScript environments) to see if this may cause them
> problems in officially implementing 3.6 (we know PyPy will be OK,
> since they did it first).
>
> VOC & Batavia *should* be OK (worst case, they return
> collections.OrderedDict from __prepare__ and also use it for __dict__
> attributes), but I'm less certain about MicroPython (since I don't
> know enough about how its current dict implementation works to know
> whether or not they'll be able to make the same change PyPy and
> CPython did)
>

>From the perspective of VOC and Batavia: As Nick notes, there may be some
changes needed to use OrderDict (or a native analog) in a couple of places,
but other than that, it doesn’t strike me as a change that will pose any
significant difficulty.

Yours,
Russ Magee %-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Nick Coghlan
On 11 September 2016 at 07:26, Guido van Rossum  wrote:
> On Sat, Sep 10, 2016 at 10:57 AM, Nick Coghlan  wrote:
>> On 11 September 2016 at 03:08, Guido van Rossum  wrote:
>>> So I'm happy to continue thinking about this, but I expect this is not
>>> such a big deal as you fear. Anyway, let's see if someone comes up
>>> with a more convincing argument by beta 2!
>>
>> For CPython specifically, I don't have anything more convincing than
>> Ethan's Enum example (where the way the metaclass works means most of
>> the interesting attributes don't live directly in the class dict, they
>> live in private data structures stored in the class dict, making
>> "list(MyEnum.__dict__)" inherently uninteresting, regardless of
>> whether it's ordered or not).
>
> But that would only matter if we also defined a helper utility that
> used __definition_order__. I expect that the implementation of Enum
> could be simplified somewhat in Python 3.6 since it can trust that the
> namespace passed into __new__ is ordered (so it doesn't have to switch
> it to an OrderedDict in __prepare__, perhaps).
>
> In any case the most likely way to use __definition_order__ in general
> was always to filter its contents through some other condition (e.g.
> "isn't a method and doesn't start with underscore") -- you can do the
> same with keys(). Classes that want to provide a custom list of
> "interesting" attributes can provide that using whatever class method
> or attribute they want -- it's just easier to keep those attributes
> ordered because the namespace is always ordered.

For example,it's already possible to expose order information via
__dir__, consumers of the information just have to bypass the implicit
sorting applied by the dir() builtin:

  >>> class Example:
  ... def __dir__(self):
  ... return "first second third fourth".split()
  ...
  >>> dir(Example())
  ['first', 'fourth', 'second', 'third']
  >>> Example().__dir__()
  ['first', 'second', 'third', 'fourth']

You've persuaded me that omitting __definition_order__ is the right
thing to do for now, so the last thing I'm going to do is to
explicitly double check with the creators of a few interesting
alternate implementations (MicroPython, VOC for JVM environments,
Batavia for JavaScript environments) to see if this may cause them
problems in officially implementing 3.6 (we know PyPy will be OK,
since they did it first).

VOC & Batavia *should* be OK (worst case, they return
collections.OrderedDict from __prepare__ and also use it for __dict__
attributes), but I'm less certain about MicroPython (since I don't
know enough about how its current dict implementation works to know
whether or not they'll be able to make the same change PyPy and
CPython did)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Guido van Rossum
On Sat, Sep 10, 2016 at 10:57 AM, Nick Coghlan  wrote:
> On 11 September 2016 at 03:08, Guido van Rossum  wrote:
>> So I'm happy to continue thinking about this, but I expect this is not
>> such a big deal as you fear. Anyway, let's see if someone comes up
>> with a more convincing argument by beta 2!
>
> For CPython specifically, I don't have anything more convincing than
> Ethan's Enum example (where the way the metaclass works means most of
> the interesting attributes don't live directly in the class dict, they
> live in private data structures stored in the class dict, making
> "list(MyEnum.__dict__)" inherently uninteresting, regardless of
> whether it's ordered or not).

But that would only matter if we also defined a helper utility that
used __definition_order__. I expect that the implementation of Enum
could be simplified somewhat in Python 3.6 since it can trust that the
namespace passed into __new__ is ordered (so it doesn't have to switch
it to an OrderedDict in __prepare__, perhaps).

In any case the most likely way to use __definition_order__ in general
was always to filter its contents through some other condition (e.g.
"isn't a method and doesn't start with underscore") -- you can do the
same with keys(). Classes that want to provide a custom list of
"interesting" attributes can provide that using whatever class method
or attribute they want -- it's just easier to keep those attributes
ordered because the namespace is always ordered.

> The proxy use cases I'm aware of (wrapt, weakref.proxy) tend to be
> used to wrap normal instances rather than class objects themselves, so
> they shouldn't be affected.
>
> With ordered-by-default class namespaces, both heap types and non-heap
> types should also mostly be populated in the "logical order" (i.e. the
> order names appear in the relevant C arrays), but that would formally
> be an implementation detail at this point, rather than something we
> commit to providing.
>
> The only other argument that occurs to me is one that didn't come up
> in the earlier PEP 520 discussions: how a not-quite-Python
> implementation (or a Python 3.5 compatible implementation that doesn't
> offer order-preserving behaviour the way PyPy does) can make sure that
> code that relies on ordered class namespaces *fails* in an informative
> way when run on that implementation.

Is that a real use case? It sounds like you're just constructing an
artificial example that would be less convenient without
__definition_order__.

> With __definition_order__, that's straightforward - the code that
> needs it will fail with AttributeError, and searching for the
> attribute named in the exception will lead affected users directly to
> PEP 520 and the Python 3.6 What's New guide.

But that code would have to be written to use __definition_order__. It
could just as easily be written to assert that sys.version_info() >=
(3, 6).

> With implicitly ordered class namespaces, you don't get an exception
> if the namespace isn't actually order preserving - you get attributes
> in an arbitrary order instead. Interpreters can't detect that the user
> specifically wanted order preserving behaviour, and library and
> application authors can't readily detect whether or not the runtime
> offers order preserving behaviour (since they may just get lucky on
> that particular run).

That sounds very philosophical. You still can't check whether *dict*
is order-preserving -- all you can do is checking whether a *class*
preserves its order. Since PEP 520 is accepted only for Python 3.6,
checking for the presence of __definition_order__ is no different than
checking the version.

> Even if we added a new flag to sys.implementation that indicated the
> use of order preserving class namespaces, there'd still be plenty of
> scope for subtle bugs where libraries and frameworks weren't checking
> that flag before relying on the new behaviour.

OK, I'm beginning to see the argument here. You want all code that
relies on the order to be explicitly declaring that it does so by
using a new API.

Unfortunately the mere presence of __definition_order__ doesn't really
help here -- since all dicts are order-preserving, there's still
nothing (apart from documentation) to stop apps from relying on the
ordering of the class __dict__ directly.

> Cheers,
> Nick.
>
> P.S. I'd actually love it if we could skip __definition_order__ -
> there really is a whole lot of runtime clutter on class objects, and
> we're adding __annotations__ as well. Unfortunately, I also think we
> made the right call the first time around in thinking it would still
> be necessary even if class namespaces became order preserving :)

Note that __annotations__ is only added when there are annotations, so
its presence could be used as a flag of sorts. (However you shouldn't
use it directly -- each class in the MRO has its own __annotations__,
and you should use typing.get_type_hints(cls) to coalesce all of
them.)

Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Eric Snow
On Sep 10, 2016 10:11, "Guido van Rossum"  wrote:
>
> Thanks for bringing this up. I think it's definitely possible to argue
> either way. I think what happened before was that I approved
> __definition_order__ because I wasn't expecting dict to be ordered by
> default. Now that Inada Naoki's patch has landed things have changed.
>
> Here's my reason for agreeing with (or convincing?) Eric to drop
> __definition_order__, as I remember it from the (lively) discussion at
> the sprint.
>

FWIW, my position was to leave __definition_order__ in place.  However,
once it became *mostly* redundant, I didn't consider the remaining benefits
to be sufficient justification for the extra complexity in the code to the
point that it was worth debating.  So I didn't object very strenuously when
Benjamin suggested removing it.  Regardless, I'm still in favor of keeping
__definition_order__. :)

-eric (phone)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Eric Snow
On Sep 10, 2016 11:00, "Nick Coghlan"  wrote:
>
> On 11 September 2016 at 03:08, Guido van Rossum  wrote:
> > So I'm happy to continue thinking about this, but I expect this is not
> > such a big deal as you fear. Anyway, let's see if someone comes up
> > with a more convincing argument by beta 2!
,
> Nick.
>
> P.S. I'd actually love it if we could skip __definition_order__ -
> there really is a whole lot of runtime clutter on class objects, and
> we're adding __annotations__ as well. Unfortunately, I also think we
> made the right call the first time around in thinking it would still
> be necessary even if class namespaces became order preserving :)
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com


-eric (phone)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Nick Coghlan
On 11 September 2016 at 03:08, Guido van Rossum  wrote:
> So I'm happy to continue thinking about this, but I expect this is not
> such a big deal as you fear. Anyway, let's see if someone comes up
> with a more convincing argument by beta 2!

For CPython specifically, I don't have anything more convincing than
Ethan's Enum example (where the way the metaclass works means most of
the interesting attributes don't live directly in the class dict, they
live in private data structures stored in the class dict, making
"list(MyEnum.__dict__)" inherently uninteresting, regardless of
whether it's ordered or not).

The proxy use cases I'm aware of (wrapt, weakref.proxy) tend to be
used to wrap normal instances rather than class objects themselves, so
they shouldn't be affected.

With ordered-by-default class namespaces, both heap types and non-heap
types should also mostly be populated in the "logical order" (i.e. the
order names appear in the relevant C arrays), but that would formally
be an implementation detail at this point, rather than something we
commit to providing.

The only other argument that occurs to me is one that didn't come up
in the earlier PEP 520 discussions: how a not-quite-Python
implementation (or a Python 3.5 compatible implementation that doesn't
offer order-preserving behaviour the way PyPy does) can make sure that
code that relies on ordered class namespaces *fails* in an informative
way when run on that implementation.

With __definition_order__, that's straightforward - the code that
needs it will fail with AttributeError, and searching for the
attribute named in the exception will lead affected users directly to
PEP 520 and the Python 3.6 What's New guide.

With implicitly ordered class namespaces, you don't get an exception
if the namespace isn't actually order preserving - you get attributes
in an arbitrary order instead. Interpreters can't detect that the user
specifically wanted order preserving behaviour, and library and
application authors can't readily detect whether or not the runtime
offers order preserving behaviour (since they may just get lucky on
that particular run).

Even if we added a new flag to sys.implementation that indicated the
use of order preserving class namespaces, there'd still be plenty of
scope for subtle bugs where libraries and frameworks weren't checking
that flag before relying on the new behaviour.

Cheers,
Nick.

P.S. I'd actually love it if we could skip __definition_order__ -
there really is a whole lot of runtime clutter on class objects, and
we're adding __annotations__ as well. Unfortunately, I also think we
made the right call the first time around in thinking it would still
be necessary even if class namespaces became order preserving :)

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Guido van Rossum
Thanks for bringing this up. I think it's definitely possible to argue
either way. I think what happened before was that I approved
__definition_order__ because I wasn't expecting dict to be ordered by
default. Now that Inada Naoki's patch has landed things have changed.

Here's my reason for agreeing with (or convincing?) Eric to drop
__definition_order__, as I remember it from the (lively) discussion at
the sprint.

- There are only a few use cases for definition order. Without trying
to be complete, the use cases I am aware of are all similar to the
django Forms interface
(https://docs.djangoproject.com/en/1.10/ref/forms/api/#module-django.forms)
where the order in which fields are defined determines the order in
which they are rendered. Possibly the traits or traitlets libraries
also have similar functionality; I know I added it to a database API I
wrote at Google as well. I have another idea for a use case where you
can define a named tuple with typed fields using PEP 526 syntax.

- I like sparsity of interfaces. A class already has dunder attributes
for bases, mro, docstring, name, qualified name, module, and probably
a few others that I've forgotten. Cruft inevitably accumulates, but I
still feel I have to fight it. If we can get the functionality needed
for those use cases without a new dunder attribute, so much the
better.

- The Forms trait[let]s use cases and named tuples can clearly be
dealt with by using list(cls.__dict__), since they all involve a
user-defined class.

- If we had had ordered dicts from the start, those use cases would
have been built upon that happily. What would Cython do? I don't know,
but  I imagine they'd come up with it -- they certainly ought to be
able with a way to construct the class __dict__ in the desired order.
Is there even a situation where Cython would need to support the
construction of forms, trait[let]s, or named tuples using Cython code
in a way that the order is discoverable afterwards? (I imagine that
Cython would love the named tuple idea, but they'd know the field
definition order at compile time, so why would they also need it at
runtime?)

So I'm happy to continue thinking about this, but I expect this is not
such a big deal as you fear. Anyway, let's see if someone comes up
with a more convincing argument by beta 2!

--Guido

On Sat, Sep 10, 2016 at 4:19 AM, Nick Coghlan  wrote:
> On 10 September 2016 at 19:27, Nick Coghlan  wrote:
>> On 10 September 2016 at 17:49, Ethan Furman  wrote:
>>> The "mostly" is what concerns me.  Much like having a custom __dir__ lets
>>> a class fine-tune what is of interest, a custom __definition_order__ allows
>>> a class to present a unified view of the class creation process.  This could
>>> be important to classes that employ __getattr__ (or __getattribute__) to
>>> provide virtual attributes, such as Enum or proxy classes.
>>
>> +1
>>
>> The reasoning for modifying the PEP post-acceptance is faulty -
>> __definition_order__ wasn't just there as a CPython implementation
>> detail, it was there as a way to allow class and metaclass developers
>> to hide their *own* irrelevant implementation details.
>>
>> Since __definition_order__ was already accepted, and the rationale for
>> removing it is incorrect, could we please have it back for beta 1?
>
> After posting this, I realised I should give a bit more detail on why
> I see PEP 520 without __definition_order__ as potentially problematic.
> Specifically, it relates to these two sections in the PEP about having
> __definition_order__ be writable and about whether or not to set it
> for classes that aren't created via the class syntax:
>
> * https://www.python.org/dev/peps/pep-0520/#why-not-a-read-only-attribute
> * https://www.python.org/dev/peps/pep-0520/#support-for-c-api-types
>
> From the first section: "Also, note that a writeable
> __definition_order__ allows dynamically created classes (e.g. by
> Cython) to still have __definition_order__ properly set. That could
> certainly be handled through specific class- creation tools, such as
> type() or the C-API, without the need to lose the semantics of a
> read-only attribute. However, with a writeable attribute it's a moot
> point. "
>
> From the second: "However, since __definition_order__ can be set at
> any time through normal attribute assignment, it does not need any
> special treatment in the C-API."
>
> Unlike the __definition_order__ tuple, making "list(cls.__dict__)" the
> official way of accessing the definition order exposes an
> implementation detail that's somewhat specific to the way Python class
> statements work, rather than being universal across all the different
> techniques that exist for putting together Python class objects.
>
> As Terry suggested, I've reopened and elevated the priority of
> http://bugs.python.org/issue24254, but only to deferred blocker -
> while I do think we need to reconsider the decision to remove
> 

Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Nick Coghlan
On 10 September 2016 at 19:27, Nick Coghlan  wrote:
> On 10 September 2016 at 17:49, Ethan Furman  wrote:
>> The "mostly" is what concerns me.  Much like having a custom __dir__ lets
>> a class fine-tune what is of interest, a custom __definition_order__ allows
>> a class to present a unified view of the class creation process.  This could
>> be important to classes that employ __getattr__ (or __getattribute__) to
>> provide virtual attributes, such as Enum or proxy classes.
>
> +1
>
> The reasoning for modifying the PEP post-acceptance is faulty -
> __definition_order__ wasn't just there as a CPython implementation
> detail, it was there as a way to allow class and metaclass developers
> to hide their *own* irrelevant implementation details.
>
> Since __definition_order__ was already accepted, and the rationale for
> removing it is incorrect, could we please have it back for beta 1?

After posting this, I realised I should give a bit more detail on why
I see PEP 520 without __definition_order__ as potentially problematic.
Specifically, it relates to these two sections in the PEP about having
__definition_order__ be writable and about whether or not to set it
for classes that aren't created via the class syntax:

* https://www.python.org/dev/peps/pep-0520/#why-not-a-read-only-attribute
* https://www.python.org/dev/peps/pep-0520/#support-for-c-api-types

>From the first section: "Also, note that a writeable
__definition_order__ allows dynamically created classes (e.g. by
Cython) to still have __definition_order__ properly set. That could
certainly be handled through specific class- creation tools, such as
type() or the C-API, without the need to lose the semantics of a
read-only attribute. However, with a writeable attribute it's a moot
point. "

>From the second: "However, since __definition_order__ can be set at
any time through normal attribute assignment, it does not need any
special treatment in the C-API."

Unlike the __definition_order__ tuple, making "list(cls.__dict__)" the
official way of accessing the definition order exposes an
implementation detail that's somewhat specific to the way Python class
statements work, rather than being universal across all the different
techniques that exist for putting together Python class objects.

As Terry suggested, I've reopened and elevated the priority of
http://bugs.python.org/issue24254, but only to deferred blocker -
while I do think we need to reconsider the decision to remove
__definition_order__ based on a proper update to the PEP that accounts
for all the points that came up in the original discussions, I also
don't see any major problem with leaving it out in beta 1, and then
restoring it in beta 2.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Terry Reedy

On 9/10/2016 5:27 AM, Nick Coghlan wrote:

On 10 September 2016 at 17:49, Ethan Furman  wrote:

Per Victor's advice I'm posting this here.

PEP 520 has been accepted, but without the __definition_order__ attribute.
The accompanying comment:


"Note: Since compact dict has landed in 3.6, __definition_order__ has
been removed. cls.__dict__ now mostly accomplishes the same thing
instead."



The "mostly" is what concerns me.  Much like having a custom __dir__ lets
a class fine-tune what is of interest, a custom __definition_order__ allows
a class to present a unified view of the class creation process.  This could
be important to classes that employ __getattr__ (or __getattribute__) to
provide virtual attributes, such as Enum or proxy classes.


+1

The reasoning for modifying the PEP post-acceptance is faulty -
__definition_order__ wasn't just there as a CPython implementation
detail, it was there as a way to allow class and metaclass developers
to hide their *own* irrelevant implementation details.

Since __definition_order__ was already accepted, and the rationale for
removing it is incorrect, could we please have it back for beta 1?


Someone (Ethan?) should ask that this be a release blocker on some issue.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Nick Coghlan
On 10 September 2016 at 17:49, Ethan Furman  wrote:
> Per Victor's advice I'm posting this here.
>
> PEP 520 has been accepted, but without the __definition_order__ attribute.
> The accompanying comment:
>
>> "Note: Since compact dict has landed in 3.6, __definition_order__ has
>> been removed. cls.__dict__ now mostly accomplishes the same thing
>> instead."
>
>
> The "mostly" is what concerns me.  Much like having a custom __dir__ lets
> a class fine-tune what is of interest, a custom __definition_order__ allows
> a class to present a unified view of the class creation process.  This could
> be important to classes that employ __getattr__ (or __getattribute__) to
> provide virtual attributes, such as Enum or proxy classes.

+1

The reasoning for modifying the PEP post-acceptance is faulty -
__definition_order__ wasn't just there as a CPython implementation
detail, it was there as a way to allow class and metaclass developers
to hide their *own* irrelevant implementation details.

Since __definition_order__ was already accepted, and the rationale for
removing it is incorrect, could we please have it back for beta 1?

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Ethan Furman

Per Victor's advice I'm posting this here.

PEP 520 has been accepted, but without the __definition_order__ attribute.
The accompanying comment:


"Note: Since compact dict has landed in 3.6, __definition_order__ has
been removed. cls.__dict__ now mostly accomplishes the same thing
instead."


The "mostly" is what concerns me.  Much like having a custom __dir__ lets
a class fine-tune what is of interest, a custom __definition_order__ allows
a class to present a unified view of the class creation process.  This could
be important to classes that employ __getattr__ (or __getattribute__) to
provide virtual attributes, such as Enum or proxy classes.

With __definition_order__ Enum can display the actual creation order of enum
members and methods, while relying on Enum.__dict__.keys() presents a jumbled
mess with many attributes the user never wrote, the enum members either
appearing /after/ all the methods (even if actually written before), or
entirely absent.

For example,  this class:


class PassBy(Enum):

... value = 1
... reference = 2
... name = 3
... object = name
... def used_by_python(self):
... return self.name == 'name'
...

shows this:


PassBy.__dict__.keys()

dict_keys([
   '_generate_next_value_',
'__module__',
'used_by_python',
   '__doc__',
'_member_names_',
'_member_map_',
'_member_type_',
'_value2member_map_',
'reference',
'object',
'__new__',
   ])

Notice that two of the members are missing, and all are after the method.

If __definition_order__ existed it would be this:


PassBy.__definition_order__

['value', 'reference', 'name', 'object', 'used_by_python']

Which is a much more accurate picture of the user's class.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com