Re: [Python-Dev] PEP 468 ready for pronouncement.

[Guido van Rossum]

Thanks Eric!

The synergy between this PEP and the compact dict is amazing BTW.
Clearly its time has come. Therefore:

PEP 468 is now accepted. You may as well call it Final, since all we
need to do now is update the docs. Congrats!!

--Guido

On Thu, Sep 8, 2016 at 1:20 PM, Eric Snow  wrote:
> see: https://github.com/python/peps/blob/master/pep-0468.txt
>
> With the introduction of the compact dict implementation for CPython
> 3.6, PEP 468 becomes no more than a change to the language reference.
> I've adjusted the PEP to specify use of an ordered mapping rather than
> exactly OrderedDict.  Thanks!
>
> -eric
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 468 ready for pronouncement.

[Eric Snow]

see: https://github.com/python/peps/blob/master/pep-0468.txt

With the introduction of the compact dict implementation for CPython
3.6, PEP 468 becomes no more than a change to the language reference.
I've adjusted the PEP to specify use of an ordered mapping rather than
exactly OrderedDict.  Thanks!

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Franklin? Lee]

Compact OrderedDicts can leave gaps, and once in a while compactify. For
example, whenever the entry table is full, it can decide whether to resize
(and only copy non-gaps), or just compactactify

Compact regular dicts can swap from the back and have no gaps.

I don't see the point of discussing these details. Isn't it enough to say
that these are solvable problems, which we can worry about if/when someone
actually decides to sit down and implement compact dicts?

P.S.: Sorry about the repeated emails. I think it was the iOS Gmail app.

On Jun 13, 2016 10:23 PM, "Ethan Furman"  wrote:
>
> On 06/13/2016 05:47 PM, Larry Hastings wrote:
>>
>> On 06/13/2016 05:05 PM, MRAB wrote:
>
>
>>> This could be avoided by expanding the items to include the index of
>>> the 'previous' and 'next' item, so that they could be handled like a
>>> doubly-linked list.
>>>
>>> The disadvantage would be that it would use more memory.
>>
>>
>> Another, easier technique: don't fill holes.  Same disadvantage
>> (increased memory use), but easier to write and maintain.
>
>
> I hope this is just an academic discussion: suddenly having Python's
dicts grow continuously is going to have nasty consequences somewhere.
>
> --
> ~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Ethan Furman]

On 06/13/2016 05:47 PM, Larry Hastings wrote:

On 06/13/2016 05:05 PM, MRAB wrote:



This could be avoided by expanding the items to include the index of
the 'previous' and 'next' item, so that they could be handled like a
doubly-linked list.

The disadvantage would be that it would use more memory.


Another, easier technique: don't fill holes.  Same disadvantage
(increased memory use), but easier to write and maintain.


I hope this is just an academic discussion: suddenly having Python's 
dicts grow continuously is going to have nasty consequences somewhere.


--
~Ethan~

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Nathaniel Smith]

On Jun 13, 2016 6:16 PM, "MRAB"  wrote:
>
> On 2016-06-14 01:47, Larry Hastings wrote:
>>
>> On 06/13/2016 05:05 PM, MRAB wrote:
>>>
>>> This could be avoided by expanding the items to include the index of
>>> the 'previous' and 'next' item, so that they could be handled like a
>>> doubly-linked list.
>>>
>>> The disadvantage would be that it would use more memory.
>>
>>
>> Another, easier technique: don't fill holes.  Same disadvantage
>> (increased memory use), but easier to write and maintain.
>>
> When iterating over the dict, you'd need to skip over the holes, so it
would be a good idea to compact it a some point, when there are too many
holes.

Right -- but if you wait for some ratio of holes to filled space before
compacting, you can amortize the cost down, and have a good big-O
complexity for both del and iteration simultaneously. Same basic principle
as using proportional overallocation when appending to a list, just in
reverse.

I believe this is what pypy's implementation actually does.

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[MRAB]

On 2016-06-14 01:47, Larry Hastings wrote:

On 06/13/2016 05:05 PM, MRAB wrote:

This could be avoided by expanding the items to include the index of
the 'previous' and 'next' item, so that they could be handled like a
doubly-linked list.

The disadvantage would be that it would use more memory.


Another, easier technique: don't fill holes.  Same disadvantage
(increased memory use), but easier to write and maintain.

When iterating over the dict, you'd need to skip over the holes, so it 
would be a good idea to compact it a some point, when there are too many 
holes.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Larry Hastings]

On 06/13/2016 05:05 PM, MRAB wrote:
This could be avoided by expanding the items to include the index of 
the 'previous' and 'next' item, so that they could be handled like a 
doubly-linked list.


The disadvantage would be that it would use more memory.


Another, easier technique: don't fill holes.  Same disadvantage 
(increased memory use), but easier to write and maintain.



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[MRAB]

On 2016-06-13 17:34, Ethan Furman wrote:

On 06/10/2016 02:13 PM, Franklin? Lee wrote:


P.S.: If anyone is missing the relevance, Raymond Hettinger's compact
dicts are inherently ordered until a delitem happens.[1] That could be
"good enough" for many purposes, including kwargs and class definition.


It would be great for kwargs, but not for class definition: del's can
happen there, so we need PEP 520 with OrderedDict so the definition
order is not lost when an item is deleted during class creation.

The order can be lost when an item is deleted because it moves the last 
item into the 'hole' left by the deleted item.


This could be avoided by expanding the items to include the index of the 
'previous' and 'next' item, so that they could be handled like a 
doubly-linked list.


The disadvantage would be that it would use more memory.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Tim Golden]

I've set him to moderation for now. Beyond that we'd have to unsubscribe 
him altogether and ask him to resubscribe later.


TJG

On 13/06/2016 22:34, Guido van Rossum wrote:

Can someone block Franklin until his mailer stops resending this message?

--Guido (mobile)

On Jun 13, 2016 2:26 PM, "Franklin? Lee" > wrote:

I am. I was just wondering if there was an in-progress effort I
should be looking at, because I am interested in extensions to it.

P.S.: If anyone is missing the relevance, Raymond
Hettinger's compact dicts are inherently ordered until a
delitem happens.[1] That could be "good enough" for many purposes,
including kwargs and class definition. If CPython implements
efficient compact dicts, it would be easier to propose
order-preserving (or initially-order-preserving) dicts in some
places in the standard.

[1] Whether delitem preserves order depends on whether you want to
allow gaps in your compact entry table. PyPy implemented compact
dicts and chose(?) to make dicts ordered.

On Saturday, June 11, 2016, Eric Snow > wrote:

On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
 wrote:
> Eric, have you any work in progress on compact dicts?

Nope.  I presume you are talking the proposal Raymond made a
while back.

-eric


___
Python-Dev mailing list
Python-Dev@python.org 
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/mail%40timgolden.me.uk



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Guido van Rossum]

Can someone block Franklin until his mailer stops resending this message?

--Guido (mobile)
On Jun 13, 2016 2:26 PM, "Franklin? Lee" 
wrote:

> I am. I was just wondering if there was an in-progress effort I should be
> looking at, because I am interested in extensions to it.
>
> P.S.: If anyone is missing the relevance, Raymond Hettinger's compact
> dicts are inherently ordered until a delitem happens.[1] That could be
> "good enough" for many purposes, including kwargs and class definition. If
> CPython implements efficient compact dicts, it would be easier to propose
> order-preserving (or initially-order-preserving) dicts in some places in
> the standard.
>
> [1] Whether delitem preserves order depends on whether you want to allow
> gaps in your compact entry table. PyPy implemented compact dicts and
> chose(?) to make dicts ordered.
>
> On Saturday, June 11, 2016, Eric Snow  wrote:
>
>> On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
>>  wrote:
>> > Eric, have you any work in progress on compact dicts?
>>
>> Nope.  I presume you are talking the proposal Raymond made a while back.
>>
>> -eric
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Franklin? Lee]

I am. I was just wondering if there was an in-progress effort I should be
looking at, because I am interested in extensions to it.

P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts
are inherently ordered until a delitem happens.[1] That could be "good
enough" for many purposes, including kwargs and class definition. If
CPython implements efficient compact dicts, it would be easier to propose
order-preserving (or initially-order-preserving) dicts in some places in
the standard.

[1] Whether delitem preserves order depends on whether you want to allow
gaps in your compact entry table. PyPy implemented compact dicts and
chose(?) to make dicts ordered.

On Saturday, June 11, 2016, Eric Snow  wrote:

> On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
> > wrote:
> > Eric, have you any work in progress on compact dicts?
>
> Nope.  I presume you are talking the proposal Raymond made a while back.
>
> -eric
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Franklin? Lee]

I am. I was just wondering if there was an in-progress effort I should be
looking at, because I am interested in extensions to it.

P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts
are inherently ordered until a delitem happens.[1] That could be "good
enough" for many purposes, including kwargs and class definition. If
CPython implements efficient compact dicts, it would be easier to propose
order-preserving (or initially-order-preserving) dicts in some places in
the standard.

[1] Whether delitem preserves order depends on whether you want to allow
gaps in your compact entry table. PyPy implemented compact dicts and
chose(?) to make dicts ordered.

On Saturday, June 11, 2016, Eric Snow  wrote:

> On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
> > wrote:
> > Eric, have you any work in progress on compact dicts?
>
> Nope.  I presume you are talking the proposal Raymond made a while back.
>
> -eric
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Franklin? Lee]

I am. I was just wondering if there was an in-progress effort I should be
looking at, because I am interested in extensions to it.

P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts
are inherently ordered until a delitem happens.[1] That could be "good
enough" for many purposes, including kwargs and class definition. If
CPython implements efficient compact dicts, it would be easier to propose
order-preserving (or initially-order-preserving) dicts in some places in
the standard.

[1] Whether delitem preserves order depends on whether you want to allow
gaps in your compact entry table. PyPy implemented compact dicts and
chose(?) to make dicts ordered.

On Saturday, June 11, 2016, Eric Snow  wrote:

> On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
> > wrote:
> > Eric, have you any work in progress on compact dicts?
>
> Nope.  I presume you are talking the proposal Raymond made a while back.
>
> -eric
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Ethan Furman]

On 06/10/2016 02:13 PM, Franklin? Lee wrote:


P.S.: If anyone is missing the relevance, Raymond Hettinger's compact
dicts are inherently ordered until a delitem happens.[1] That could be
"good enough" for many purposes, including kwargs and class definition.


It would be great for kwargs, but not for class definition: del's can 
happen there, so we need PEP 520 with OrderedDict so the definition 
order is not lost when an item is deleted during class creation.


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Franklin? Lee]

I am. I was just wondering if there was an in-progress effort I should be
looking at, because I am interested in extensions to it.

P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts
are inherently ordered until a delitem happens.[1] That could be "good
enough" for many purposes, including kwargs and class definition. If
CPython implements efficient compact dicts, it would be easier to propose
order-preserving (or initially-order-preserving) dicts in some places in
the standard.

[1] Whether delitem preserves order depends on whether you want to allow
gaps in your compact entry table. PyPy implemented compact dicts and
chose(?) to make dicts ordered.

On Saturday, June 11, 2016, Eric Snow  wrote:

> On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
> > wrote:
> > Eric, have you any work in progress on compact dicts?
>
> Nope.  I presume you are talking the proposal Raymond made a while back.
>
> -eric
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Franklin? Lee]

I am. I was just wondering if there was an in-progress effort I should be
looking at, because I am interested in extensions to it.

P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts
are inherently ordered until a delitem happens.[1] That could be "good
enough" for many purposes, including kwargs and class definition. If
CPython implements efficient compact dicts, it would be easier to propose
order-preserving (or initially-order-preserving) dicts in some places in
the standard.

[1] Whether delitem preserves order depends on whether you want to allow
gaps in your compact entry table. PyPy implemented compact dicts and
chose(?) to make dicts ordered.

On Saturday, June 11, 2016, Eric Snow  wrote:

> On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
> > wrote:
> > Eric, have you any work in progress on compact dicts?
>
> Nope.  I presume you are talking the proposal Raymond made a while back.
>
> -eric
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Franklin? Lee]

I am. I was just wondering if there was an in-progress effort I should be
looking at, because I am interested in extensions to it.

P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts
are inherently ordered until a delitem happens.[1] That could be "good
enough" for many purposes, including kwargs and class definition. If
CPython implements efficient compact dicts, it would be easier to propose
order-preserving (or initially-order-preserving) dicts in some places in
the standard.

[1] Whether delitem preserves order depends on whether you want to allow
gaps in your compact entry table. PyPy implemented compact dicts and
chose(?) to make dicts ordered.

On Saturday, June 11, 2016, Eric Snow  wrote:

> On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
> > wrote:
> > Eric, have you any work in progress on compact dicts?
>
> Nope.  I presume you are talking the proposal Raymond made a while back.
>
> -eric
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Eric Snow]

On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee
 wrote:
> Eric, have you any work in progress on compact dicts?

Nope.  I presume you are talking the proposal Raymond made a while back.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Franklin? Lee]

Eric, have you any work in progress on compact dicts?

On Fri, Jun 10, 2016 at 12:54 PM, Eric Snow  wrote:
> On Thu, Jun 9, 2016 at 1:10 PM, Émanuel Barry  wrote:
>> As stated by Guido (and pointed out in the PEP):
>>
>> Making **kwds ordered is still open, but requires careful design and
>> implementation to avoid slowing down function calls that don't benefit.
>>
>> The PEP has not been updated in a while, though. Python 3.5 has been
>> released, and with it a C implementation of OrderedDict.
>>
>> Eric, are you still interested in this?
>
> Yes, but wasn't planning on dusting it off yet (i.e. in time for 3.6).
> I'm certainly not opposed to someone picking up the banner.
> 
>
>> IIRC that PEP was one of the
>> motivating use cases for implementing OrderedDict in C.
>
> Correct, though I'm not sure OrderedDict needs to be involved any more.
>
>> Maybe it's time for
>> a second round of discussion on Python-ideas?
>
> Fine with me, though I won't have a lot of time in the 3.6 timeframe
> to handle a high-volume discussion or push through an implementation.
>
> -eric
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/leewangzhong%2Bpython%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[zreed]

I would be super excited for this feature, so if there's a reasonable
chance of it being picked up I don't mind doing the implementation work.


On Fri, Jun 10, 2016, at 11:54 AM, Eric Snow wrote:
> On Thu, Jun 9, 2016 at 1:10 PM, Émanuel Barry  wrote:
> > As stated by Guido (and pointed out in the PEP):
> >
> > Making **kwds ordered is still open, but requires careful design and
> > implementation to avoid slowing down function calls that don't benefit.
> >
> > The PEP has not been updated in a while, though. Python 3.5 has been
> > released, and with it a C implementation of OrderedDict.
> >
> > Eric, are you still interested in this?
> 
> Yes, but wasn't planning on dusting it off yet (i.e. in time for 3.6).
> I'm certainly not opposed to someone picking up the banner.
> 
> 
> > IIRC that PEP was one of the
> > motivating use cases for implementing OrderedDict in C.
> 
> Correct, though I'm not sure OrderedDict needs to be involved any more.
> 
> > Maybe it's time for
> > a second round of discussion on Python-ideas?
> 
> Fine with me, though I won't have a lot of time in the 3.6 timeframe
> to handle a high-volume discussion or push through an implementation.
> 
> -eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Eric Snow]

On Thu, Jun 9, 2016 at 1:10 PM, Émanuel Barry  wrote:
> As stated by Guido (and pointed out in the PEP):
>
> Making **kwds ordered is still open, but requires careful design and
> implementation to avoid slowing down function calls that don't benefit.
>
> The PEP has not been updated in a while, though. Python 3.5 has been
> released, and with it a C implementation of OrderedDict.
>
> Eric, are you still interested in this?

Yes, but wasn't planning on dusting it off yet (i.e. in time for 3.6).
I'm certainly not opposed to someone picking up the banner.


> IIRC that PEP was one of the
> motivating use cases for implementing OrderedDict in C.

Correct, though I'm not sure OrderedDict needs to be involved any more.

> Maybe it's time for
> a second round of discussion on Python-ideas?

Fine with me, though I won't have a lot of time in the 3.6 timeframe
to handle a high-volume discussion or push through an implementation.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Eric Snow]

On Thu, Jun 9, 2016 at 12:41 PM,   wrote:
> Is there any further thoughts on including this in 3.6?

I don't have any plans and I don't know of anyone willing to champion
the PEP for 3.6.  Note that the implementation itself shouldn't take
very long.

>  Similar to the
> recent discussion on OrderedDict namespaces for metaclasses, this would
> simplify / enable a number of type factory use cases where proper
> metaclasses are overkill. This feature would also be quite nice in say
> pandas where the (currently unspecified) field order used in the
> definition of frames is preserved in user-visible displays.

Good point.  One weakness of the PEP has been sufficient
justification.  The greater number of compelling use cases, the
better.  So thanks! :)

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468

[Émanuel Barry]

> From: zr...@fastmail.com
> Subject: [Python-Dev] PEP 468
> 
> Is there any further thoughts on including this in 3.6?  Similar to the
> recent discussion on OrderedDict namespaces for metaclasses, this would
> simplify / enable a number of type factory use cases where proper
> metaclasses are overkill. This feature would also be quite nice in say
> pandas where the (currently unspecified) field order used in the
> definition of frames is preserved in user-visible displays.

As stated by Guido (and pointed out in the PEP):

Making **kwds ordered is still open, but requires careful design and
implementation to avoid slowing down function calls that don't benefit.

The PEP has not been updated in a while, though. Python 3.5 has been
released, and with it a C implementation of OrderedDict.

Eric, are you still interested in this? IIRC that PEP was one of the
motivating use cases for implementing OrderedDict in C. Maybe it's time for
a second round of discussion on Python-ideas?

-Emanuel
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 468

[zreed]

Is there any further thoughts on including this in 3.6?  Similar to the
recent discussion on OrderedDict namespaces for metaclasses, this would
simplify / enable a number of type factory use cases where proper
metaclasses are overkill. This feature would also be quite nice in say
pandas where the (currently unspecified) field order used in the
definition of frames is preserved in user-visible displays.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468 (Ordered kwargs)

[Chris Withers]

On 28/01/2015 07:14, Gregory P. Smith wrote:


It is a potentially bad idea if order is the default behavior of
iteration, items(), keys() and values(). Ideally order should only be
exposed when explicitly asked for to help prevent bugs and mitigate
potential information leaks.


I have to be honest, I think that's the opposite of most new users 
assumption...



Experience cleaning up our huge code base at work to turn on hash
randomization by default a couple years ago has shown that people depend
on iteration order in code often without intending to. This often leads
to latent bugs. Keep iteration order unstable by default and you prevent
people from doing that.


Hmm, well, or you could say that always having ordering would mean the 
behaviour would match new users experimental understanding and so 
eliminate all bugs that occur when people accidentally rely on ordering.


Personally, I'd prefer to see us be explicit about data structures used 
when security matters, an explicit RandomOrderedDict would make that 
clear.


cheers,

Chris
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468 (Ordered kwargs)

[Armin Rigo]

Hi all,

On 24 January 2015 at 11:50, Maciej Fijalkowski fij...@gmail.com wrote:
 I would like to point out that we implemented rhettingers idea in PyPy
 that makes all the dicts ordered by default and we don't have any
 adverse performance effects (in fact, there is quite significant
 memory saving coming from it). The measurments on CPython could be
 different, but in principle OrderedDict can be implemented as
 efficiently as normal dict.

I would like to add that http://bugs.python.org/issue16991 is the same
as today's dicts with an additional doubly-linked list for the order.
I'm unsure why you would do that after the 2012 thread started by
Raymond Hettinger, but anyway, don't conclude from only this that in
the CPython case ordered dictionaries would be slower and bigger.  My
guess is that, with a simple port of what is now in PyPy, they would
not be (but of course no-one can be sure at this point).  Let's say,
if you could imagine that CPython's dictionaries, tomorrow, are always
magically fully ordered, then would it still be a bad idea?

If such a discussion would resurface (soon or one day), and if other
related issues are resolved (like what to do in Jython and
IronPython), and if the conclusion would tentatively turn out
positive... then, provided there would at that point still be no
Raymond-style implementation of dicts, I would volunteer to port
PyPy's one to CPython[1].  As you may have guessed I don't consider
this particularly likely to occur, but it is a standing offer
nevertheless :-)


A bientôt,

Armin.


[1]  Someone could also do such a port for the goal of getting an
alternate `odictobject.c`.  He would be welcome to #pypy to get some
help from the PyPy guys, including me --- but my offer above doesn't
apply in this case.  I want to remove a thorn in the foot of
python-dev discussing about the language; I'm not really interested in
contributing to the `collections.OrderedDict` type.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468 (Ordered kwargs)

[Gregory P. Smith]

On Tue Jan 27 2015 at 2:13:08 PM Armin Rigo ar...@tunes.org wrote:

 Hi all,

 On 24 January 2015 at 11:50, Maciej Fijalkowski fij...@gmail.com wrote:
  I would like to point out that we implemented rhettingers idea in PyPy
  that makes all the dicts ordered by default and we don't have any
  adverse performance effects (in fact, there is quite significant
  memory saving coming from it). The measurments on CPython could be
  different, but in principle OrderedDict can be implemented as
  efficiently as normal dict.

 I would like to add that http://bugs.python.org/issue16991 is the same
 as today's dicts with an additional doubly-linked list for the order.
 I'm unsure why you would do that after the 2012 thread started by
 Raymond Hettinger, but anyway, don't conclude from only this that in
 the CPython case ordered dictionaries would be slower and bigger.  My
 guess is that, with a simple port of what is now in PyPy, they would
 not be (but of course no-one can be sure at this point).  Let's say,
 if you could imagine that CPython's dictionaries, tomorrow, are always
 magically fully ordered, then would it still be a bad idea?


It is a potentially bad idea if order is the default behavior of iteration,
items(), keys() and values(). Ideally order should only be exposed when
explicitly asked for to help prevent bugs and mitigate potential
information leaks.

But I'm not sure how big of a deal this actually is. The insertion order
nicely doesn't give away anything related to the hash seed used for hash
randomization which is a nice bonus over today's implementation (and 2.7 
3.3's very poor hash randomization implementation).

Experience cleaning up our huge code base at work to turn on hash
randomization by default a couple years ago has shown that people depend on
iteration order in code often without intending to. This often leads to
latent bugs. Keep iteration order unstable by default and you prevent
people from doing that. Make people request an ordered or stable iteration
when their code explicitly needs it.

If such a discussion would resurface (soon or one day), and if other
 related issues are resolved (like what to do in Jython and
 IronPython), and if the conclusion would tentatively turn out
 positive... then, provided there would at that point still be no
 Raymond-style implementation of dicts, I would volunteer to port
 PyPy's one to CPython[1].  As you may have guessed I don't consider
 this particularly likely to occur, but it is a standing offer
 nevertheless :-)


CPython should benefit from it regardless for the memory savings alone.

-gps



 A bientôt,

 Armin.


 [1]  Someone could also do such a port for the goal of getting an
 alternate `odictobject.c`.  He would be welcome to #pypy to get some
 help from the PyPy guys, including me --- but my offer above doesn't
 apply in this case.  I want to remove a thorn in the foot of
 python-dev discussing about the language; I'm not really interested in
 contributing to the `collections.OrderedDict` type.
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe: https://mail.python.org/mailman/options/python-dev/
 greg%40krypto.org

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468 (Ordered kwargs)

[Eric Snow]

On Sat, Jan 24, 2015 at 3:50 AM, Maciej Fijalkowski fij...@gmail.com wrote:
 I would like to point out that we implemented rhettingers idea in PyPy
 that makes all the dicts ordered by default and we don't have any
 adverse performance effects (in fact, there is quite significant
 memory saving coming from it). The measurments on CPython could be
 different, but in principle OrderedDict can be implemented as
 efficiently as normal dict.

 Writeup: 
 http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html

 Previous discussion:
 https://mail.python.org/pipermail/python-dev/2012-December/123028.html

Cool.  I'll add a note to the PEP.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468 (Ordered kwargs)

[Maciej Fijalkowski]

Hi Guido.

I *think* part of the reason why our implementation works is that
machines are significantly different than at the times of Knuth.
Avoiding cache misses is a very effective way to improve performance
these days.

Cheers,
fijal

On Sat, Jan 24, 2015 at 7:39 PM, Guido van Rossum gu...@python.org wrote:
 Wow, very cool. When I implemented the very first Python dict (cribbing from
 an algorithm in Knuth) I had no idea that 25 years later there would still
 be ways to improve upon it! I've got a feeling Knuth probably didn't expect
 this either...

 On Sat, Jan 24, 2015 at 2:51 AM, Maciej Fijalkowski fij...@gmail.com
 wrote:

 On Sat, Jan 24, 2015 at 12:50 PM, Maciej Fijalkowski fij...@gmail.com
 wrote:
  Hi
 
  I would like to point out that we implemented rhettingers idea in PyPy
  that makes all the dicts ordered by default and we don't have any
  adverse performance effects (in fact, there is quite significant
  memory saving coming from it). The measurments on CPython could be
  different, but in principle OrderedDict can be implemented as
  efficiently as normal dict.
 
  Writeup:
  http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
 
  Previous discussion:
  https://mail.python.org/pipermail/python-dev/2012-December/123028.html
 
  Cheers,
  fijal

 also as a sidenote: PEP should maybe mention that PyPy is already
 supporting it, a bit by chance
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/guido%40python.org




 --
 --Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468 (Ordered kwargs)

[Guido van Rossum]

Wow, very cool. When I implemented the very first Python dict (cribbing
from an algorithm in Knuth) I had no idea that 25 years later there would
still be ways to improve upon it! I've got a feeling Knuth probably didn't
expect this either...

On Sat, Jan 24, 2015 at 2:51 AM, Maciej Fijalkowski fij...@gmail.com
wrote:

 On Sat, Jan 24, 2015 at 12:50 PM, Maciej Fijalkowski fij...@gmail.com
 wrote:
  Hi
 
  I would like to point out that we implemented rhettingers idea in PyPy
  that makes all the dicts ordered by default and we don't have any
  adverse performance effects (in fact, there is quite significant
  memory saving coming from it). The measurments on CPython could be
  different, but in principle OrderedDict can be implemented as
  efficiently as normal dict.
 
  Writeup:
 http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
 
  Previous discussion:
  https://mail.python.org/pipermail/python-dev/2012-December/123028.html
 
  Cheers,
  fijal

 also as a sidenote: PEP should maybe mention that PyPy is already
 supporting it, a bit by chance
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/guido%40python.org




-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468 (Ordered kwargs)

[Guido van Rossum]

On Sat, Jan 24, 2015 at 11:11 AM, Maciej Fijalkowski fij...@gmail.com
wrote:

 Hi Guido.

 I *think* part of the reason why our implementation works is that
 machines are significantly different than at the times of Knuth.
 Avoiding cache misses is a very effective way to improve performance
 these days.


Right. We might look what Knuth has to say about data structures that are
stored on disk and loaded into memory piecemeal. :-)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 468 (Ordered kwargs)

[Maciej Fijalkowski]

On Sat, Jan 24, 2015 at 12:50 PM, Maciej Fijalkowski fij...@gmail.com wrote:
 Hi

 I would like to point out that we implemented rhettingers idea in PyPy
 that makes all the dicts ordered by default and we don't have any
 adverse performance effects (in fact, there is quite significant
 memory saving coming from it). The measurments on CPython could be
 different, but in principle OrderedDict can be implemented as
 efficiently as normal dict.

 Writeup: 
 http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html

 Previous discussion:
 https://mail.python.org/pipermail/python-dev/2012-December/123028.html

 Cheers,
 fijal

also as a sidenote: PEP should maybe mention that PyPy is already
supporting it, a bit by chance
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 468 (Ordered kwargs)

[Maciej Fijalkowski]

Hi

I would like to point out that we implemented rhettingers idea in PyPy
that makes all the dicts ordered by default and we don't have any
adverse performance effects (in fact, there is quite significant
memory saving coming from it). The measurments on CPython could be
different, but in principle OrderedDict can be implemented as
efficiently as normal dict.

Writeup: 
http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html

Previous discussion:
https://mail.python.org/pipermail/python-dev/2012-December/123028.html

Cheers,
fijal
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com