Re: [Python-Dev] The current dict is not an "OrderedDict"

[Eric Snow]

On Nov 7, 2017 08:12, "INADA Naoki"  wrote:

Additionally, class namespace should keep insertion order.  It's language
spec from 3.6.  So we should have two mode for such optimization.
It makes dict more complicated.


FWIW, PEP 520 (Preserving Class Attribute Definition Order) originally
specified leaving the class namespace alone.  Instead, the default class
*definition* namespace was changed to OrderedDict, and the ordering from
that namespace was stored as a tuple of names in a new __definition_order__
attribute on classes.  That approach effectively decoupled the final class
namespace from the proposed feature.

If it's an issue now then we might consider reviving __definition_order__
(which, as a bonus, has other minor benefits).  However, I expect we will
make the current dict implementation's behavior official, which renders any
changes unnecessary.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Armin Rigo]

Hi Antoine,

On 8 November 2017 at 10:28, Antoine Pitrou  wrote:
> Yet, PyPy has no reference counting, and it doesn't seem to be a cause
> of concern.  Broken code is fixed along the way, when people notice.

It is a major cause of concern.  This is the main blocker for
pure-Python code compatibility between CPython and all other
implementations of Python, but most of the Python community plays nice
and says since long ago "don't do that".   As a result,
nowadays, people generally know better than rely on deterministic
__del__, and the language evolution usually cares not to add new
dependencies on that.  The problem is mostly confined to some
pre-existing large code bases (like OpenStack), where no good solution
exists.

It's roughly OK to have one big blocker that people need to know
about.  I don't think it's anywhere close to OK to have a miriad of
small ones.  PyPy has worked very hard to get where it is now, and
continues to regularly "fix" very obscure compatibility issues where
CPython's behaviour differs from its own documentation---by copying
the CPython actual behaviour, of course.


A bientôt,

Armin.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Nathaniel Smith]

On Thu, Nov 9, 2017 at 1:46 PM, Cameron Simpson  wrote:
> On 08Nov2017 10:28, Antoine Pitrou  wrote:
>>
>> On Wed, 8 Nov 2017 13:07:12 +1000
>> Nick Coghlan  wrote:
>>>
>>> On 8 November 2017 at 07:19, Evpok Padding 
>>> wrote:
>>> > On 7 November 2017 at 21:47, Chris Barker 
>>> > wrote:
>>> >> if dict order is preserved in cPython , people WILL count on it!
>>> >
>>> > I won't, and if people do and their code break, they'll have only
>>> > themselves
>>> > to blame.
>>> > Also, what proof do you have of that besides anecdotal evidence ?
>>>
>>> ~27 calendar years of anecdotal evidence across a multitude of CPython
>>> API behaviours (as well as API usage in other projects).
>>>
>>> Other implementation developers don't say "CPython's runtime behaviour
>>> is the real Python specification" for the fun of it - they say it
>>> because "my code works on CPython, but it does the wrong thing on your
>>> interpreter, so I'm going to stick with CPython" is a real barrier to
>>> end user adoption, no matter what the language specification says.
>>
>>
>> Yet, PyPy has no reference counting, and it doesn't seem to be a cause
>> of concern.  Broken code is fixed along the way, when people notice.
>
>
> I'd expect that this may be because that would merely to cause temporary
> memory leakage or differently timed running of __del__ actions.  Neither of
> which normally affects semantics critical to the end result of most
> programs.

It's actually a major problem when porting apps to PyPy. The common
case is servers that crash because they rely on the GC to close file
descriptors, and then run out of file descriptors. IIRC this is the
major obstacle to supporting OpenStack-on-PyPy. NumPy is currently
going through the process to deprecate and replace a core bit of API
[1] because it turns out to assume a refcounting GC.

-n

[1] See:
  https://github.com/numpy/numpy/pull/9639
  https://mail.python.org/pipermail/numpy-discussion/2017-November/077367.html

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Cameron Simpson]

On 08Nov2017 10:28, Antoine Pitrou  wrote:

On Wed, 8 Nov 2017 13:07:12 +1000
Nick Coghlan  wrote:

On 8 November 2017 at 07:19, Evpok Padding  wrote:
> On 7 November 2017 at 21:47, Chris Barker  wrote:
>> if dict order is preserved in cPython , people WILL count on it!
>
> I won't, and if people do and their code break, they'll have only themselves
> to blame.
> Also, what proof do you have of that besides anecdotal evidence ?

~27 calendar years of anecdotal evidence across a multitude of CPython
API behaviours (as well as API usage in other projects).

Other implementation developers don't say "CPython's runtime behaviour
is the real Python specification" for the fun of it - they say it
because "my code works on CPython, but it does the wrong thing on your
interpreter, so I'm going to stick with CPython" is a real barrier to
end user adoption, no matter what the language specification says.


Yet, PyPy has no reference counting, and it doesn't seem to be a cause
of concern.  Broken code is fixed along the way, when people notice.


I'd expect that this may be because that would merely to cause temporary memory 
leakage or differently timed running of __del__ actions.  Neither of which 
normally affects semantics critical to the end result of most programs.


However, code which relies on an ordering effect which works in the usual case 
but (often subtly) breaks in some unusual case can be hard to debug, because 
(a) recognising the salient error situation may be hard to do and (b) reasoning 
about the failure is difficult when the language semantics are not what you 
thought they were.


I think the two situations are not as parallel as you think.

Cheers,
Cameron Simpson  (formerly c...@zip.com.au)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Antoine Pitrou]

On Wed, 8 Nov 2017 13:07:12 +1000
Nick Coghlan  wrote:
> On 8 November 2017 at 07:19, Evpok Padding  wrote:
> > On 7 November 2017 at 21:47, Chris Barker  wrote:  
> >> if dict order is preserved in cPython , people WILL count on it!  
> >
> > I won't, and if people do and their code break, they'll have only themselves
> > to blame.
> > Also, what proof do you have of that besides anecdotal evidence ?  
> 
> ~27 calendar years of anecdotal evidence across a multitude of CPython
> API behaviours (as well as API usage in other projects).
> 
> Other implementation developers don't say "CPython's runtime behaviour
> is the real Python specification" for the fun of it - they say it
> because "my code works on CPython, but it does the wrong thing on your
> interpreter, so I'm going to stick with CPython" is a real barrier to
> end user adoption, no matter what the language specification says.

Yet, PyPy has no reference counting, and it doesn't seem to be a cause
of concern.  Broken code is fixed along the way, when people notice.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Guido van Rossum]

I'll probably get complaints because I'm not waiting for the benchmark
results to come in, but I think I've seen enough.

To me the only defensible behavior *other than the pre-3.6 behavior* is
that after deletions the order remains preserved and new insertions happen
at the end -- i.e. the same where they would go if the deleted items were
never inserted.

I find it hard to believe that there would be a speed difference that's
noticeable outside micro-benchmarks or applications making extreme use of
dicts.

PS. It seems odd that people arguing that the behavior after deletions
doesn't matter are also arguing that deletions are uncommon? Surely there
should be no speed penalty if you never delete anything from a dict, so if
you believe deletions are rare anyway, why would you care about paying a
bit extra for them?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Steven D'Aprano]

On Tue, Nov 07, 2017 at 05:37:15PM +0200, Serhiy Storchaka wrote:
> 07.11.17 16:56, Steven D'Aprano пише:
> >To clarify: if we start with an empty dict, add keys A...D, delete B,
> >then add E...H, we could expect:
[...]

> Rather
> 
> {A: 1, D: 4, C: 3}  # move the last item in place of removed
> {A: 1, D: 4, C: 3, E: 5}

Thanks for the correction.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Random832]

On Tue, Nov 7, 2017, at 09:56, Steven D'Aprano wrote:
> Don't let the perfect be the enemy of the good.
> 
> For many applications, keys are never removed from the dict, so this 
> doesn't matter. If you never delete a key, then the remaining keys will 
> never be reordered.
> 
> I think that Nick's intent was not to say that after a single deletion, 
> the ordering guarantee goes away "forever", but that a deletion may be 
> permitted to reorder the keys, after which further additions will honour 
> insertion order. At least, that's how I interpret him.

Honestly, I think the more intuitive way would be the "other way around"
- deletions don't themselves change the order, but they do cause
subsequent insertions to be allowed to insert at places other than the
end.

Does the implementation in CPython have this property?

One way I have seen this done is that the items themselves live in an
array, and each new item is always inserted in the first empty slot in
the array (empty slots form a freelist). The hash buckets contain
indices into the array.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Nick Coghlan]

On 8 November 2017 at 07:19, Evpok Padding  wrote:
> On 7 November 2017 at 21:47, Chris Barker  wrote:
>> if dict order is preserved in cPython , people WILL count on it!
>
> I won't, and if people do and their code break, they'll have only themselves
> to blame.
> Also, what proof do you have of that besides anecdotal evidence ?

~27 calendar years of anecdotal evidence across a multitude of CPython
API behaviours (as well as API usage in other projects).

Other implementation developers don't say "CPython's runtime behaviour
is the real Python specification" for the fun of it - they say it
because "my code works on CPython, but it does the wrong thing on your
interpreter, so I'm going to stick with CPython" is a real barrier to
end user adoption, no matter what the language specification says.
Blaming users for not writing portable code doesn't achieve anything
in that scenario - it just puts an extra road block in the way of
those users trying out the alternative interpreter implementation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Evpok Padding]

On 7 November 2017 at 21:47, Chris Barker  wrote:

> On Tue, Nov 7, 2017 at 11:50 AM, Tim Peters  wrote:
>
>> Is it worth guaranteeing that will always "work" (as intended)?  Not
>> to me, but I do have code that relies on it now -
>
>
> This is critically important -- no one looks at the language spec to
> figure out how something works -- they try it, and if it works assume it
> will continue to work.
>
> if dict order is preserved in cPython , people WILL count on it!
>

I won't, and if people do and their code break, they'll have only
themselves to blame.
Also, what proof do you have of that besides anecdotal evidence ?

E
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Chris Barker]

On Tue, Nov 7, 2017 at 11:50 AM, Tim Peters  wrote:

> Is it worth guaranteeing that will always "work" (as intended)?  Not
> to me, but I do have code that relies on it now -


This is critically important -- no one looks at the language spec to figure
out how something works -- they try it, and if it works assume it will
continue to work.

if dict order is preserved in cPython , people WILL count on it!

And similarly, having order preserved only until a delete is going to cause
bugs in people's code that are less than careful :-( -- and that's most of
us.

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Tim Peters]

[Peter Ludemann]
> Does it matter whether the dict order after pop/delete is explicitly
> specified, or just specified that it's deterministic?

Any behavior whatsoever becomes essential after it becomes known ;-)

For example, dicts as currently ordered easily support LRU (least
recently used) purging like so:

On access:

result = d.pop(key)
d[key] = result

This moves `key` from wherever it was to the _most_ recently used position.

To purge the `numtopurge` least recently used keys (since traversing
the dict is always from least-recently to most-recently added):

topurge = tuple(itertools.islice(d, numtopurge))
for key in topurge:
del d[key]

Is it worth guaranteeing that will always "work" (as intended)?  Not
to me, but I do have code that relies on it now - and we can count on
someone else saying it's utterly crucial ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Barry Warsaw]

Antoine Pitrou wrote:

> Well... It really depends what kind of problem you're solving.  I
> certainly delete or pop items from dicts quite often.
> 
> Let's not claim that deleting items from a dict is a rare or advanced
> feature.  It is not.

+1.  It's a pretty common pattern for handling optional keyword
arguments, e.g. in subclass methods.

class Foo(Bar):
def foo(self, *args **kws):
mine = kws.pop('mine', None)
super().foo(self, *args, **kws)
do_something_myself(mine)

Now the question is, what guarantees does the language make about the
ordering of kws that Foo.foo() is passing to Bar.foo()?

-Barry

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Peter Ludemann via Python-Dev]

Does it matter whether the dict order after pop/delete is explicitly
specified, or just specified that it's deterministic?

On 7 November 2017 at 11:28, Antoine Pitrou  wrote:

> On Wed, 8 Nov 2017 06:19:46 +1100
> Chris Angelico  wrote:
> >
> > I've used a good few dictionary objects in my time, but most of them
> > have literally never had any items deleted from them.
>
> Well... It really depends what kind of problem you're solving.  I
> certainly delete or pop items from dicts quite often.
>
> Let's not claim that deleting items from a dict is a rare or advanced
> feature.  It is not.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> pludemann%40google.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Antoine Pitrou]

On Wed, 8 Nov 2017 06:19:46 +1100
Chris Angelico  wrote:
> 
> I've used a good few dictionary objects in my time, but most of them
> have literally never had any items deleted from them.

Well... It really depends what kind of problem you're solving.  I
certainly delete or pop items from dicts quite often.

Let's not claim that deleting items from a dict is a rare or advanced
feature.  It is not.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Chris Angelico]

On Wed, Nov 8, 2017 at 1:32 AM, Antoine Pitrou  wrote:
> On Wed, 8 Nov 2017 00:01:04 +1000
> Nick Coghlan  wrote:
>
>> On 7 November 2017 at 23:48, Stefan Krah  wrote:
>> >
>> >
>> > This is just a reminder that the current dict is not an "OrderedDict":
>> >
>>  from collections import OrderedDict
>>  OrderedDict(a=0, b=1) == OrderedDict(b=1, a=0)
>> > False
>>  dict(a=0, b=1) == dict(b=1, a=0)
>> > True
>> >
>> > The recent proposal was primarily about guaranteeing the insertion order of
>> > dict literals.
>> >
>> > If further guarantees are proposed, perhaps it would be a good idea to
>> > open a new thread and state what exactly is being proposed.
>>
>> "Insertion ordered until the first key removal" is the only guarantee
>> that's being proposed.
>
> Is it?  It seems to me that many arguments being made are only relevant
> under the hypothesis that insertion is ordered even after the first key
> removal.  For example the user-friendliness argument, for I don't
> think it's very user-friendly to have a guarantee that disappears
> forever on the first __del__.

I've used a good few dictionary objects in my time, but most of them
have literally never had any items deleted from them. Consider a
simple anagram finder:

anagrams = defaultdict(list)
for word in words:
anagrams[''.join(sorted(word))].append(word)
for words in anagrams.values():
if len(words) > 5:
print(words)

New items get added to the dictionary, but nothing is ever removed. I
can assume, with CPython's current semantics, that the final iteration
will be in order of first seen; whatever order the incoming word list
was in, the output will be in too. This IS a useful guarantee, even
with the caveats.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Barry Warsaw]

On Nov 7, 2017, at 07:12, Antoine Pitrou  wrote:

> The problem is this is taking things to a level of precision that makes
> the guarantee tedious to remember and reason about.
> 
> The only thing that's friendly to (non-expert) users is either "dicts
> are always ordered [by insertion order], point bar" or "dicts are not
> ordered, point bar".  Anything in-between, with reservations depending
> on which operations are invoked and when, is not really helpful to the
> average (non-expert) user.
> 
> Which is why I think the user-friendliness argument does not apply if
> order ceases to be guaranteed after __del__ is called.

That’s a very important point.  If it’s difficult to explain, teach, and retain 
the different ordering guarantees between built-in dict and OrderedDict, it 
might in fact be better to not guarantee any ordering for built-in dict *in the 
language specification*.  Otherwise we might need a section as big as chapter 5 
in the Python Language Reference just to dict ordering semantics. ;)

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[INADA Naoki]

>> > If further guarantees are proposed, perhaps it would be a good idea to
>> > open a new thread and state what exactly is being proposed.
>>
>> "Insertion ordered until the first key removal" is the only guarantee
>> that's being proposed.
>
> Is it?  It seems to me that many arguments being made are only relevant
> under the hypothesis that insertion is ordered even after the first key
> removal.  For example the user-friendliness argument, for I don't
> think it's very user-friendly to have a guarantee that disappears
> forever on the first __del__.
>

I agree with Antoine.  It's "hard to explain" than "preserving insertion order".

Dict performance is important because it's used for namespace.
But delete-heavy workload is not happen for namespace.

It may make workloads like LRU caching slightly.
But I don't think performance gain is large enough.  Many overhead
comes from API layer wrapping LRU cache. (e.g. functools.lru_cache)

So I expect performance difference can be found only on some
micro benchmarks.

Additionally, class namespace should keep insertion order.  It's language
spec from 3.6.  So we should have two mode for such optimization.
It makes dict more complicated.

So I'm +0.5 on making dict order as language spec, and -1 on "preserves
insertion order until deletion" idea.

But my expect may be wrong.  Serhiy is working on it so I'm waiting it
to benchmark.

Regards,

INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Serhiy Storchaka]

07.11.17 16:56, Steven D'Aprano пише:

To clarify: if we start with an empty dict, add keys A...D, delete B,
then add E...H, we could expect:

{A: 1}
{A: 1, B: 2}
{A: 1, B: 2, C: 3}
{A: 1, B: 2, C: 3, D: 4}
{D: 4, A: 1, C: 3}  # some arbitrary reordering
{D: 4, A: 1, C: 3, E: 5}
{D: 4, A: 1, C: 3, E: 5, F: 6}
{D: 4, A: 1, C: 3, E: 5, F: 6, G: 7}
{D: 4, A: 1, C: 3, E: 5, F: 6, G: 7, H: 8}


Rather

{A: 1, D: 4, C: 3}  # move the last item in place of removed
{A: 1, D: 4, C: 3, E: 5}
{A: 1, D: 4, C: 3, E: 5, F: 6}
{A: 1, D: 4, C: 3, E: 5, F: 6, G: 7}
{A: 1, D: 4, C: 3, E: 5, F: 6, G: 7, H: 8}

or

{A: 1, C: 3, D: 4}
{A: 1, E: 5, C: 3, D: 4}  # place the new item in place of removed
{A: 1, E: 5, C: 3, D: 4, F: 6}
{A: 1, E: 5, C: 3, D: 4, F: 6, G: 7}
{A: 1, E: 5, C: 3, D: 4, F: 6, G: 7, H: 8}

or

{A: 1, C: 3, D: 4}
{A: 1, C: 3, D: 4, E: 5}  # add new items at end until fill the array
{A: 1, F: 6, C: 3, D: 4, E: 5}  # and fill holes after that
{A: 1, F: 6, C: 3, D: 4, E: 5, G: 7}  # reallocate the array
{A: 1, F: 6, C: 3, D: 4, E: 5, G: 7, H: 8}

These scenarios are more probably.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Antoine Pitrou]

On Tue, 7 Nov 2017 09:44:07 -0500
Yury Selivanov  wrote:
> 
> One common pattern that I see frequently is this:
> 
>  def foo(**kwargs):
>   kwargs.pop('somekey', None)
>   bar(**kwargs)

I see it frequently too, but that's in code meant to be
Python 2-compatible (and therefore cannot count on any ordering
guarantee, even de facto). On Python 3 you can write:

   def foo(somekey=None, **kwargs):
   # do something with somekey?
   bar(**kwargs)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Antoine Pitrou]

On Wed, 8 Nov 2017 01:56:42 +1100
Steven D'Aprano  wrote:
> 
> I think that Nick's intent was not to say that after a single deletion, 
> the ordering guarantee goes away "forever", but that a deletion may be 
> permitted to reorder the keys, after which further additions will honour 
> insertion order. At least, that's how I interpret him.

The problem is this is taking things to a level of precision that makes
the guarantee tedious to remember and reason about.

The only thing that's friendly to (non-expert) users is either "dicts
are always ordered [by insertion order], point bar" or "dicts are not
ordered, point bar".  Anything in-between, with reservations depending
on which operations are invoked and when, is not really helpful to the
average (non-expert) user.

Which is why I think the user-friendliness argument does not apply if
order ceases to be guaranteed after __del__ is called.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Steven D'Aprano]

On Tue, Nov 07, 2017 at 03:32:29PM +0100, Antoine Pitrou wrote:

[...]
> > "Insertion ordered until the first key removal" is the only guarantee
> > that's being proposed.
> 
> Is it?  It seems to me that many arguments being made are only relevant
> under the hypothesis that insertion is ordered even after the first key
> removal.  For example the user-friendliness argument, for I don't
> think it's very user-friendly to have a guarantee that disappears
> forever on the first __del__.

Don't let the perfect be the enemy of the good.

For many applications, keys are never removed from the dict, so this 
doesn't matter. If you never delete a key, then the remaining keys will 
never be reordered.

I think that Nick's intent was not to say that after a single deletion, 
the ordering guarantee goes away "forever", but that a deletion may be 
permitted to reorder the keys, after which further additions will honour 
insertion order. At least, that's how I interpret him.

To clarify: if we start with an empty dict, add keys A...D, delete B, 
then add E...H, we could expect:

{A: 1}
{A: 1, B: 2}
{A: 1, B: 2, C: 3}
{A: 1, B: 2, C: 3, D: 4}
{D: 4, A: 1, C: 3}  # some arbitrary reordering
{D: 4, A: 1, C: 3, E: 5}
{D: 4, A: 1, C: 3, E: 5, F: 6}
{D: 4, A: 1, C: 3, E: 5, F: 6, G: 7}
{D: 4, A: 1, C: 3, E: 5, F: 6, G: 7, H: 8}


Nick, am I correct that this was your intent?



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Yury Selivanov]

On Tue, Nov 7, 2017 at 9:32 AM, Antoine Pitrou  wrote:
> On Wed, 8 Nov 2017 00:01:04 +1000
> Nick Coghlan  wrote:
>
>> On 7 November 2017 at 23:48, Stefan Krah  wrote:
>> >
>> >
>> > This is just a reminder that the current dict is not an "OrderedDict":
>> >
>>  from collections import OrderedDict
>>  OrderedDict(a=0, b=1) == OrderedDict(b=1, a=0)
>> > False
>>  dict(a=0, b=1) == dict(b=1, a=0)
>> > True
>> >
>> > The recent proposal was primarily about guaranteeing the insertion order of
>> > dict literals.
>> >
>> > If further guarantees are proposed, perhaps it would be a good idea to
>> > open a new thread and state what exactly is being proposed.
>>
>> "Insertion ordered until the first key removal" is the only guarantee
>> that's being proposed.
>
> Is it?  It seems to me that many arguments being made are only relevant
> under the hypothesis that insertion is ordered even after the first key
> removal.  For example the user-friendliness argument, for I don't
> think it's very user-friendly to have a guarantee that disappears
> forever on the first __del__.

One common pattern that I see frequently is this:

 def foo(**kwargs):
  kwargs.pop('somekey', None)
  bar(**kwargs)

With ordering breaking on first pop/delete we essentially have no
guarantee about kwargs order, or at least it's very easy to break.  It
would make writing wrappers like this extremely tedious -- we are
essentially forcing people to use OrderedDict to just pop an item from
kwargs.  Not to mention that this isn't cheap in terms of performance.

Is there a *real* motivation for saying that pop/delete can break the order?

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Antoine Pitrou]

On Wed, 8 Nov 2017 00:01:04 +1000
Nick Coghlan  wrote:

> On 7 November 2017 at 23:48, Stefan Krah  wrote:
> >
> >
> > This is just a reminder that the current dict is not an "OrderedDict":
> >  
>  from collections import OrderedDict
>  OrderedDict(a=0, b=1) == OrderedDict(b=1, a=0)  
> > False  
>  dict(a=0, b=1) == dict(b=1, a=0)  
> > True
> >
> > The recent proposal was primarily about guaranteeing the insertion order of
> > dict literals.
> >
> > If further guarantees are proposed, perhaps it would be a good idea to
> > open a new thread and state what exactly is being proposed.  
> 
> "Insertion ordered until the first key removal" is the only guarantee
> that's being proposed.

Is it?  It seems to me that many arguments being made are only relevant
under the hypothesis that insertion is ordered even after the first key
removal.  For example the user-friendliness argument, for I don't
think it's very user-friendly to have a guarantee that disappears
forever on the first __del__.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Stefan Krah]

On Wed, Nov 08, 2017 at 12:01:04AM +1000, Nick Coghlan wrote:
> > The recent proposal was primarily about guaranteeing the insertion order of
> > dict literals.
> >
> > If further guarantees are proposed, perhaps it would be a good idea to
> > open a new thread and state what exactly is being proposed.
> 
> "Insertion ordered until the first key removal" is the only guarantee
> that's being proposed.
> 
> OrderedDict just comes into the discussion because reaching for its
> stronger guarantees is currently the only way to obtain that guarantee
> in a formally implementation-independent and future-proof way.

Ok good, I was primarily worried about collections.UnorderedDict coming up
and users thinking that OrderedDict could be replaced entirely by dict().



Stefan Krah




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Nick Coghlan]

On 7 November 2017 at 23:48, Stefan Krah  wrote:
>
>
> This is just a reminder that the current dict is not an "OrderedDict":
>
 from collections import OrderedDict
 OrderedDict(a=0, b=1) == OrderedDict(b=1, a=0)
> False
 dict(a=0, b=1) == dict(b=1, a=0)
> True
>
> The recent proposal was primarily about guaranteeing the insertion order of
> dict literals.
>
> If further guarantees are proposed, perhaps it would be a good idea to
> open a new thread and state what exactly is being proposed.

"Insertion ordered until the first key removal" is the only guarantee
that's being proposed.

OrderedDict just comes into the discussion because reaching for its
stronger guarantees is currently the only way to obtain that guarantee
in a formally implementation-independent and future-proof way.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] The current dict is not an "OrderedDict"

[Stefan Krah]

This is just a reminder that the current dict is not an "OrderedDict":

>>> from collections import OrderedDict
>>> OrderedDict(a=0, b=1) == OrderedDict(b=1, a=0)
False
>>> dict(a=0, b=1) == dict(b=1, a=0)
True


The recent proposal was primarily about guaranteeing the insertion order of
dict literals.

If further guarantees are proposed, perhaps it would be a good idea to
open a new thread and state what exactly is being proposed.



Stefan Krah



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com