[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Change by Antony Lee : -- nosy: -Antony.Lee ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Change by Nathaniel Smith : -- pull_requests: -19900 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Change by Daniel Fortunov : -- keywords: +patch nosy: +dfortunov nosy_count: 4.0 -> 5.0 pull_requests: +19900 stage: -> patch review pull_request: https://github.com/python/cpython/pull/20687 ___ Python tracker

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Nathaniel Smith added the comment: Just found this while searching to see if we had an existing way to combine WeakValueDictionary + defaultdict. The use case I've run into a few times is where you need a keyed collection of mutexes, like e.g. asyncio.Lock objects. You want to make sure

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Antony Lee added the comment: Thanks for the clarification! "at least if the original key is not `obj` but some other object equal to `obj`" does not apply in my case so I'm fine, but the example shows that this is tricky to get right... -- ___

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Antoine Pitrou added the comment: Antony, if you write (line numbers added for clarity): 1. try: 2.d[obj] 3. except KeyError: 4.d[obj] = some new value... 5. # expect d[obj] to exist at this point it is possible that d[obj] still exists at line 2 but not anymore at line 5 (because

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Antony Lee added the comment: For my use case, it was easy enough to wrap the `uid = d[obj]` in a try... catch... plus locking. Using setdefault would cause the counter to be incremented every time. In truth, here I did not care about having consecutive uids, so that would have worked just

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Raymond Hettinger added the comment: One way to do it: - diff --git a/Lib/weakref.py b/Lib/weakref.py index 1802f32a20..18f26ea8b2 100644 --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -136,6 +136,8 @@ class WeakValueDictionary(collections.abc.MutableMapping):

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Raymond Hettinger added the comment: Just for comparison, what is your best solution without the proposed API change?It is helpful to see user code before-and-after to know whether there is an improvement that makes the change worth it. -- ___

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Antony Lee added the comment: The original example did not cover the use case and was only there to show the current behavior. The real use case is more something like obj = (get obj as argument to function, caller has a hard reference to it) uid = d[obj] --

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Raymond Hettinger added the comment: [Anthony Lee] > The use case is to generate a mapping of weakly-held objects > to unique ids, with something like > > id_map = WeakKeyDictionaryWithMissing(lambda *, _counter=itertools.count(): > > next(_counter)) Where are you keeping hard references to

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Antoine Pitrou added the comment: > Of course, as always when using defaultdict, it is easy enough to instead > implement this by manually checking if the key is present and store a new id > in this case Or, better, use setdefault(), since manually checking suffers from a race condition in

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Antony Lee added the comment: The use case is to generate a mapping of weakly-held objects to unique ids, with something like id_map = WeakKeyDictionaryWithMissing(lambda *, _counter=itertools.count(): next(_counter)) Of course, as always when using defaultdict, it is easy enough to instead

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Raymond Hettinger added the comment: We should still ask about use cases though. Without a motivating use case, why churn the code, complexify the API (possibly making it more tricky to use), and risk introducing bugs? AFAICT, the OP's sole motivation was "still, it would seem preferable

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Antoine Pitrou added the comment: If someone wants to submit a PR, it would help judge the complexity and fragility of adding support for this. I cannot think of any problem upfront myself, but you're right that weak containers have been very difficult to make robust against various

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

Raymond Hettinger added the comment: For WeakKeyDictionary, I suspect that adding a __missing__() call would make the API more tricky and would likely cause more problems than it solves. Without a compelling use case, my recommendation would be to leave it alone. (FWIW, all of the weak

[issue31254] WeakKeyDictionary/Mapping doesn't call __missing__

New submission from Antony Lee: The following example, which raises a KeyError, shows that in a WeakKeyDictionary subclass that defines __missing__, that method doesn't get called. from weakref import WeakKeyDictionary class WeakKeyDictionaryWithMissing(WeakKeyDictionary):