[issue34586] collections.ChainMap should have a get_where method

[Zahari Dim]

Zahari Dim  added the comment:

>
> I've discussed this with other core devs and spent a good deal of time 
> evaluating this proposal.  I'm going to pass on the this one but do think it 
> was a inspired suggestion.  Thank you for the proposal.

Thank you for taking the time to consider it. I understand that there
are many proposals.

>
> --
>
> Note, the original get_where() recipe has an issue.  Upon successful lookup, 
> it returns a 2-tuple but on failure it calls __missing__ which typically 
> returns a scalar (if it doesn't raise an exception).

FWIW this was intended to work when `__missing__` was subclassed to
raise a more specific exception. The case where it is made to return a
value clearly doesn't play well with the proposed method, and would
likely need to be subclassed as well. I consider this an acceptable
trade off because I find this use case rather esoteric: the same
functionality could be achieved in an arguably clearer way by passing
a mapping with the desired missing semantics as the outermost scope.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I've discussed this with other core devs and spent a good deal of time 
evaluating this proposal.  I'm going to pass on the this one but do think it 
was a inspired suggestion.  Thank you for the proposal.

--

Note, the original get_where() recipe has an issue.  Upon successful lookup, it 
returns a 2-tuple but on failure it calls __missing__ which typically returns a 
scalar (if it doesn't raise an exception).

The proposed use case is plausible but rare.  As Serhiy mentioned, it is also 
reasonable for the user to write their own loops over the *maps* attribute.  
That loop can have subtleties but it is better to let the user choose between 
try/except semantics and if-else semantics according to the use case.

FWIW, I tried out several variants of the proposal including "cm.index(key) -> 
int" and "cm.search(key) -> mapping".  Each of the variants had some 
awkwardness when trying to use it there didn't seem to be a variant that was a 
net win.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Zahari Dim]

Zahari Dim  added the comment:

On Sat, Sep 8, 2018 at 1:15 PM Serhiy Storchaka  wrote:
>
>
> Serhiy Storchaka  added the comment:
>
> I concur with Raymond. The purpose of ChainMap is providing a mapping that 
> hides the implementation detail of using several mappings as fallbacks. If 
> you don't want to hide the implementation detail, you don't need to use 
> ChainMap.
>
> ChainMap exposes underlying mappings as the maps attribute, so you can use 
> this implementation detail if you know that it is a ChainMap an not a general 
> mapping. It is easy to write a code for searching what mapping contains the 
> specified key.

I don't know where the idea that the underlying mappings are an
implementation detail comes from. It certainly isn't from the
documentation, which mentions uses such as nested scopes and
templates, which cannot be attained with a single mapping. It also
doesn't match my personal usage, where as discussed, even the simpler
cases benefit from information on the underlying mappings. It is a
surprising claim to make given than the entirety of the public
interface specific to ChainMap (maps, new_child and parents) deals
with the fact that there is more structure than one mapping. I also
have a hard time discerning this idea from Raymond's messages.

>
> for m in cm.maps:
> if key in m:
> found = m
> break
> else:
> # raise an error or set a default,
> # what is appropriate for your concrete case

This "trivial snatch of code" contains at least two issues that make
it fail in situations where the actual implementation of `__getitem__`
would work, opening the door for hard to diagnose corner cases. If
anything, in my opinion, the fact that this code is being proposed as
an alternative reinforces the idea that the implementation of the
searching method should be in the standard library.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I concur with Raymond. The purpose of ChainMap is providing a mapping that 
hides the implementation detail of using several mappings as fallbacks. If you 
don't want to hide the implementation detail, you don't need to use ChainMap.

ChainMap exposes underlying mappings as the maps attribute, so you can use this 
implementation detail if you know that it is a ChainMap an not a general 
mapping. It is easy to write a code for searching what mapping contains the 
specified key.

for m in cm.maps:
if key in m:
found = m
break
else:
# raise an error or set a default,
# what is appropriate for your concrete case

or even

found = next(m for m in cm.maps if key in m)

if the key is always found or StopIteration is appropriate.

I don't think that such trivial snatch of code is worth adding a special 
ChainMap method or even documenting it explicitly in the ChainMap documentation.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Zahari Dim]

Zahari Dim  added the comment:

> ISTM that this is the wrong stage to perform validation of allowable values.  
> That should occur upstream when the underlying mappings are first created.  
> At that earlier stage it possible to give a more timely response to erroneous 
> input and there is access to more information (such as the line and row 
> number of an error in a configuration file).
>
> It doesn't make sense to me to defer value validation downstream after a 
> ChainMap instance has been formed and after a successful lookup has occurred. 
> That just complicates the task of tracing back to the root cause.

This is certainly the case in the situation where the validation only
depends on the value of the corresponding configuration entry, as it
admittedly does in the example above. However the example was
oversimplified insofar non trivial validation depends on the whole
ensemble configuration settings. For example taking the example
described at the top of

I think it would be useful to have an error message of the form:
f"User '{db_username}', defined in {configsetttings[user_index]} is
not found in database '{database}', defined in
{configsettings[database_index]}'

>
> > Maybe the method could be called ChainMap.search?
>
> That would be better than get_where().
>
> --
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> I would argue that all of the examples would benefit from error
> checking done along the lines of the snippet above.

ISTM that this is the wrong stage to perform validation of allowable values.  
That should occur upstream when the underlying mappings are first created.  At 
that earlier stage it possible to give a more timely response to erroneous 
input and there is access to more information (such as the line and row number 
of an error in a configuration file).  

It doesn't make sense to me to defer value validation downstream after a 
ChainMap instance has been formed and after a successful lookup has occurred. 
That just complicates the task of tracing back to the root cause.

> Maybe the method could be called ChainMap.search?

That would be better than get_where().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Zahari Dim]

Zahari Dim  added the comment:

I believe an argument for including this functionality in the standard library
is that it facilitates writing better error messages and thus better code. Some
results that are returned when one searches for *python ChainMap* are:

  - 

  - 

  - 

All of these mention prominently a layered configuration of some kind. I would
argue that all of the examples would benefit from error checking done along the
lines of the snippet above.

An additional consideration is that the method is best implemented by copying
the `__getitem__` method, which, while short, contains a couple of non trivial
details.

One analog could be `re.search`, which returns an object with information of
both the value that is found and its location, though the `span` attribute of
the Match object. Maybe the method could be called ChainMap.search?
On Thu, Sep 6, 2018 at 6:07 AM Raymond Hettinger  wrote:
>
>
> Raymond Hettinger  added the comment:
>
> I haven't run across this requirement before but it does seem plausible that 
> a person might want to know which underlying mapping found a match (compare 
> with the "which" utility in Bash). On the other hand, we haven't had requests 
> for anything like this for other lookup chains such as determining where a 
> variable appears in the sequence 
> locals-to-nested-scopes-to-globals-to-builtins.
>
> Also, I'm not sure I like the proposed API (the method name and signature).  
> Perhaps, this should be a docs recipe for a ChainMap subclass or be an 
> example of a standalone search function that the takes the *maps* attribute 
> as one of its arguments.  Will discuss this with the other core devs to get 
> their thoughts.
>
> --
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I haven't run across this requirement before but it does seem plausible that a 
person might want to know which underlying mapping found a match (compare with 
the "which" utility in Bash). On the other hand, we haven't had requests for 
anything like this for other lookup chains such as determining where a variable 
appears in the sequence locals-to-nested-scopes-to-globals-to-builtins.

Also, I'm not sure I like the proposed API (the method name and signature).  
Perhaps, this should be a docs recipe for a ChainMap subclass or be an example 
of a standalone search function that the takes the *maps* attribute as one of 
its arguments.  Will discuss this with the other core devs to get their 
thoughts.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34586] collections.ChainMap should have a get_where method

[Zahari Dim]

New submission from Zahari Dim :

When using ChainMap I have frequently needed to know the mapping inside the list
that contains the effective instance of a particular key. I have needed this
when using ChainMap to contain a piece of configuration with multiple sources,
like for example

```
from mycollections import ChainMap
configsources = ["Command line", "Config file", "Defaults"]
config = ChainMap(config_from_commandline(), config_from_file(),
  default_config())

class BadConfigError(Exception): pass
def get_key(key):
try:
index, value = config.get_where(key)
except KeyError as e:
raise BadConfigError(f"No such key: '{key}'") from e
try:
result = validate(key, value)
except ValidationError as e:
raise BadConfigError(f"Key '{key}' defined in {configsources[index] }"
 f"is invalid: {e}") from e
return result
```

I have also needed this when implementing custom DSLs (e.g. specifying which
context is a particular construct allowed to see).

I think this method would be generally useful for the ChainMap class and
moreover the best way of implementing it I can think of is  by copying the
`__getitem__` method and retaining the index:

```
class ChainMap(collections.ChainMap):
def get_where(self, key):
for i, mapping in enumerate(self.maps):
try:
return i, mapping[key] # can't use 'key in mapping' 
with defaultdict
except KeyError:
pass
return self.__missing__(key)# support subclasses that 
define __missing__
```

I'd be happy to write a patch that does just this.

--
components: Library (Lib)
messages: 324632
nosy: Zahari.Dim
priority: normal
severity: normal
status: open
title: collections.ChainMap should have a get_where method
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com