Response inline.
On Mon, 2021-05-03 at 23:30 +0100, Matt del Valle wrote:
> @Paul Bryan, I'll try to address your questions one-by-one
>
> > 1. It seems that I could get close to what you're aiming for by
> > just using underscores in names, and grouping them together in the
> > class definition. Is there any advantage to declaring methods in a
> > namespace beyond indentation and the dot notation?
> >
>
>
> What you're suggesting (using underscores for namespacing) is
> something I've seen done many times (and done myself plenty of times)
> and it's precisely one of the main things that I think could be
> better under this proposal. Reusing a prefix like that across methods
> has always felt a bit hacky to me and like it doesn't follow the DRY
> principle very well. If you wanted to rename the prefix at a later
> point you would have to go through every single method that uses it,
> instead of just doing a single refactor/rename on the name at the top
> of the namespace block.
Sure, it's a bit hacky, but virtually every editor can do it in a few
keystrokes, assuming no naming collision that requires you to cherry-
pick values to be substituted.
> But you've pretty much perfectly identified the benefits here, I'll
> just elaborate on them a bit.
>
> - the indentation visually separates blocks of conceptually-grouped
> attributes/methods in the actual code (a gain in clarity when code is
> read)
> - the dot notation you use to invoke such methods improves the
> experience for library consumers by giving a small amount
> conceptually-linked autocompletions at each namespace step within a
> class with a large API, rather getting a huge flat list.
>
> Furthermore, the proposed namespaces have other selling-points
> outside of the use-case of grouping methods within a class.
>
> While the benefit of namespaces within modules is more dubious
> because you can use class blocks for namespacing (and people often
> do), can see a few ways that the namespace proposal is better:
>
> - you can put functions inside a namespace block, which would become
> methods if you had put them in a class block
> - you don't have the same (in some cases extremely unintuitive)
> scoping/variable binding rules that you do within a class block (see
> the link in my doc). It's all just module scope.
> - it mode clearly indicates intent (you don't want a whole new class,
> just a new namespace)
>
> When using the namespaces within a method (using self):
>
> - It allows you to namespace out your instance attributes without
> needing to create intermediate objects (an improvement to the memory
> footprint, and less do-nothing classes to clutter up your codebase)
> - While the above point of space complexity will not alway be
> relevant I think the more salient point is that creating intermediate
> objects for namespacing is often cognitively more effort than it's
> worth. And humans are lazy creatures by nature. So I feel like having
> an easy and intuitive way of doing it would have a positive effect on
> people's usage patterns. It's one of those things where you likely
> wouldn't appreciate the benefits until you'd actually gotten to play
> around with it a bit in the wild. For example, you could rewrite
> this:
>
> class Legs:
> def __init__(self, left, right):
> self.left, self.right = left, right
>
>
> class Biped:
> def __init__(self):
> self.legs = Legs(left=LeftLeg(), right=RightLeg())
>
> As this:
>
> class Biped:
> def __init__(self):
> namespace self.legs:
> left, right = LeftLeg(), RightLeg()
>
> And sure, the benefit for a single instance of this is small. But
> across a large codebase it adds up. It completely takes away the
> tradeoff between having neatly namespaced code where it makes sense
> to do so and writing a lot of needless intermediate classes.
So, is it a prerequisite that whatever object in which I'm trying to
establish a namespace must support getattr/setattr?
Also, it occurs to me that if I can declare a namespace in any object,
I might be tempted to (or might inadvertently) monkey patch external
objects with it. Any thoughts on guarding against that, or is this
"we're adults here" case?
> SimpleNamespace does not help you here as much as you would think
> because it cannot be understood by static code analysis tools when
> invoked like this:
>
> class Biped:
> def __init__(self):
> self.legs = SimpleNamespace(left=LeftLeg(), right=RightLeg())
>
> So it is a terrible idea to use it in this way to write any kind of
> library code. You could invoke it declaratively:
>
> class Biped:
> def __init__(self):
> class Legs(SimpleNamespace):
> left, right = LeftLeg(), RightLeg()
>
> self.legs = Legs
>
> Not only is this an unintuitive usage pattern and creates lots of
> unnecessary classes (one for each new instance of Biped), but it
> significantly reduces code clarity (imagine having lots of these
> inside a single method). By contrast, a namespace block seems to me
> to increase code clarity, though I will grant that this is
> subjective.
>
>
> > 2. If __dict__ contains "B.C" and "B", then presumably the
> > interpreter would need to try combinations against the outer
> > __dict__ as well as B. Is the namespace proxy you've mentioned
> > intended to prevent further lookup in the "B" attribute?
> >
>
>
> The namespace proxy must know its fully-qualified name all the way up
> to its parent scope (this is the bit that would require some magic in
> the python implementation), so it only needs to forward on a single
> attribute lookup to its parent scope. It does not need to perform
> several intermediate lookups on all of its parent namespaces.
>
> So in the case of:
>
> namespace A:
> namespace B:
> C = True
>
>
> >>>A.B
> <namespace object <A.B> of <module '__main__' (built-in)>>
>
>
> Note that namespace B 'knows' that its name is 'A.B', not just 'B'
>
> The implementation of this namespace proxy object might look
> something like this, only implemented in C to hopefully be more
> performant than this:
>
> class NamespaceProxy:
> ...
> def __getattr__(self, name):
> return getattr(self.__parent_scope__,
> f"{self.__namespace_name__}.{name}")
>
> def __setattr__(self, name, value):
> setattr(self.__parent_scope__, f"{self.__namespace_name__}.{name}",
> value)
>
>
> The names of the dunders are hypothetical and not something I'm
> concerned about at this stage.
>
> Traversing all the way through A.B.C does involve 2 intermediate
> lookups (looking up 'A.B' on the parent scope from namespace A, then
> looking up 'A.B.C' on the parent scope from namespace A.B). But once
> you have a reference to a deeply nested namespace, looking up any
> value on it is only a single lookup step.
A problem I sense here is the fact that the interpreter would always
need to attempt to resolve "A.B.C" as getattr(getattr(A, "B"), "C") and
getattr(A, "B.C"). Since the proxy would be there to serve up the
namespace's attributes, why not just let it and do away with "B.C" in
A.__dict__? What will the performance costs be of attempting to get an
attribute in two calls instead of one?
>
>
> > 3. Can namespaces be nested? If so, will their attributed they
> > always resolve to flat set of attributes in the encapsulating
> > class?
> >
>
>
> Yes, namespaces can be nested arbitrarily, and they will always set
> their attributes in the nearest real scope (module/class/locals).
> There's an example of this early on in the doc:
> namespace constants:
> NAMESPACED_CONSTANT = True
>
> namespace inner:
> ANOTHER_CONSTANT = "hi"
> Which is like:
> vars(sys.modules[__name__])["constants.NAMESPACED_CONSTANT"] = True
> vars(sys.modules[__name__])["constants.inner.ANOTHER_CONSTANT"] =
> "hi"
So, if in a nested scenario, A.B.C.D, I'm trying to understand the
combination of getattr calls to resolve D. Would it just still be two
attempts, getattr(A, "B.C.D") and getattr(getattr(getattr(A, "B"),
"C"), "D")? If it were to become a Cartesian product of calls, there'll
likely be a problem. 🤔️
>
> > 4. What would you expect getattr(A.B, "C") to yield?
> >
>
>
> It would be no different than:
>
> >>>A.B.C
>
> It happens in two steps:
>
> 1) A.B - this looks up vars(<module '__main__' (built-in)>)['A.B'],
> which is a namespace object
> 2) getattr(<namespace object <A.B> of <module '__main__' (built-
> in)>>, "C") - this looks up "C" on namespace "A.B", which forwards
> this lookup on to the module as: vars(<module '__main__' (built-
> in)>)['A.B.C']
>
> The namespace proxies created in a namespace block do nothing more
> than forward on any attribute access/attribute setting operations
> performed on them to their parent scope, so you can do any attribute
> operations to them that you could do to a 'real' intermediate object.
>
>
> I hope I've answered everything with reasonable clarity.
>
> I'm kind of exhausted from writing this up, but I'll finally just
> quickly mention in response to David Mertz that your proposal to use
> a metaclass for this wouldn't really serve to provide virtually any
> of the benefits I listed out in my answer to Paul (part 1, in
> particular). it wouldn't work for modules/locals, and it can't be
> used at the instance level with `self`, only the class level. It
> would also restrict usage of other metaclasses. What it would end up
> doing (saving class attributes of nested classes to its own __dict__
> with a fully-qualified name) is an implementation detail of my
> proposal rather than one of the primary benefits.
>
> From where I'm standing it feels like you just took an immediate
> initial dislike to the proposal and didn't even really give it a
> chance by actually thinking through the pros/cons before giving your
> -1.
>
> It's fine if you think it's not useful enough to be added. Maybe
> that's true. But it would be more helpful to maybe ask questions like
> other people are doing and be sure you understand the
> rationale/implications of the proposal before making your mind up.
>
> Three negative responses in you still didn't understand a really
> simple implementation point that's written very early on in the toy
> example of the doc, which differentiates it from
> types.SimpleNamespace. Then when Steve pointed that out to you, you
> immediately dug your heels in again without really thinking your 'it
> can be done with a metaclass' counterpoint all the way through. At
> least give yourself the chance to change your mind! Even if you end
> up settling on your initial intuition, at least that way you'll know
> that you've explored all the options and done your best to land on
> the most informed opinion you can.
>
> Anyways, I'm sorry if I totally misrepresented the situation. It's
> hard sometimes to tell over a medium like this what page someone else
> is on. I just figured I'd tell you how it felt to me. I hope I wasn't
> totally off-base.
>
> On Mon, May 3, 2021 at 8:16 PM Paul Bryan <pbr...@anode.ca> wrote:
> > Correction:
> >
> > 4. What would you expect getattr(A.B, "C") to yield?
> >
> > Paul
> >
> > On Mon, 2021-05-03 at 12:10 -0700, Paul Bryan wrote:
> > > I've read the proposal, and this thread.
> > >
> > > Questions:
> > >
> > > 1. It seems that I could get close to what you're aiming for by
> > > just using underscores in names, and grouping them together in
> > > the class definition. Is there any advantage to declaring methods
> > > in a namespace beyond indentation and the dot notation?
> > >
> > > 2. If __dict__ contains "B.C" and "B", then presumably the
> > > interpreter would need to try combinations against the outer
> > > __dict__ as well as B. Is the namespace proxy you've mentioned
> > > intended to prevent further lookup in the "B" attribute?
> > >
> > > 3. Can namespaces be nested? If so, will their attributed they
> > > always resolve to flat set of attributes in the encapsulating
> > > class?
> > >
> > > 4. What would you expect getattr("A.B", "C") to yield?
> > >
> > > Paul
> > >
> > > On Mon, 2021-05-03 at 19:49 +0100, Stestagg wrote:
> > > > The first example in the doc lays out the difference:
> > > >
> > > > Assignments within the namespace block have this special
> > > > behaviour whereby the assigned-to name is changed to be:
> > > > ‘<namespace name>.<assignment name>’
> > > > And the assignment is made in the ‘parent scope’ of the
> > > > namespace.
> > > >
> > > > I.e. (again, as described in the doc):
> > > >
> > > > class A:
> > > > Namespace B:
> > > > C = 1
> > > >
> > > > Results in:
> > > >
> > > > A.__dict__ == {‘B.C’: 1, ‘B’: <namespace proxy>}
> > > >
> > > > Note the ‘B.C’ entry
> > > >
> > > > Now for me, the only place where is starts to be interesting is
> > > > with methods within namespaces, where the ‘self’ binding is
> > > > made against to top-level class, and not against the
> > > > namespace. This does make for some nicer nested API
> > > > definitions (a-la pandas DataFrame.str.xxx methods) where an
> > > > intermediate name os used just to group and organise methods.
> > > >
> > > >
> > > >
> > > > On Mon, 3 May 2021 at 19:40, David Mertz <me...@gnosis.cx>
> > > > wrote:
> > > > > On Mon, May 3, 2021 at 6:37 PM Stestagg <stest...@gmail.com>
> > > > > wrote:
> > > > > > On Mon, 3 May 2021 at 19:24, David Mertz <me...@gnosis.cx>
> > > > > > wrote:
> > > > > > > So yes... as I thought, SimpleNamespace does EVERYTHING
> > > > > > > described by the proposal, just without needing more
> > > > > > > keywords:
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Except that the code and description of the proposal
> > > > > > explicitly outline behaviours that SimpleNamespace does not
> > > > > > provide (and aren’t trivially possible to add)
> > > > > >
> > > > >
> > > > >
> > > > > I've looked and been unable to find an example of that. Can
> > > > > you show one?
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Python-ideas mailing list -- python-ideas@python.org
> > > > > To unsubscribe send an email to python-ideas-le...@python.org
> > > > >
> > > > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > > > > Message archived
> > > > >
> > > >
> >
> at
> https://mail.python.org/archives/list/python-ideas@python.org/message/YTARLBP3TIARJ4FUEPPDZAUIS33P2C3Q/
> > > > > Code of Conduct: http://python.org/psf/codeofconduct/
> > >
> > > _______________________________________________
> > > Python-ideas mailing list -- python-ideas@python.org
> > > To unsubscribe send an email to python-ideas-le...@python.org
> > > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > > Message archived
> > >
> >
> at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UAZBIPDC3DRSA5A2GST72KDO2Y2R6RBX/
> > > Code of Conduct: http://python.org/psf/codeofconduct/
> >
> >
> > _______________________________________________
> > Python-ideas mailing list -- python-ideas@python.org
> > To unsubscribe send an email to python-ideas-le...@python.org
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > Message archived
> >
> at
> https://mail.python.org/archives/list/python-ideas@python.org/message/Q2OTIBAPH7GUBAMKOY3DF3HZWQI4OZMO/
> > Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/3C6TVDF352F43SYZO5KBKIQCO7PQ2G7F/
Code of Conduct: http://python.org/psf/codeofconduct/
