[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
Christopher Barker writes: > This suggestion would mean that “=>” would create a deferred expression, > but it would not be a general purpose one. I would prefer that "=>" create a general purpose evaluation-deferred expression, but that the general-purpose one be tuned such that you don't need new syntax to dereference it in the common use cases. This generality might be implemented by a "context" attribute on the object, or as Eric suggests it might be a rule that "deferred objects" that are defined as default function arguments get evaluated before the function body (ie, as in PEP 671, I think). > Though maybe there could be a way to evaluate it in a more general way, > kind of like eval — where you can control the namespaces used. There is some such facility in Common Lisp, so I think this is quite feasible. There will be a lot of bikeshedding to be done about *which* namespaces deserve the simplest syntax in *which* contexts. Eg, I don't think there will be much disagreement about evaluating "deferred objects" used as argument defaults just before the function body is entered using the namespaces used in PEP 671, but the case of an actual argument that is a "deferred object" probably would be more contentious. > I guess what I’m suggesting is that we could create a very specific > kind of deferred object, and in the future expand it to more > general use. I don't think this works very well. At the very least, some effort should go into thinking about potential generalization and reserving some names in the class namespace so that all the good names don't get taken by early user-derived classes. I'm not sure this is a good example, but consider the case of Decimal which has a lot of good properties from the point of view of "naive" user expectations about numerical computations. Unfortunately it was late enough to the party that it's not vary attractive unless you actually grok floating point; newbies are just going to use floats. Of course Decimal has all the same kind of traps as float, but they're far more familiar. People expect 1/3 to result in an approximately equal value; they do not expect 1/5 to do so. I wonder if making Decimal the default for decimal literals wouldn't be a good deal, but I guess I'm gonna have to wonder forever. :-) [1] Footnotes: [1] *Not* intended as a suggestion. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HQ3GQKDTQFPFHNAP4BLOGIXNOYF7YV3E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
Okay, here we go:
```
import typing
def extract_docstrings(K):
"""Extract docstrings from the class K's annotations.
Class attribute docstrings are extracted from the second item
in Annotated[...] attributes. The original annotation is left
unchanged.
FIXME:
Handling of PEP 563 string annotations may not be best practice.
https://www.python.org/dev/peps/pep-0563/
"""
d = {}
for key, value in K.__annotations__.items():
if type(value) is str:
if value.startswith('Annotated[') and value.endswith(']'):
s = value[10: -1]
items = s.split(',')
if len(items) >= 2:
doc = items[1].strip()
d[key] = doc
elif typing.get_origin(value) is typing.Annotated:
d[key] = value.__metadata__[0]
K.__attrdocs__ = d
return K
```
And here's an example:
>>> @extract_docstrings
... class Lunch:
... spam: "Annotated[int, 'a delicious meat-like product']"
... eggs: Annotated[float, 'something that goes with spam']
...
>>> Lunch.__attrdocs__
{'spam': "'a delicious meat-like product'", 'eggs': 'something that goes with
spam'}
--
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/I4FZKLAVNMJ6OWICFU47JPZFII4DHRBY/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sun, Dec 12, 2021 at 12:38:06AM -0500, Ricky Teachey wrote: > But Steve, since the most utilized documentation tool in the python > universe, sphinx, doesn't look at Annotated Yet. > or in the lowercase-annotation part of an expression Yet. > for this and instead looks at a bare string below the > member definition, and since I presume it had been doing this for a long > time (before Annotated was added not long ago, and probably before the > annotation too?), doesn't this mean that right now there are at least three > competing ways being used to provide attribute docstrings? And before we had PEP 484 there were multiple competing and incompatible conventions for putting type hints in docstrings. https://www.python.org/dev/peps/pep-0484/ There are: - Epytext (Javadoc style) - ReST (Sphinx) - Googledoc - Numpydoc although some of them may no longer be widely used. https://stackoverflow.com/questions/3898572/what-are-the-most-common-python-docstring-formats https://www.adrian.idv.hk/2018-02-18-docstring/ Static and runtime tools that wanted to get type hints had to guess the format and parse the docstring to get the types. Who still does that? If dataclasses, and other classes, move to using Annotated and __attrdoc__ for docstrings, the tools will follow. Runtime tools will look at the dunder, static tools will look at the annotation directly. **Which they will have to do anyway** if we add the convention that string literals following an assignment get recorded in __attrdoc__. Runtime tools will want to look in the dunder, not parse the source code. And static tools which aren't Sphinx will need to be adjusted to look at the string literals. So either way the tooling has to change. Sphinx is not the decider here. I dare say that they will want to continue supporting bare strings into the indefinite future, but new tools, and those with weaker backwards-compatibility constaints, will surely prefer to extract docstrings from __attrdoc__ than parsing the source code. > As far as using the lowercase-annotation for the docstring: in a world of > more and more type hinted python (to be clear: I don't always use type > hints but I do like this world), if you want to make you don't create a > problem later, using the annotation slot for your docstring: > > x: "spam" > > ... isn't really an option. Of course it is. Just decorate your class with @no_type_hints. > - without an official blessed agreement that the existing Annotated feature > ought to be used for docstrings, there is little reason for other 3rd party > help/documentation tools to look in that location for these docstrings All it takes is literally one stdlib class to start doing it officially, and people will pay attention. > I just think there's more to do here than add a decorator. That depends on whether you mean, what do *we* need to do get the ball rolling, or what does the entire Python ecosystem need to do to transition to the day that every tool and library is actively using attribute docstrings both statically and at runtime? Of course the second part is a bigger job. But for the first, we can either (1): * bikeshed this for another six ~~months~~ weeks; * get at least one core dev to agree to sponsor a PEP; * write a PEP; * bikeshed some more; * wait for the Steering Council to hopefully accept the PEP; * modify the interpreter; * wait for the new version to be available; or we can (2) * implement and support the feature today; to get to the point that external tools can start using it. I'm not kidding, if somebody cared enough to do this, they could probably have a patch for dataclasses to support this within a couple of hours and a plugin for Sphinx within a day. (Spoken with the supreme confidence of somebody who knows that he will absolutely not have to do either.) As far as I am concerned, this is exactly the sort of use-case that Annotated was invented for. Docstrings are metadata. Annotated is for attaching metadata to an annotation that is associated with some variable. Done and done. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/4SMDBNO2FUQCJECDD6ILYK6RAQZ25TNP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sun, Dec 12, 2021 at 5:02 PM Stephen J. Turnbull wrote: > > Chris Angelico writes: > > On Sat, Dec 11, 2021 at 8:07 PM Stephen J. Turnbull > > wrote: > > > > This isn't about your proposal, it's about more general syntax. Not > > > everything being discussed is about your proposal, and I suspect one > > > reason you have trouble figuring out what other people are talking > > > about is that you persistently try to force everything into that > > > context. > > > > Yes, it's silly of me to think of everything in a PEP 671 thread as if > > it's about argument defaults. Carrying on. > > Silly, no, I would say "human", but either way I believe it is > impeding *your* understanding, and almost nobody else's. > > If you're not in *this* subthread to understand alternative ideas > (again, *there is nothing wrong with ending this subthread here*), I > have nothing further to say in it. If you are, you need to calm down > and start asking questions that specify what you want to know rather > than adding a question mark to a grunt as in > By "alternative ideas", do you mean "alternative ways to implement argument defaults", or "completely different ideas that have absolutely nothing to do with argument defaults"? Because if it's the latter, please, change the subject line so it isn't confusing. We can have all manner of completely independent discussions happening at once, and there's no problem. But if you mean "alternative ways to implement argument defaults (and a bunch of other stuff too)", which is what seemed to be the case when people said that PEP 671 should be rescinded in favour of a more generic system, is it really wrong of me to try to think of how this affects PEP 671? Are you, or are you not, asking me to change or retract PEP 671? Does your proposal in any way reflect upon argument defaults? Genuine question. I am utterly, completely, Fblthp-level lost here. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/6LE6EACLWNXZ2JOHRGXUJNBP65ZKHPVX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
Chris Angelico writes: > On Sat, Dec 11, 2021 at 8:07 PM Stephen J. Turnbull > wrote: > > This isn't about your proposal, it's about more general syntax. Not > > everything being discussed is about your proposal, and I suspect one > > reason you have trouble figuring out what other people are talking > > about is that you persistently try to force everything into that > > context. > > Yes, it's silly of me to think of everything in a PEP 671 thread as if > it's about argument defaults. Carrying on. Silly, no, I would say "human", but either way I believe it is impeding *your* understanding, and almost nobody else's. Of course, the *thread* is generally about argument defaults, but "everything" in it is not specifically about defaults. In *this* subthread Eric was arguing for waiting for a facility for *generic* deferral of expression evaluation, and I was trying (unsuccessfully) to see if your syntax for defaults could be extended to the more generic idea.[1] Elsewhere in the thread, you often ask about others' ideas for such a facility, instead of saying "that's off-topic, you have my proposal, let's keep discussion strictly to that" or alternatively, "nobody claims that's more than vaporware, I say now is better than never, nothing to see here, move on." In that sense, yes, you can treat everything in this thread as being about argument defaults by cutting short any other discussion (or just ignoring it). There's nothing wrong with doing that -- but you did not. Instead you talk about being confused, not understanding the suggested alternatives, and you ask about them. In that context, it's on you to try to channel others' thinking rather than demand that they channel your confusion. If you're not in *this* subthread to understand alternative ideas (again, *there is nothing wrong with ending this subthread here*), I have nothing further to say in it. If you are, you need to calm down and start asking questions that specify what you want to know rather than adding a question mark to a grunt as in > How? Footnotes: [1] I think that's important because elsewhere I suggested that defaults for actual arguments are a sufficiently important use case to deserve separate syntax from the generic evaluation-deferring syntax if needed. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/3764RB4XOG7ML635A4UWY2JBDTZEJX4A/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sat, Dec 11, 2021, 10:58 PM Steven D'Aprano wrote:
> On Sat, Dec 11, 2021 at 05:02:39PM -0800, Christopher Barker wrote:
> > On Sat, Dec 11, 2021 at 3:03 PM Steven D'Aprano
> wrote:
>
> > > Didn't we decide there was an existing feature for this, no need for
> > > new syntax?
>
> > Well, no. In fact, you could always put anything you wanted into an
> > annotation:
>
> Right, and we want to put a documentation string in it! :-)
>
>
> > Anyway, take a look at the docs for Annotated:
> >
> > https://docs.python.org/3/library/typing.html#typing.Annotated
> >
> > It has a number of possible unspecified uses, but fundamentally, it's
> about
> > the adding information to the type, not to the attribute -- e.g. not
> really
> > intended for docstrings.
>
> I don't think that the docs argue against this. Annotated is documented
> as the way to put arbitrary metadata into an annotation. What you do
> with it is up to the consumer of the metadata.
>
> The docs talk about this associating the metadata with the type, but
> that's because this is the *typing* module and *type* hints are still
> the primary use-case for annotations.
>
> And despite what the docs say, the metadata is actually, literally,
> associated with the variable (or its name at least). The annotation that
> gets stored in __annotations__ is **not**
>
> {T: "metadata"}
>
> as a naive reading of the docs might trick you into believing, but
>
> {"var": Annotated[T, "metadata"]}
>
>
> In any case, we can still use annotations for anything we like, and
> while typing is definitely and certainly the primary use for them, there
> will always be a way to opt-out. (Just don't run mypy!)
>
> Don't need type hints at all? Great, put your doc string as the
> annotation.
>
> μ: 'Number of steps from x0 to the start of the cycle.'
> λ: 'Length (period) of the cycle.'
> y: 'Value of f at the start of the cycle.' = None
>
> You might want to annotate the class with `no_type_check` if there is
> any chance the user might want to run mypy over your module, but that's
> entirely up to you.
>
> You do want type hints? Great, use Annotated. If there's an attribute
> that you don't want to give a type hint to, just use Any. (A small price
> to pay for using an existing feature, instead of waiting until Python
> 3.11 or 3.27 when it gets added to the core language.)
>
> μ: Annotated[int,
> 'Number of steps from x0 to the start of the cycle.']
> λ: Annotated[int, 'Length (period) of the cycle.']
> y: Annotated[Any,
> 'Value of f at the start of the cycle.'
> ] = None
>
> No new syntax required. No bare strings floating around. You can
> generate the doc strings dynamically, they don't have to be literals.
>
> lunch: Annotated[Meal, ' '.join(['spam']*SPAM_COUNT)]
>
>
> > Sure, typing.Annotated *could* be used that way, but I don't think we can
> > say that the problem is solved.
>
> I think that from a language standpoint, we absolutely can say that the
> problem is solved. We don't need new syntax.
>
> We *might* want to standardise on a decorator or other tool to extract
> those docstrings, possibly sticking them in a new __attrdoc__ dunder.
> Being a dunder, we should get the core dev's approval for this, even if
> it is a third-party tool that uses it.
>
> (I assume we don't already use __attrdoc__ for something?)
>
> If the first use of this is dataclasses, then that would be de facto
> core dev approval.
>
> So we have a tool that takes an annotation:
>
> var: Annotated[T, docstring, *args]
>
> extracts out the docstring to an __attrdoc__ dunder:
>
> # Class.__attrdoc__
> {'var': docstring}
>
> and removes the annotation from the annotation:
>
> # Class.__annotation__
> {'var': Annotated[T, *args]} # or just T if *args is empty?
>
> Dataclasses can automatically apply this tool (subject to backwards
> compatibility constraints), or we can apply it as a decorator to any
> other class we want.
>
> So we've gone from the *hard* problem of making a language change (new
> syntax, build the functionality into the interpreter) to the *easy*
> problem of writing a decorator.
>
>
> > Heck you could just as easily provide a tuple for the annotation:
> >
> > In [17]: class C:
> > ...: x: ("Doc string", int) = 123
> > ...:
>
>
> Sure, annotations can be anything you like. But external static typing
> tools may not support this, and will be confused or break. Same for
> third-party runtime tools.
>
> Best practice is to stick as closely to the standard type-hinting
> behaviour as you can, or to completely opt-out with no_type_hints.
>
>
> > The use cases for annotations are a bit up in the air at the moment --
> see
> > the ongoing discussion on python-dev. But I don't think there's any doubt
> > that any future endorsed use of them will be compatible with static
> typing
> > (if not restricted to it), so extending the use of annotations for
> > docstrings
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sat, 11 Dec 2021 at 16:30, Christopher Barker wrote:
>
> Sorry, accidentally off-list.
I did exactly the same a few days ago.
On Thu, 9 Dec 2021 at 07:49, Chris Angelico wrote:
>
> BTW, did you intend for this to be entirely off-list?
Nope, and apologies to all, but at least it's given me the opportunity
to correct a typo & do some slight reformatting. Here's it is:
On Thu, 9 Dec 2021 at 07:25, Adam Johnson wrote:
>
> On Fri, 3 Dec 2021 at 22:38, Chris Angelico wrote:
> >
> > On Sat, Dec 4, 2021 at 6:33 AM Adam Johnson wrote:
> > > The first unwelcome surprise was:
> > >
> > > >>> def func(a=>[]):
> > > ... return a
> > > ...
> > >
> > > >>> import inspect
> > > >>> inspect.signature(func).parameters['a'].default
> > > Ellipsis
> > >
> > > Here the current behaviour of returning `Ellipsis` is very unfortunate,
> > > and I think could lead to a lot of head scratching — people wondering
> > > why they are getting ellipses in their code, seemingly from nowhere.
> > > Sure, it can be noted in the official documentation that `Ellipsis` is
> > > used as the indicator of late bound defaults, but third-party resources
> > > which aim to explain the uses of `Ellipsis` would (with their current
> > > content) leave someone clueless.
> >
> > Yes. Unfortunately, since there is fundamentally no object that can be
> > valid here, this kind of thing WILL happen. So when you see Ellipsis
> > in a default, you have to do one more check to figure out whether it's
> > a late-bound default, or an actual early-bound Ellipsis...
>
> My discomfort is that any code that doesn't do that extra check will
> continue to function, but incorrectly operate under the assumption that
> `Ellipsis` was the actual intended value. I wouldn't go so far as to say
> this is outright backwards-incompatible, but perhaps
> 'backwards-misleading'.
>
> When attempting to inspect a late-bound default I'd much rather an
> exception were raised than return value that, as far as any existing
> machinery is concerned, could be valid. (More on this thought later...)
>
> > > Additionally I don't think it's too unreasonable an expectation that,
> > > for a function with no required parameters, either of the following (or
> > > something similar) should be equivalent to calling `func()`:
> > >
> > > pos_only_args, kwds = [], {}
> > > for name, param in inspect.signature(func).parameters.items():
> > > if param.default is param.empty:
> > > continue
> > > elif param.kind is param.POSITIONAL_ONLY:
> > > pos_only_args.append(param.default)
> > > else:
> > > kwds[name] = param.default
> > >
> > > func(*pos_only_args, **kwds)
> > >
> > > # or, by direct access to the dunders
> > >
> > > func(*func.__defaults__, **func.__kwdefaults__)
> >
> > The problem is that then, parameters with late-bound defaults would
> > look like mandatory parameters. The solution is another check after
> > seeing if the default is empty:
> >
> > if param.default is ... and param.extra: continue
>
> In some situations, though, late-bound defaults do essentially become
> mandatory. Picking an example you posted yourself (when demonstrating
> that not using the functions own context could be surprising):
>
> def g(x=>(a:=1), y=>a): ...
>
> In your implementation `a` is local to `g` and gets bound to `1` when no
> argument is supplied for `x` and the default is evaluated, however
> **supplying an argument for `x` leaves `a` unbound**. Therefore, unless
> `y` is also supplied, the function immediately throws an
> `UnboundLocalError` when attempting to get the default for `y`.
>
> With the current implementation it is possible to avoid this issue, but
> it's fairly ugly — especially if calculating the value for `a` has side
> effects:
>
> def g(
> x => (a:=next(it)),
> y => locals()['a'] if 'a' in locals() else next(it),
> ): ...
>
> # or, if `a` is needed within the body of `g`
>
> def g(
> x => (a:=next(it)),
> y => locals()['a'] if 'a' in locals() else (a:=next(it)),
> ): ...
>
> > > The presence of the above if statement's first branch (which was
> > > technically unnecessary, since we established for the purpose of this
> > > example all arguments of `func` are optional / have non-empty defaults)
> > > hints that perhaps `inspect.Parameter` should grow another sentinel
> > > attribute similar to `Parameter.empty` — perhaps `Parameter.late_bound`
> > > — to be set as the `default` attribute of applicable `Parameter`
> > > instances (if not also to be used as the sentinel in `__defaults__` &
> > > `__kwdefaults__`, instead of `Ellipsis`).
> >
> > Ah, I guess you didn't see .extra then. Currently the only possible
> > meanings for extra are None and a string, and neither has meaning
> > unless the default is Ellipsis; it's possible that, in the future,
> > other alternate defaults will be implemented, which is why I
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On 2021-12-12 at 17:28:23 +1300, Greg Ewing wrote: > On 11/12/21 1:22 pm, Christopher Barker wrote: > > Darn — the P and A are swapped there. > > "Argument" and "actual" both start with "A" -- does that help? Then Parameters must be Potential (before they become Actualized as Arguments)? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2T3NHAA3D5JKWTJRCEZWCWPCRVSKD6W3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: inline Python functions and methods
On 11/12/21 5:40 pm, TobiasHT wrote: > The right function to perform inlining on shall be determined at runtime and cached in the same scope as where it’s performing it’s operations from cases where the program performs large iterations or even in infinite loops and other cases that need optimization. If it's to be a run-time optimisation, you could consider dropping the inline declaration and just have the implementation decide whether it's worth inlining things, based on factors such as the size of the function and how often it's called. Then the language wouldn't have to be changed at all, and programmers wouldn't need to have foresight to decide when to declare things inline. -- Greg ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/UXKJ4JA42IGZXN6CITX2USZKJ3FGOTRH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On 11/12/21 1:22 pm, Christopher Barker wrote: Darn — the P and A are swapped there. "Argument" and "actual" both start with "A" -- does that help? -- Greg ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/6XS45CV4GJHYGMEAHKLTBKXEFFDSK3TX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
Chris Angelico writes: > So [a "deferred object" is] a lambda function that gets called the > moment you touch it in any way. I'll take that as a question, though you present it like a fact. It's a code object plus "other stuff" that gets called automatically when dereferencing a name in most contexts (like a descriptor but works with "plain" identifiers rather than attribute names). Other cases I'm not sure about, and I don't know whether they can be resolved in a way *generally* useful enough to make them worth adding. For example, let a and b[0] both refer to a "deferred object". 1. The statement x = a will evaluate the contained expression. 2. I don't know when x = b[0] will evaluate the contained expression. 3. I don't know when f(a) will evaluate the contained expression. 4. There will be APIs such as "isdeferred(x)" and "unevalled x" that do not evaluate the contained expression. I think my answers are "2. before binding x" and "3. as in PEP 671", but I'm somewhat unsure of both, since I don't know David's and Eric's use cases. (FWIW, the alternatives I had in mind were "2. when x is dereferenced" and "3. when a is dereferenced in the body of f". There may be others.) The "other stuff" mentioned above similarly depends on use cases that I don't know. I suspect that "use cases I don't know" are characteristic of everyone's reluctance to do a "generic deferred evaluation" PEP. I get the feeling there are a number of them. > > > (Another theoretical difference is that a deferred expression is > > > parsed in the context of its *usage* rather than its *definition*, but > > > that would break all manner of things in Python and is quite > > > impractical.) > > > > I'm a little confused by "theoretical" and "parsed". > > What I mean is that I don't know whether you intend it one way or the > other, so I don't know whether it's an actual difference in your > proposal, or something that could in theory be. Do you not realize that "you" is plural, and "proposal" is not unique at this point? In particular, I was asking about you channelling Eric. I was not making any statement about my own proposal (which is starting to gel but until a day or so ago was 100% nonexistent), or Eric's for that matter. I still don't understand why time of parsing matters. Do you mean time of compilation? > If name lookups in these temporary expressions have to refer to > names in the target function, not in their current context, it > causes all kinds of problems. That's what Ruby blocks do, and Rubyists love them. They certainly don't think blocks have huge problems. Alternatively, it would certainly be possible to do as Common Lisp does, and provide for optional closure over creation-time arguments as well as access to namespaces active at evaluation time via explicit APIs. Sure, all this would be complex, and maybe "too complicated to be a good idea". On the other hand, there may be other use cases that can take advantage of evaluation of a simple reference to z after z = defer foo(*args) besides the case of late binding of function arguments. If those use cases are sufficiently compatible and compelling, it may be possible to define the implicit APIs and add explicit APIs as needed in the future. > Is that your intention? Otherwise, what is x there? I don't understand why you are pushing so hard for these details, when I doubt any of the advocates of a more general deferred evaluation facility have communicated with each other yet, and the details probably vary substantially. And even David has acknowledged that he doesn't have a worked-out proposal at the moment, only a wisp of a threat to write one. It's reasonable to say, as you have, "my proposal is the only one on the table." That's true (and acknowledged by all who prefer to wait for a generic deferred evaluation proposal), but existence of a proposal is a necessary condition for adoption, not sufficient. If nonexistence of a competing proposal is not the point you're trying to make, I'm not sure what you're after in this subthread. Please don't waste your time answering any implicit questions in this post. If you find them interesting, go right ahead, but I'm not asking for answers from you, I'm trying to provide answers to your questions as I understand them. (David, Eric, and any lurkers may have different answers, though!) Regards, Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TQVLHXENCFAMQCCBLC2KIUFUPU5PMEGU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sat, Dec 11, 2021 at 05:02:39PM -0800, Christopher Barker wrote:
> On Sat, Dec 11, 2021 at 3:03 PM Steven D'Aprano wrote:
> > Didn't we decide there was an existing feature for this, no need for
> > new syntax?
> Well, no. In fact, you could always put anything you wanted into an
> annotation:
Right, and we want to put a documentation string in it! :-)
> Anyway, take a look at the docs for Annotated:
>
> https://docs.python.org/3/library/typing.html#typing.Annotated
>
> It has a number of possible unspecified uses, but fundamentally, it's about
> the adding information to the type, not to the attribute -- e.g. not really
> intended for docstrings.
I don't think that the docs argue against this. Annotated is documented
as the way to put arbitrary metadata into an annotation. What you do
with it is up to the consumer of the metadata.
The docs talk about this associating the metadata with the type, but
that's because this is the *typing* module and *type* hints are still
the primary use-case for annotations.
And despite what the docs say, the metadata is actually, literally,
associated with the variable (or its name at least). The annotation that
gets stored in __annotations__ is **not**
{T: "metadata"}
as a naive reading of the docs might trick you into believing, but
{"var": Annotated[T, "metadata"]}
In any case, we can still use annotations for anything we like, and
while typing is definitely and certainly the primary use for them, there
will always be a way to opt-out. (Just don't run mypy!)
Don't need type hints at all? Great, put your doc string as the
annotation.
μ: 'Number of steps from x0 to the start of the cycle.'
λ: 'Length (period) of the cycle.'
y: 'Value of f at the start of the cycle.' = None
You might want to annotate the class with `no_type_check` if there is
any chance the user might want to run mypy over your module, but that's
entirely up to you.
You do want type hints? Great, use Annotated. If there's an attribute
that you don't want to give a type hint to, just use Any. (A small price
to pay for using an existing feature, instead of waiting until Python
3.11 or 3.27 when it gets added to the core language.)
μ: Annotated[int,
'Number of steps from x0 to the start of the cycle.']
λ: Annotated[int, 'Length (period) of the cycle.']
y: Annotated[Any,
'Value of f at the start of the cycle.'
] = None
No new syntax required. No bare strings floating around. You can
generate the doc strings dynamically, they don't have to be literals.
lunch: Annotated[Meal, ' '.join(['spam']*SPAM_COUNT)]
> Sure, typing.Annotated *could* be used that way, but I don't think we can
> say that the problem is solved.
I think that from a language standpoint, we absolutely can say that the
problem is solved. We don't need new syntax.
We *might* want to standardise on a decorator or other tool to extract
those docstrings, possibly sticking them in a new __attrdoc__ dunder.
Being a dunder, we should get the core dev's approval for this, even if
it is a third-party tool that uses it.
(I assume we don't already use __attrdoc__ for something?)
If the first use of this is dataclasses, then that would be de facto
core dev approval.
So we have a tool that takes an annotation:
var: Annotated[T, docstring, *args]
extracts out the docstring to an __attrdoc__ dunder:
# Class.__attrdoc__
{'var': docstring}
and removes the annotation from the annotation:
# Class.__annotation__
{'var': Annotated[T, *args]} # or just T if *args is empty?
Dataclasses can automatically apply this tool (subject to backwards
compatibility constraints), or we can apply it as a decorator to any
other class we want.
So we've gone from the *hard* problem of making a language change (new
syntax, build the functionality into the interpreter) to the *easy*
problem of writing a decorator.
> Heck you could just as easily provide a tuple for the annotation:
>
> In [17]: class C:
> ...: x: ("Doc string", int) = 123
> ...:
Sure, annotations can be anything you like. But external static typing
tools may not support this, and will be confused or break. Same for
third-party runtime tools.
Best practice is to stick as closely to the standard type-hinting
behaviour as you can, or to completely opt-out with no_type_hints.
> The use cases for annotations are a bit up in the air at the moment -- see
> the ongoing discussion on python-dev. But I don't think there's any doubt
> that any future endorsed use of them will be compatible with static typing
> (if not restricted to it), so extending the use of annotations for
> docstrings would take a bit of work to define and accept that use.
Annotated is explicitly documented as *arbitrary* metadata that is
entirely up to the application to interpret. There's no reason to think
that will change.
> Perhaps a better way to do that than
[Python-ideas] Re: Make it easier to set function globals
On Sun, Dec 12, 2021 at 1:23 PM Steven D'Aprano wrote: > > TL;DR: it is hard, but not impossible, to set function `__globals__` > dunder to a ChainMap. Let's make it easier! > > Attempt 1: set __globals__ directly. > > >>> func.__globals__ = ns > Traceback (most recent call last): > File "", line 1, in > AttributeError: readonly attribute > > I think we should have: > > - allow func.__globals__ to be writable; CodeType has a replace() method that returns a new code object with identical attributes but for the ones you asked to change. If making __globals__ writable causes problems, a FunctionType.replace() method would probably be a good alternative. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EQ5LEXW7IU24UVDJ3XHC4SEXYCZFK5CM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Make it easier to set function globals
TL;DR: it is hard, but not impossible, to set function `__globals__`
dunder to a ChainMap. Let's make it easier!
Background:
(1) Comprehensions skip class scope, leading to bug reports like these:
https://bugs.python.org/issue3692
https://bugs.python.org/issue26951
Note Guido's comment in the first:
"perhaps __globals__ could be set to a chainmap referencing the class
dict and the globals?"
(2) I want to experiment with namespaces, similar to the C++ feature.
The details don't matter, but one way I can do that experiment is to use
a class and ChainMap.
So let's try an experiment: set up a namespace using ChainMap, and try
to monkey-patch a function to use it instead of the regular globals.
Here is my setup code:
def func():
return (a, b, c)
a = b = c = "global"
from collections import ChainMap
ns = ChainMap({'a': 'CM-0'}, {'b': 'CM-1'}, globals())
Let's see what it takes to set func.__globals__ to ns.
Attempt 1: set __globals__ directly.
>>> func.__globals__ = ns
Traceback (most recent call last):
File "", line 1, in
AttributeError: readonly attribute
Attempt 2: use the FunctionType constructor.
>>> from types import FunctionType
>>> f = FunctionType(func.__code__, globals=ns)
Traceback (most recent call last):
File "", line 1, in
TypeError: function() argument 'globals' must be dict, not ChainMap
Attempt 3: fool the constructor.
>>> class MyChainMap(ChainMap, dict):
... pass
...
>>> ns = MyChainMap({'a': 'CM-0'}, {'b': 'CM-1'}, globals())
>>> f = FunctionType(func.__code__, globals=ns)
Success!
And now the function behaves as desired:
>>> f()
('CM-0', 'CM-1', 'global')
This was too hard! What can we do to make it easier?
I think we should have:
- allow func.__globals__ to be writable;
- allow anything that inherits from the Mapping ABC.
Thoughts?
--
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/5AUFO6NMDR5XABYCEU2LKXVDBQDXORUT/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sat, 2021-12-11 at 17:02 -0800, Christopher Barker wrote: > It [Annotated] has a number of possible unspecified uses, but > fundamentally, it's about the adding information to the type, not to > the attribute -- e.g. not really intended for docstrings. Ah, good point. I've conflated the two because dataclass attributes are used to define the schema of the dataclass. For general purpose attribute docstrings (module, class), I agree Annotated is probably inappropriate. > Perhaps a better way to do that than to use Annotated would be to > introduce new "thing", maybe in teh typoing module, like > "Documented": I think this will suffer from the same problem as Annotated: it would document the type, not the attribute itself. > Anyway, what I'm getting at is that it needs to be determined whare > to put the docstrings before we can get very far with this idea. How is Sphinx is pulling the string below the attribute? Whatever mechanism is to be used, I propose it must allow for runtime introspection. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LG7NQUZ67XTVXUAEXRTNHTXFXIH4NDYC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sat, Dec 11, 2021 at 3:03 PM Steven D'Aprano wrote:
> On Sat, Dec 11, 2021 at 10:07:50AM -0800, Christopher Barker wrote:
> > Where/how should class attribute doc strings be stored?
> >
> > Tacked on to the class __doc__ ?
> > Another dict?
> > __attr_doc__
> > Added to __annotaions__ ?
> > Something else?
>
> Didn't we decide there was an existing feature for this, no need for
> new syntax?
>
> >>> from typing import Annotated
> >>> class C:
> ... x: Annotated[int, "Doc string"] = 123
> ...
> >>> C.x
> 123
> >>> C.__annotations__
> {'x': typing.Annotated[int, 'Doc string']}
>
Well, no. In fact, you could always put anything you wanted into an
annotation:
In [10]: class C2:
...: x: "a docstring" = 123
...:
In [11]: inspect.get_annotations(C2)
Out[11]: {'x': 'a docstring'}
So the typing module contains a nifty thing they've called "Annotatated",
and, I imagine type checkers (like MyPy) do something sensible with them,
but there's nothing else going on here. Actually yes, there is, the
typing,get_type_hints function strips out the annotation:
In [13]: class C:
...: x: Annotated[int, "Doc string"] = 123
...:
In [14]: typing.get_type_hints(C)
Out[14]: {'x': int}
Anyway, take a look at the docs for Annotated:
https://docs.python.org/3/library/typing.html#typing.Annotated
It has a number of possible unspecified uses, but fundamentally, it's about
the adding information to the type, not to the attribute -- e.g. not really
intended for docstrings.
Sure, typing.Annotated *could* be used that way, but I don't think we can
say that the problem is solved.
Heck you could just as easily provide a tuple for the annotation:
In [17]: class C:
...: x: ("Doc string", int) = 123
...:
In [18]: inspect.get_annotations(C)
Out[18]: {'x': ('Doc string', int)}
The use cases for annotations are a bit up in the air at the moment -- see
the ongoing discussion on python-dev. But I don't think there's any doubt
that any future endorsed use of them will be compatible with static typing
(if not restricted to it), so extending the use of annotations for
docstrings would take a bit of work to define and accept that use.
Perhaps a better way to do that than to use Annotated would be to introduce
new "thing", maybe in teh typoing module, like "Documented":
class C:
x: Documented(doc_string="this is the doc string", type=int) = 123
and then inspect.get_annotations and typing,get_type_hints could be taught
to extract the type from the Documented object, and we could add an
inspect.get_docstrings() function, so that:
typing.get_type_hints(C)
{'x': int}
inspect.get_docstrings(C)
{'x': 'this is the doc string'}
And, of course, future syntax could automatically create a Documented()
object if there was a docstring in the appropriate place.
Or just have a new dunder for it, e.g. __attr_doc__
Anyway, what I'm getting at is that it needs to be determined whare to put
the docstrings before we can get very far with this idea.
-CHB
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UQV3GZLVHDCHXKN4L2QCIHYLFTWBFEW3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Christopher Barker, PhD (Chris)
Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/LEFJD7BVN43E4EUNZOKTT2M7YOYU262R/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sat, Dec 11, 2021 at 10:49 AM Ricky Teachey wrote: > The __annotations__ already exists. Is that a point in favor? > Yes and no. Right now, any object can be stored in annotations — having a docstring tacked on would break who knows how much code. However, there is the new inspect.get_annotations function, which could be adapted to process doc strings. https://docs.python.org/3/howto/annotations.html If the syntax could become sugar for creating an Annotated object in > __annotations__, this would be a pretty convenient location to find them. > > On the other hand, not every type hinted variable will have a docstring. > And not every attribute with a docstring would have an annotation. It seems like storing them in a so called __attr_doc__ might be the most > straightforward thing to do. > Yes, as doc strings really are a different thing. Though one might to also document other things like function parameters— which would be an argument for extending what __annotations__ is used for. If that were done, does it matter that the contents of __attr_doc__ and > __annotations__ are not coupled? > I don’t think so — they are quite different concepts. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OTIC243UANQAVFSI3JAHQZATFRDKT7KJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
I think that because Sphinx could interpret strings below attributes
for documentation, that perhaps we should go in that direction in
Python proper.
Personally, I find this more readable:
class C:
x: Annotated[str, "Doc string"]
y: Annotated[int, "Doc string"]
over:
class C:
x: str
"Doc string"
y: int
"Doc string"
On Sun, 2021-12-12 at 10:00 +1100, Steven D'Aprano wrote:
> On Sat, Dec 11, 2021 at 10:07:50AM -0800, Christopher Barker wrote:
>
> > Where/how should class attribute doc strings be stored?
> >
> > Tacked on to the class __doc__ ?
> > Another dict?
> > __attr_doc__
> > Added to __annotaions__ ?
> > Something else?
>
> Didn't we decide there was an existing feature for this, no need for
> new syntax?
>
> > > > from typing import Annotated
> > > > class C:
> ... x: Annotated[int, "Doc string"] = 123
> ...
> > > > C.x
> 123
> > > > C.__annotations__
> {'x': typing.Annotated[int, 'Doc string']}
>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/MEFYE3FPMI7L4QPO4PRJ424IJ3HPXFEN/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sat, Dec 11, 2021 at 10:07:50AM -0800, Christopher Barker wrote:
> Where/how should class attribute doc strings be stored?
>
> Tacked on to the class __doc__ ?
> Another dict?
> __attr_doc__
> Added to __annotaions__ ?
> Something else?
Didn't we decide there was an existing feature for this, no need for
new syntax?
>>> from typing import Annotated
>>> class C:
... x: Annotated[int, "Doc string"] = 123
...
>>> C.x
123
>>> C.__annotations__
{'x': typing.Annotated[int, 'Doc string']}
--
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/UQV3GZLVHDCHXKN4L2QCIHYLFTWBFEW3/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sat, Dec 11, 2021, 1:19 PM Christopher Barker wrote: > > > Where/how should class attribute doc strings be stored? > > Tacked on to the class __doc__ ? > > Another dict? > > __attr_doc__ > > Added to __annotaions__ ? > > Something else? > > If they are to be available at run time, they need to go somewhere… > > -CHB > >> >> -- > Christopher Barker, PhD (Chris) > The __annotations__ already exists. Is that a point in favor? If the syntax could become sugar for creating an Annotated object in __annotations__, this would be a pretty convenient location to find them. On the other hand, not every type hinted variable will have a docstring. It might be inconvenient to have them there only. It seems like storing them in a so called __attr_doc__ might be the most straightforward thing to do. If that were done, does it matter that the contents of __attr_doc__ and __annotations__ are not coupled? The strings might need to be modified. Would be inconvenient to have to modify two places in such a situation. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7S4RM4APDP4FC7L6DBHJU7GAHES3ABPV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Sat, Dec 11, 2021, 12:57 AM Stephen J. Turnbull < stephenjturnb...@gmail.com> wrote: Simão Afonso writes: > On 2021-12-10 12:20:44, Ricky Teachey wrote: > > I meant to ask about a (global) module member, not the module docstring > > itself. Like MY_GLOBAL below: > > > > """This is the module docs""" > > > > MY_GLOBAL = None > > """MY_GLOBAL docs""" > > > > Is this "global docstring" recognized by Sphinx as a docstring, too? > > My bad. > > Double checked and that's right, it is recognised as such. To my mind Sphinx is sufficiently widely used that this settles the "above or below" question. Steve I agree. Even with this convention one could still accomplish separating the documentation section from the definition section, like: class Steel: """A simple material model of steel.""" # parameters E_s: float "the linear elastic modulus" nu: float "the poisson's ratio" gamma: float "the density" # values E_s = 29e6 # ksi nu = 0.3 gamma = 490 # lbs per cu ft As has already been observed, we already have something LIKE an attribute docstring, which is an Annotated type hint object. Should the bare docstring below the member become sugar for creating an Annotated type hint? x = 1 "spam" Would then be equivalent to: x: typing.Annotated[typing.Any, "spam"] This seems really nice to me. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LO7KDBDFKKDI5XJJ7GE227PBTTM3D7NN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2
On Fri, Dec 10, 2021 at 9:57 PM Stephen J. Turnbull < stephenjturnb...@gmail.com> wrote: > To my mind Sphinx is sufficiently widely used And the system used for the official Python docs. that this settles the > "above or below" question. So yes :-) However, what I haven’t seen in this thread is discussion of what I think is the key question: Where/how should class attribute doc strings be stored? Tacked on to the class __doc__ ? Another dict? __attr_doc__ Added to __annotaions__ ? Something else? If they are to be available at run time, they need to go somewhere… -CHB > > -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RWTU2EJ2TIC62IU2WZZUPO3CEPLRLTRY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On 12/11/2021 12:11 PM, Chris Angelico wrote: On Sun, Dec 12, 2021 at 4:06 AM Christopher Barker wrote: Here’s a new (not well thought out) idea: @dataclasses.dataclass class A: Input_list: list length: int => len(input_list) So length gets set to a “late bound default expression” that Is an actual value. It would have to store the expression itself, probably as a string, and it would get evaluated in the namespace of the function, when the function was called. So the generated __init__ would be the same as: def __init__(self, input_list, length=>len(input_list): If I'm not mistaken, dataclasses generate __init__ using exec anyway, so this would work with a plain string. The current version does, but I've produced previous versions that don't. They built up the AST themselves and use eval() on that. Eric ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FPKXOT6T4H36YC5QIKSJXEASIHTM7IES/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sun, Dec 12, 2021 at 4:06 AM Christopher Barker wrote: > Here’s a new (not well thought out) idea: > > @dataclasses.dataclass > class A: > Input_list: list > length: int => len(input_list) > > So length gets set to a “late bound default expression” that Is an actual > value. It would have to store the expression itself, probably as a string, > and it would get evaluated in the namespace of the function, when the > function was called. So the generated __init__ would be the same as: > > def __init__(self, input_list, length=>len(input_list): If I'm not mistaken, dataclasses generate __init__ using exec anyway, so this would work with a plain string. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/26KTI6DXSB5GJBXEHHFXZEW3222NVWWK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Fri, Dec 10, 2021 at 7:54 PM Eric V. Smith wrote: > Not sure if you meant this to go to the list or not. > I did — ( I know it seems to be the consensus, but I really think respond to list should be the default…) Moving on… On 12/10/2021 7:50 PM, Christopher Barker wrote: > > Thanks Eric, this is a great example, thanks. > > It does raise some questions. Though. > > If we could use any expression as a deferred expression, then we still > have the two key questions: > > When does it get evaluated, and what namespaces does it use? > > I agree those are good questions. I think, like PEP 671, it would get > evaluated at the start of the function (in this case, __init__). It's > easier for dataclasses, because I could just force the evaluation there. > But for normal function arguments, maybe we'd have to say that before the > function starts executing, any arguments which are deferred objects > automatically are evaluated. And I think the namespace would be where it's > defined. > Ahh, then that would make them less useful for default parameters— even for the dataclass case. It would work fine if you used only builtins (or literals) but that’s about it. So Chris’ canonical example of n => len(input_list) wouldn’t work. Which is why I think that if there were a general purpose deferred object, we’d need special syntax or rules for when used as late-bound defaults. I'll admit I haven't thought all of this through in enough detail to > implement it. I'm just trying to point out that we could use the general > concept in other places. > Which is exactly what I asked for — thanks! I think this is a really interesting use case, not so much as it’s a general purpose deferred object, but because it’s about how to do/use late-bound defaults in meta programming. > def fun(n): > return `len(n)` > > @dataclasses.dataclass > class A: > length: n = fun() > > What would that put in the signature? What namespace would the express e > evaluated in? > > I think it should be evaluated in the context of "fun". Clearly it would > need to create a closure. > Which would be a potentially useful deferred object, but not a good one for late-bound defaults. Here’s a new (not well thought out) idea: @dataclasses.dataclass class A: Input_list: list length: int => len(input_list) So length gets set to a “late bound default expression” that Is an actual value. It would have to store the expression itself, probably as a string, and it would get evaluated in the namespace of the function, when the function was called. So the generated __init__ would be the same as: def __init__(self, input_list, length=>len(input_list): And you could also do: get_length => len(input_list) @dataclasses.dataclass class A: Input_list: list length: int = get_length Not that that would be a good idea. This suggestion would mean that “=>” would create a deferred expression, but it would not be a general purpose one. Though maybe there could be a way to evaluate it in a more general way, kind of like eval — where you can control the namespaces used. I guess what I’m suggesting is that we could create a very specific kind of deferred object, and in the future expand it to more general use. Rather than the general case rendering the late-bound default concept. -CHB > -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7PYISXVUZMK2A7VUPNYU4Q2QD6JJEEUC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python standard library TOML module
The line of reasoning for the packaging tooling choosing TOML is elaborated upon in PEP 518 (https://www.python.org/dev/peps/pep-0518/#other-file-formats) and that choice was informed by a survey of the existing formats: https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SDAJ4DCK7GJIMEONNYJ3NNXKJYGQWJDF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
And my response again, although slightly edited. On 12/11/2021 11:28 AM, Christopher Barker wrote: Sorry, accidentally off-list. Here it is again. On Fri, Dec 10, 2021 at 4:50 PM Christopher Barker wrote: Thanks Eric, this is a great example, thanks. It does raise some questions. Though. If we could use any expression as a deferred expression, then we still have the two key questions: When does it get evaluated, and what namespaces does it use? I agree those are good questions. I think, like PEP 671, it would get evaluated at the start of the function (in this case, __init__). It's easier for dataclasses, because I could just force the evaluation there. But for normal function arguments, maybe we'd have to say that before the function starts executing, any arguments which are deferred objects automatically are evaluated. And I think the namespace would be where it's defined. I'll admit I haven't thought all of this through in enough detail to implement it. I'm just trying to point out that we could use the general concept in other places. One thing in particular I haven't thought through: what if you really want to pass in a "deferred object" to a function? Can you keep it from being evaluated? Would you specify that on the caller side, or the callee side? @dataclasses.dataclass class A: my_list: list = dataclasses.field(default_factory=list) What I'd like to be able to say: @dataclasses.dataclass class A: my_list: list = `[]` I think in the data classes case, you could clearly define both of those. But in the general case? def fun(n): return `len(n)` @dataclasses.dataclass class A: length: n = fun() What would that put in the signature? What namespace would the express e evaluated in? I think it should be evaluated in the context of "fun". Clearly it would need to create a closure. Yes, that is completely contrived, but it does bring up the complications of a “general” solution. Maybe we could solve the dataclass problem with late bound class attributes: @dataclasses.dataclass class A: length: n => len(n) Though I suppose that would still get evaluated before the dataclass decorator would see it :-( There are absolutely some issues that need thinking through. It's entirely possible the idea can't be made to work. I'm just saying we should think it through and see if it could be made to work before we just say "we only want late-bound things as function parameters, and we can't use them anywhere else". I'm worried that if we only do it for function parameters, we might need to change the syntax (and I guess possibly the semantics) in the future in order to support other uses for deferred objects. Side note: I wonder if dataclasses could be a bit smarter for the common. case: if the type is a callable, and the value is a particular Sentinel, the. Call it to get the default: @dataclasses.dataclass class A: my_list: list = dataclasses.Empty That would work for most of the cases where I need to use field explicitly. That's an interesting idea. Eric - CHB In the class A, before @dataclass is called, I want A.my_list to be a "deferred object" that I could then use when @dataclass is generating __init__(). Exactly how and when the "deferred object" would get evaluated is debatable, but not so important for the sake of this discussion. Suffice it to say that it would either be explicitly or implicitly evaluated at the start of __init__. I think you can see that this would benefit from similar functionality to late-bound parameters, and that if we had this more general mechanism that late-bound parameters could use the same underlying mechanism. And in case I wasn't clear: to get the late-bound parameter functionality using this syntax, you'd use: def foo(my_list = `[]`): That's why I think we should have a larger concept that just late-bound parameters: I think there's a general concept here that can be extended beyond parameters. And that's why I thing not restricting it to a function-definition-only syntax is important: we should produce a syntax that can be used in more places than just functions. This is why I think we need to decide on this larger scope before accepting the narrow function-definition-only syntax: if we decide to add "deferred objects" later, we'd have two ways to specify late-bound parameters [0]. Eric [0]: Or arguments, I can never remember which is which: someone needs to invent a memorable mnemonic device. Had these "deferred objects" existed when I designed dataclasses, I would
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
Sorry, accidentally off-list. Here it is again. On Fri, Dec 10, 2021 at 4:50 PM Christopher Barker wrote: > Thanks Eric, this is a great example, thanks. > > It does raise some questions. Though. > > If we could use any expression as a deferred expression, then we still > have the two key questions: > > When does it get evaluated, and what namespaces does it use? > > @dataclasses.dataclass >> class A: >> my_list: list = dataclasses.field(default_factory=list) >> >> What I'd like to be able to say: >> >> @dataclasses.dataclass >> class A: >> my_list: list = `[]` >> >> I think in the data classes case, you could clearly define both of those. > But in the general case? > > def fun(n): > return `len(n)` > > @dataclasses.dataclass > class A: > length: n = fun() > > What would that put in the signature? What namespace would the express e > evaluated in? > > Yes, that is completely contrived, but it does bring up the complications > of a “general” solution. > > Maybe we could solve the dataclass problem with late bound class > attributes: > > @dataclasses.dataclass > class A: > length: n => len(n) > > Though I suppose that would still get evaluated before the dataclass > decorator would see it :-( > > Side note: I wonder if dataclasses could be a bit smarter for the common. > case: if the type is a callable, and the value is a particular Sentinel, > the. Call it to get the default: > > @dataclasses.dataclass > class A: > my_list: list = dataclasses.Empty > > That would work for most of the cases where I need to use field explicitly. > > - CHB > > > > > In the class A, before @dataclass is called, I want A.my_list to be a >> "deferred object" that I could then use when @dataclass is generating >> __init__(). Exactly how and when the "deferred object" would get evaluated >> is debatable, but not so important for the sake of this discussion. Suffice >> it to say that it would either be explicitly or implicitly evaluated at the >> start of __init__. >> >> I think you can see that this would benefit from similar functionality to >> late-bound parameters, and that if we had this more general mechanism that >> late-bound parameters could use the same underlying mechanism. >> >> And in case I wasn't clear: to get the late-bound parameter functionality >> using this syntax, you'd use: >> >> def foo(my_list = `[]`): >> >> That's why I think we should have a larger concept that just late-bound >> parameters: I think there's a general concept here that can be extended >> beyond parameters. And that's why I thing not restricting it to a >> function-definition-only syntax is important: we should produce a syntax >> that can be used in more places than just functions. This is why I think we >> need to decide on this larger scope before accepting the narrow >> function-definition-only syntax: if we decide to add "deferred objects" >> later, we'd have two ways to specify late-bound parameters [0]. >> >> Eric >> >> [0]: Or arguments, I can never remember which is which: someone needs to >> invent a memorable mnemonic device. >> >> Had these "deferred objects" existed when I designed dataclasses, I would >> have used them instead of the clunky default_factory. PEP 671 does not help >> with this use case, where a late-bound parameter isn't specified in a >> function definition. I need the late-bound parameter to be stored in an >> object I can refer to later. >> >> Eric >> >> ___ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/T62V6IMOJDADAFWJ2DIAS4WHJQMI4CHU/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -- > Christopher Barker, PhD (Chris) > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IDKDVY4LNNB2WLQVS6QM3OZX6C6QXE7Y/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python standard library TOML module
Yes!! This is something I've thought about proposing for a while, but I was too lazy to do it. TOML is a wonderful language with an important place in the Python ecosystem. -- Finn (Mobile) On Fri, Dec 10, 2021, 9:57 AM Sebastian Koslowski < sebastian.koslow...@gmail.com> wrote: > There is already some info/discussion over on the bug tracker: > > https://bugs.python.org/issue40059 > > Sebastian > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/ZQOD6TEOWAB47WL2IMQS2W67IAKS46NR/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YNA3NQDQEUY56M54INRQHJCYD44I4SI7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sat, Dec 11, 2021 at 8:07 PM Stephen J. Turnbull wrote: > > Chris Angelico writes: > > On Sat, Dec 11, 2021 at 5:35 PM Stephen J. Turnbull > > wrote: > > > > foo(x=>[a]) > > > I'm not sure what that last line would mean. > > This isn't about your proposal, it's about more general syntax. Not > everything being discussed is about your proposal, and I suspect one > reason you have trouble figuring out what other people are talking > about is that you persistently try to force everything into that > context. Yes, it's silly of me to think of everything in a PEP 671 thread as if it's about argument defaults. Carrying on. The reason I thought it might be about arg defaults is that you did also show an arg default in the exact same block. That kinda sullies the waters a bit. Actually, a lot. If you want to talk about deferred expressions, can you restrict it to one example rather than two? It's hard to parse, especially with all the one-letter names. > > My proposal doesn't change the call site in any way, so I'm trying > > to figure out what you mean by that call. Are you saying that > > x=>[a] would be an assignment statement that sets x to an > > unevaluated expression? > > No, I'm saying it would be an assignment expression (like :=), but > that's what it would do. But that binding would be ignored, and the > value passed into the function in the usual way. A keyword argument > would be set with the even uglier x = (x=>[a]). uhhh I'm lost. Are you saying that "x=>" is a magic token that doesn't actually assign to x, but it just means that the thing that follows it is a deferred expression? If so, why not a keyword like "defer"? > > It would also require significant changes to the way that nonlocal > > names are looked up (or would be restricted in what nonlocals it > > can refer to, possibly none at all). > > All of which should clue you in that that's probably not what I'm > talking about, especially when I explicitly wrote "while in an > executable context, it could return the object, as := does". What's an executable context though? I don't understand. > > Indeed. But here's the question: If a deferred object is to be a > > replacement for default_factory, then it must by definition be able to > > be stored for later. So it can't be autoevaluated unless there's some > > mechanism for delaying the autoevaluation. This seems like an > > incredibly messy approach. > > Such an object is not evaluated when created; it's evaluated when > referenced, like a descriptor. Descriptors are messy, too, but > very useful. How? > > What would have overlap with argument defaults isn't the same thing > > that would be useful for dataclasses. > > If you say so, but forgive me if I table your comment and wait for > Eric to weigh in on requirements related to dataclasses. In any case, > if the default is such a deferred object x, I'm pretty sure that doing > x=x before entering the function body is equivalent to your proposal. > I see no particular reason why we couldn't have that rule for > "deferred objects" created in "function signature scope". I *really* don't like that idea. If you want to propose that kind of thing, where it magically changes when you look at it, then go ahead, but that has nothing whatsoever to do with PEP 671, as it doesn't answer the same use-cases, is largely orthogonal, can be better explained without argument defaults being involved at all, and happens at the call site rather than the function signature. If I'm understanding your proposal correctly, using one of these deferreds as an argument default would look like "def f(x=x=>[]):", which, as well as being weirdly ugly, wouldn't even be incompatible with the proposal I'm making. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LK5VSKOLSEH5N6JI2IVZSCNQH53E4UJF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sat, Dec 11, 2021 at 8:07 PM Stephen J. Turnbull wrote: > > Chris Angelico writes: > > > It's larger than argument defaults, but also smaller: > > Aside: I'm quite confused by your whole line of discussion here, but > I'll let Eric follow up. > > > The overlap with late-bound defaults is the simple case of mutable > > objects that you want to freshly construct every time, but > > ultimately, that's not hugely different from a lambda function: > > Of course it's hugely different from a lambda function. It will be > evaluated at the time of reference, whereas a lambda function will > not, it won't be evaluated until called. (This means that to access a > deferred object without evaluating it, a separate API will be needed, > sort of the dual of function call.) So it's a lambda function that gets called the moment you touch it in any way. > > (Another theoretical difference is that a deferred expression is > > parsed in the context of its *usage* rather than its *definition*, but > > that would break all manner of things in Python and is quite > > impractical.) > > I'm a little confused by "theoretical" and "parsed". I guess by > "theoretical" you mean that this is a design choice, and by "parsed in > the context" you mean that the expression could be represented in the > deferred object as a string, an AST, or a code object. Please > confirm. > What I mean is that I don't know whether you intend it one way or the other, so I don't know whether it's an actual difference in your proposal, or something that could in theory be. If name lookups in these temporary expressions have to refer to names in the target function, not in their current context, it causes all kinds of problems. Is that your intention? Otherwise, what is x there? ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZBFMAOLMZKDYEH5ZIYZID4NZMVTCFCRB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
Chris Angelico writes: > On Sat, Dec 11, 2021 at 5:35 PM Stephen J. Turnbull > wrote: > > foo(x=>[a]) > I'm not sure what that last line would mean. This isn't about your proposal, it's about more general syntax. Not everything being discussed is about your proposal, and I suspect one reason you have trouble figuring out what other people are talking about is that you persistently try to force everything into that context. > My proposal doesn't change the call site in any way, so I'm trying > to figure out what you mean by that call. Are you saying that > x=>[a] would be an assignment statement that sets x to an > unevaluated expression? No, I'm saying it would be an assignment expression (like :=), but that's what it would do. But that binding would be ignored, and the value passed into the function in the usual way. A keyword argument would be set with the even uglier x = (x=>[a]). > If so, that's independent of the default argument. Of course it is, because it's a device to pass a general "deferred object" which is constructed in the actual argument list. > If it's a special way to pass keyword arguments to a > function, No. In fact in this context x is a dummy, as ugly as any crash test dummy after the crash. I probably should have used _. > It would also require significant changes to the way that nonlocal > names are looked up (or would be restricted in what nonlocals it > can refer to, possibly none at all). All of which should clue you in that that's probably not what I'm talking about, especially when I explicitly wrote "while in an executable context, it could return the object, as := does". > Indeed. But here's the question: If a deferred object is to be a > replacement for default_factory, then it must by definition be able to > be stored for later. So it can't be autoevaluated unless there's some > mechanism for delaying the autoevaluation. This seems like an > incredibly messy approach. Such an object is not evaluated when created; it's evaluated when referenced, like a descriptor. Descriptors are messy, too, but very useful. > The use-cases for deferred evaluation differ Yup. It's an object, those use-cases can "almost certainly" (IMHO, but I suspect David and Eric agree) be distinguished by attributes set on the object, or auxiliary APIs for special use-cases where you want the unevaluated object. I suspect that most folks who want "deferred objects" haven't really thought about this issue, but have focused on their immediate applications. > What would have overlap with argument defaults isn't the same thing > that would be useful for dataclasses. If you say so, but forgive me if I table your comment and wait for Eric to weigh in on requirements related to dataclasses. In any case, if the default is such a deferred object x, I'm pretty sure that doing x=x before entering the function body is equivalent to your proposal. I see no particular reason why we couldn't have that rule for "deferred objects" created in "function signature scope". Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LCMG3JZLBNY45CWH6YNVLAW2EYIOXBFD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
Chris Angelico writes: > It's larger than argument defaults, but also smaller: Aside: I'm quite confused by your whole line of discussion here, but I'll let Eric follow up. > The overlap with late-bound defaults is the simple case of mutable > objects that you want to freshly construct every time, but > ultimately, that's not hugely different from a lambda function: Of course it's hugely different from a lambda function. It will be evaluated at the time of reference, whereas a lambda function will not, it won't be evaluated until called. (This means that to access a deferred object without evaluating it, a separate API will be needed, sort of the dual of function call.) > (Another theoretical difference is that a deferred expression is > parsed in the context of its *usage* rather than its *definition*, but > that would break all manner of things in Python and is quite > impractical.) I'm a little confused by "theoretical" and "parsed". I guess by "theoretical" you mean that this is a design choice, and by "parsed in the context" you mean that the expression could be represented in the deferred object as a string, an AST, or a code object. Please confirm. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5NFYE4Z5PU7QCBDBV2IUMYLCPMGGY7TX/ Code of Conduct: http://python.org/psf/codeofconduct/
