subject:"\[Python\-ideas\] Re\: Runtime\-accessible attribute docstrings

On Tue, Dec 14, 2021 at 06:48:57PM -0800, Paul Bryan wrote:
> Interesting. Some apparent downsides:
> 
> - doesn't apply to class attributes

There is that.

> - objects with `__slots__` can't dynamically add attributes 

Just give it a `__dict__` slot:

__slots__ = {'__dict__': None}

and now it can.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YG2CHX6JBCGLGE2NXAZYEJO57T7MV65R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-14 Thread Ricky Teachey

On Tue, Dec 14, 2021, 9:49 PM Paul Bryan  wrote:

> Interesting. Some apparent downsides:
>
> - doesn't apply to class attributes
> - objects with `__slots__` can't dynamically add attributes
>

Also doesn't apply to module level members.

To my mind these are significant downsides. And it also represents yet
another way people are documenting attributes in the wild.

Here's another I learned if which is supported by sphinx autodoc:

comment tag colon docstring
attribute statement

Example:

#: spam
x = 1

Using this style, the documentation has to come before the attribute.

Obviously- unless VERY big change is introduced- this "docstring" will
never be runtime accessible since it's a comment. But it does represent yet
another way people are doing documentation in the wild.

Sigh.

Rick.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NTYRIIDYNJUUKT4PCVJEHOBFQDOQ32JP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-14 Thread Paul Bryan

Interesting. Some apparent downsides:

- doesn't apply to class attributes
- objects with `__slots__` can't dynamically add attributes 

On Wed, 2021-12-15 at 11:13 +1100, Steven D'Aprano wrote:
> Hmmm, well it seems that we already have support for class attribute 
> docstrings, since Python 3.8.
> 
> It's not documented, but soon will be.
> 
> https://bugs.python.org/issue46076
> 
> 
>    class Card:
>   """A card from a standard French deck"""
>   __slots__ = {
>   'suit': 'Either "Spades", "Hearts", "Clubs" or "Diamonds"',
>   'rank': 'A positive integer in the range 2 <= x <= 14'
>   }
> 
> 
> help() already knows to read the docstrings from the slot dict
> values.
> 
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4XBHQSDGK57UDXOCF67XRXC5Q4DDJZSL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

Hmmm, well it seems that we already have support for class attribute 
docstrings, since Python 3.8.

It's not documented, but soon will be.

https://bugs.python.org/issue46076


   class Card:
  """A card from a standard French deck"""
  __slots__ = {
  'suit': 'Either "Spades", "Hearts", "Clubs" or "Diamonds"',
  'rank': 'A positive integer in the range 2 <= x <= 14'
  }


help() already knows to read the docstrings from the slot dict values.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BI2XVQAVFMMUHY3WAEGB3FYQ6D6RHX26/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-14 Thread Christopher Barker

Hey all:

There is discussion RIGHT NOW in the SC, and on python-dev about future
"policy" around annotations, in response to the PEP 563 and 649 -- not
clear where it's going to end up, but it is clear that the whole "are
annotations only for typing" question will be made more clear.

Anyway, I had the idea that one could create a subclass of
typing.Annotated, and call it "Documented", and then have it do something a
little special, while still looking like an Annotated to the Typing system,
e.g. get_type_hints(). However:

In [2]: class Documented(typing.Annotated):
   ...: pass
   ...:
---
TypeError Traceback (most recent call last)
 in 
> 1 class Documented(typing.Annotated):
  2 pass
  3

~/miniconda3/envs/py310/lib/python3.10/typing.py in __init_subclass__(cls,
*args, **kwargs)
   1676
   1677 def __init_subclass__(cls, *args, **kwargs):
-> 1678 raise TypeError(
   1679 "Cannot subclass {}.Annotated".format(cls.__module__)
   1680 )

TypeError: Cannot subclass __main__.Annotated

So those type objects in typing are weird.

I'm giving up :-(

-CHB



On Tue, Dec 14, 2021 at 8:36 AM Ricky Teachey  wrote:

> On Tue, Dec 14, 2021 at 10:23 AM Joao S. O. Bueno 
> wrote:
>
>> Just a short one, for everyone agreeing type.Annotated does the job,
>> but thinks we need new syntax, because it is verbose:
>>
>> You can already do:
>>
>> from typing import Annotated as A
>>
>> And:
>>
>> attr: A[type, "docstring goes here"]
>>
>> I see no need for any new syntax.
>>
>> (and maybe adding typing.Docstring for the cases when one
>> just wants the docs, but no type annotation)
>>
>
> You're not wrong exactly. But there has been a desire to reduce the need
> for typing module imports in general, and I think eliminating the need to
> import Annotation for this specific use case might have merit, if is
> becomes blessed as The Way, and if we think it is a good idea to say to
> people "hey! providing docstrings for members is a really good practice,
> and so we're making is super duper easy and standardizing it!" (just as we
> encouraged people to provide docstrings in the past for modules and classes
> by making that syntax dead simple to use).
>
> ---
> Ricky.
>
> "I've never met a Kentucky man who wasn't either thinking about going home
> or actually going home." - Happy Chandler
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/H5NIEA24YOZOY6TUFIRAYV4IK5FNYQQJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SF3ZV2H6FV566IQJOUAGAWV6KAPEJT6R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-14 Thread Ricky Teachey

On Tue, Dec 14, 2021 at 10:23 AM Joao S. O. Bueno 
wrote:

> Just a short one, for everyone agreeing type.Annotated does the job,
> but thinks we need new syntax, because it is verbose:
>
> You can already do:
>
> from typing import Annotated as A
>
> And:
>
> attr: A[type, "docstring goes here"]
>
> I see no need for any new syntax.
>
> (and maybe adding typing.Docstring for the cases when one
> just wants the docs, but no type annotation)
>

You're not wrong exactly. But there has been a desire to reduce the need
for typing module imports in general, and I think eliminating the need to
import Annotation for this specific use case might have merit, if is
becomes blessed as The Way, and if we think it is a good idea to say to
people "hey! providing docstrings for members is a really good practice,
and so we're making is super duper easy and standardizing it!" (just as we
encouraged people to provide docstrings in the past for modules and classes
by making that syntax dead simple to use).

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H5NIEA24YOZOY6TUFIRAYV4IK5FNYQQJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-14 Thread Joao S. O. Bueno

Just a short one, for everyone agreeing type.Annotated does the job,
but thinks we need new syntax, because it is verbose:

You can already do:

from typing import Annotated as A

And:

attr: A[type, "docstring goes here"]

I see no need for any new syntax.

(and maybe adding typing.Docstring for the cases when one
just wants the docs, but no type annotation)

On Tue, 14 Dec 2021 at 06:19, Steven D'Aprano  wrote:

> On Tue, Dec 14, 2021 at 12:38:55AM +0900, Stephen J. Turnbull wrote:
> > Steven D'Aprano writes:
> >  > On Sun, Dec 12, 2021 at 08:44:25PM -0500, Ricky Teachey wrote:
> >
> >  > > class C:
> >  > > x: Annotated [Any, "spam"]
> >  > >
> >  > > help(C.x)
> >  >
> >  > > And it seems reasonable to try and create a way for it to work.
> >  >
> >  > By default, for arbitrary classes, no, it *shouldn't* work, because
> for
> >  > arbitrary classes we cannot treat type hints as doc strings.
> >
> > Why not?  In the sense that we establish a standard protocol for
> > attaching them, and if there isn't one there, then 'help' would ignore
> > that attribute or provide an empty docstring or whatever default
> > action seems appropriate (presumably it would at least tell us the
> > type! even if the hint is not an Annotated).
>
> Anything is possible! And if this is for your own code you can do
> anything you like.
>
> But if you want this to be a language feature, that means getting
> Steering Council approval, and while I don't speak for them, I am
> confident, well, 90% confident, that they will agree with Guido that
> *annotations are primarily for type-hinting*.
>
> I am also confident (say, 75% confident) that they will *disagree* that
> annotations are **only** for type-hinting, so there is that.
>
> But of course if anyone thinks different, and believes that the SC will
> bless "string annotations are docstrings" as a language feature, write a
> PEP and go for it!
>
> >  > At least not without buy in from the core devs, including people like
> >  > Guido who care very much about typing. And that will probably need a
> >  > PEP.
> >
> > It's true that changing help will require buy-in from the core devs,
> > as it's a built-in.  But:
> >
> >  > But to change the language
> >
> > Where's the language change?  What am I missing?  Or are you using
> > this loosely to include builtins or even the whole stdlib?
>
> Yes, it is somewhat loose, meaning the language proper, the builtins and
> stdlib.
>
> Right now, we have:
>
> class C:
> attr: expression
>
> causes the annotation to be stored in the `__annotations__` dunder. If
> all you want is to write your own metaclass or decorator to read the
> __annotations__ and extract strings and stick them into C.mydocstrings,
> you can do that right now, no questions asked.
>
> But if you want to use a dunder like __attrdocs__, say, then dunders are
> reserved for use for the language and we're supposed to ask first before
> inventing new ones. So there's that.
>
> And if you want this to new behaviour -- extracting strings from
> annotations to use as docstrings -- to be in the language (including the
> stdlib and builtins) then again we need approval. At the very least we
> need at least one core dev to review, accept and merge the PR. And I
> expect that the core devs would push it back to the typing group and
> SC for a decision.
>
> (That's what I would do if I knew how to merge PRs :-)
>
>
> > It needs one thing: a standard place to put the documentation.  I don't
> > think just stuffing a string in __metadata__ is a good idea; that will
> > be an annoyance to existing users of the __metadata__ attribute.
>
> There is no guarantee or language-wide meaning to the metadata, every
> consumer of Annotated type aliases *must* be prepared to skip metadata
> they don't know how to process.
>
> It's like consumers of email. The email headers can contain arbitrary
> metadata that you have no clue how to handle, you are required to leave
> it and skip over it.
>
> But the beauty of Annotated is that it already exists and we can start
> using it today. If anyone wishes to push for a syntax change, or a new
> convenience type in the typing module:
>
> attr: Docstring["this is a doc string"]
>
> then you can do so, but you will still have to get the change approved.
>
> And what are type checkers supposed to do with it? Now every type
> checker has to learn to try Docstring as an alias for Any. What if you
> want to declare a type as well?
>
>
> > I think it probably should be an attribute on Annotated instances, to
> > avoid tromping on existing uses of the __metadata__ attribute,
>
> Is there a syntax change? If there is, then you definitely need SC
> approval for syntax changed.
>
> If not, how does the Annotated type know which piece of metadata gets
> moved into the attribute?
>
> Annotated[Any, fe, fi, fo, fum, "docstring", eeny, meany, miny, mo]
>
> *We* can recognise that clearly item number 5 is the docstring, but how
> does

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Tue, Dec 14, 2021 at 12:38:55AM +0900, Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
>  > On Sun, Dec 12, 2021 at 08:44:25PM -0500, Ricky Teachey wrote:
> 
>  > > class C:
>  > > x: Annotated [Any, "spam"]
>  > > 
>  > > help(C.x)
>  > 
>  > > And it seems reasonable to try and create a way for it to work.
>  > 
>  > By default, for arbitrary classes, no, it *shouldn't* work, because for 
>  > arbitrary classes we cannot treat type hints as doc strings.
> 
> Why not?  In the sense that we establish a standard protocol for
> attaching them, and if there isn't one there, then 'help' would ignore
> that attribute or provide an empty docstring or whatever default
> action seems appropriate (presumably it would at least tell us the
> type! even if the hint is not an Annotated).

Anything is possible! And if this is for your own code you can do 
anything you like.

But if you want this to be a language feature, that means getting 
Steering Council approval, and while I don't speak for them, I am 
confident, well, 90% confident, that they will agree with Guido that 
*annotations are primarily for type-hinting*.

I am also confident (say, 75% confident) that they will *disagree* that 
annotations are **only** for type-hinting, so there is that.

But of course if anyone thinks different, and believes that the SC will 
bless "string annotations are docstrings" as a language feature, write a 
PEP and go for it!

>  > At least not without buy in from the core devs, including people like 
>  > Guido who care very much about typing. And that will probably need a 
>  > PEP.
> 
> It's true that changing help will require buy-in from the core devs,
> as it's a built-in.  But:
> 
>  > But to change the language
> 
> Where's the language change?  What am I missing?  Or are you using
> this loosely to include builtins or even the whole stdlib?

Yes, it is somewhat loose, meaning the language proper, the builtins and 
stdlib.

Right now, we have:

class C:
attr: expression

causes the annotation to be stored in the `__annotations__` dunder. If 
all you want is to write your own metaclass or decorator to read the 
__annotations__ and extract strings and stick them into C.mydocstrings, 
you can do that right now, no questions asked.

But if you want to use a dunder like __attrdocs__, say, then dunders are 
reserved for use for the language and we're supposed to ask first before 
inventing new ones. So there's that.

And if you want this to new behaviour -- extracting strings from 
annotations to use as docstrings -- to be in the language (including the 
stdlib and builtins) then again we need approval. At the very least we 
need at least one core dev to review, accept and merge the PR. And I 
expect that the core devs would push it back to the typing group and 
SC for a decision.

(That's what I would do if I knew how to merge PRs :-)

> It needs one thing: a standard place to put the documentation.  I don't
> think just stuffing a string in __metadata__ is a good idea; that will
> be an annoyance to existing users of the __metadata__ attribute.

There is no guarantee or language-wide meaning to the metadata, every 
consumer of Annotated type aliases *must* be prepared to skip metadata 
they don't know how to process.

It's like consumers of email. The email headers can contain arbitrary 
metadata that you have no clue how to handle, you are required to leave 
it and skip over it.

But the beauty of Annotated is that it already exists and we can start 
using it today. If anyone wishes to push for a syntax change, or a new 
convenience type in the typing module:

attr: Docstring["this is a doc string"]

then you can do so, but you will still have to get the change approved.

And what are type checkers supposed to do with it? Now every type 
checker has to learn to try Docstring as an alias for Any. What if you 
want to declare a type as well?

> I think it probably should be an attribute on Annotated instances, to
> avoid tromping on existing uses of the __metadata__ attribute,

Is there a syntax change? If there is, then you definitely need SC 
approval for syntax changed.

If not, how does the Annotated type know which piece of metadata gets 
moved into the attribute?

Annotated[Any, fe, fi, fo, fum, "docstring", eeny, meany, miny, mo]

*We* can recognise that clearly item number 5 is the docstring, but how 
does Annotated know? We need to have a set of rules in mind.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/36IKKWZOC5ONCOSFI7OQPUD6PG64WP6L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Mon, Dec 13, 2021 at 05:10:52PM -0800, Paul Bryan wrote:

> In other words, strings would be reserved to specify documentation.

We can't reserve strings to specify documentation.

(1) Strings can be used for forward references.

class Node:
payload: Any  # Arbitrary data.
next: Node  # Fails, because Node doesn't exist yet!

So we use a string for forward references:

class Node:
payload: Any  # Arbitrary data.
next: "Node"

Technically, we could ask folks to write that as

next: ForwardRef["Node"]

but I think most people will say "Not a chance".

Besides, if PEP 563 becomes standard, then all annotations will be 
strings.

(2) Annotated[...] metadata can be arbitrary objects, so we have to 
assume that somebody out there is already using strings as metadata.

This is why I think that the best we can do is an opt-in system where 
individual classes can choose to use string annotations as docs, but not 
a language default.

But you never know what the Steering Council will say unless you ask.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FIBS54PQVCTYTCIB2LPDF6U3IXUUZ2IU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-13 Thread Brendan Barnwell


On 2021-12-13 17:07, Steven D'Aprano wrote:

>As an aside, seeing what the docs say about Annotated makes me think
>that "Annotated" is a very bad name for this thing.  It confuses the
>idea of a type annotation (i.e., attached to a variable) with this
>type-incorporating-a-label, where neither the label nor the type is
>actually an annotation in the type-annotation sense (because they have
>not been attached to a variable to annotate it).

Right!



>It seems it would have
>been better to called Annotated "Tagged" or "Labeled" or some such thing
>to make it clear that when you using it you are defining a new
>special-purpose type for use in later annotations./


	Then let's revisit this issue after we have fixed this problem by 
renaming typing.Annotated to something else (or creating a new name and 
deprecating the old name but keeping it for backward compatibility).


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LK62MWDXWRR7BSZU4L2Y34JHLXARBJH2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-13 Thread Paul Bryan

If we proceed with using `Annotated`, I suggest it be the last string
in the metadata. Using your example:

spam: Annotated[
  int,
  Positive,
  GreaterThan[1],
  "Doc string goes here..."
  Union[Odd|Literal[2]],
  Prime,
  Wibble,
  Wobble,
] = 2

In other words, strings would be reserved to specify documentation.


On Tue, 2021-12-14 at 11:38 +1100, Steven D'Aprano wrote:
> Thinking more about my example here:
> 
> On Tue, Dec 14, 2021 at 01:50:45AM +1100, Steven D'Aprano wrote:
> 
> >     class MyClass:
> >     """Blah blah blah.
> > 
> >     Variable annotations of the form Annotated[T, 'string',
> > *args]
> >     always interpret the string in the second position as a 
> >     docstring. If you want to use Annotated but without a
> > docstring,
> >     put None in the second position.
> >     """
> >     spam: Annotated[int, 'Yummy meat-like product']
> >     eggs: Annotated[float, 'Goes well with spam', Domain[0, 1]]
> >     cheese: Sequence[str]  # No docstring.
> 
> perhaps a better convention might be that the docstring is the *last*
> item in the Annotated metadata, rather than the second item (after
> the 
> initial type).
> 
> The will allow people to use arbitrarily long augmented types, and 
> follow it with the docstring:
> 
>     spam: Annotated[int,
>     Positive,
>     GreaterThan[1],
>     Union[Odd|Literal[2]],
>     Prime,
>     Wibble,
>     Wobble,
>     "Doc string goes here..."
>     ]=2,
> 
> That will also help ensure that the docstring stands out visually, as
> it 
> will be last in the list, not stuck in the middle.
> 
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LKGQPLEOKJ5J5M4DWQIHLLVPXZE7ILSM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sun, Dec 12, 2021 at 07:43:16PM -0800, Brendan Barnwell wrote:

>   The question is what does this annotate:
> 
> Annotated[int, "some text here"]

Nothing. There's no annotation there.

That's an expression which returns a type alias. It is just an object. 
You can print it, put it in a list, pass it to a function, use it as a 
dict key, or garbage collect it. Or even use it to annotate a variable, 
but isolated as show, it isn't annotating anything.

An annotation is "A label associated with a variable, a class attribute 
or a function parameter or return value" and that expression above is 
not a label associated with anything.

https://docs.python.org/3/glossary.html#term-annotation

To be an annotation, it must be part of a parameter annotation, or 
variable assignment annotation:

def func(param:expression) -> expression:
variable: expression

All three of those expressions could legitimately be called an 
annotation, even if they're not types.

Here's an analogy: you know how dicts work, right? (Of course you do.) 
We agree that in the dict

{"Hello world": 42}

the string "Hello world" is the key associated with value 42. So in the 
context of that dict, "Hello world" is a key.

# helloworld.py
print("Hello world")

Is that string "Hello world" a key in a dict now? What's its value?

How about in isolation, "Hello world", with no implied context and most 
especially no dicts anywhere in sight. Is it still a key if there is no 
dict for it to be a key of?

I'm not asking whether it *could* be a key, if there was a dict. 
Obviously the answer is yes. I'm asking, what dict is "Hello world" the 
key of in the helloworld.py program above.

If we can understand that being a key describes the *role* of a string, 
in a specific context, and is not a property of the string itself, we 
should be able to understand that being an annotation is likewise 
descibing the *role* of an object, in a specific context, and is not a 
property of the object itself.

types: "Hello world" = (int, float, Annotated[int, "..."], str)

Here, "Hello world" is an annotation, and the Annotated type alias is a 
element of a tuple. Earlier the same string was the output of a program, 
and before that it was a key. But it's the same string.

Annotated[int, "some text here"]

can likewise be an annotation, or the output of a program, or a key in a 
dict, or an element of a tuple, depending on context. With no context, 
it is just a type object.

If its not part of an annotation statement or a function annotation, 
it's not an annotation.

https://docs.python.org/3/glossary.html#term-function-annotation

https://docs.python.org/3/reference/simple_stmts.html#annassign

> In other words what does the use of Annotated in itself annotate.  

In itself, nothing.

There's no annotation syntax in that expression. It's not a function 
annotation or a variable annotation. It doesn't create an 
`__annotations__` dunder. It meets none of the definitions of 
"annotation".

>   As Paul pointed out in his earlier message, you can create this 
> Annotated 
> thing without attaching it to any attribute.  It seems that Annotated 
> itself is annotated the type. In fact, there can be no debate about 
> what Annotated annotates, since the documentation even says it 
> explicitly (https://docs.python.org/3/library/typing.html#typing.Annotated):
> 
> > Specifically, a type T can be annotated with metadata x via the 
> typehint Annotated[T, x].

You are right about the first part of your sentence: there can be no 
debate about this. The doc is simply wrong, and it has confused you and 
others into thinking that Annotated[T, ...] has annotated T.

But if anyone wants to debate it, if you think the documentation is 
correct, then it is easy to prove me wrong, and I will happily admit 
that I have learned something new.

All you need to do is show me the annotation syntax `target: expression` 
in either of the following:

# A bare expression, with no context.
Annotated[int, "some text"]

# A type alias.
MyAlias = Annotated[int, "some text"]

Easy peasy. Point to the annotation syntax colon to prove that I am 
wrong, and the docs are right.

Or you can show me the `__annotated__` mapping with int as the key that 
either of those two lines create.

But if you can't do any of those things, then:

- there is no annotation syntax;

- there is no "label associated with a variable, a class attribute or a 
  function parameter or return value";

- there is no `__annotations__` dunder to hold that association;

and therefore **there is no annotation**. It's just an expression 
returning a type.

> The text specified in `Annotated[T, text]` is creating a sort of 
> augmented type

The docs and the implementation call them Type Aliases. "Augmented" is 
an excellent description, thank you for suggesting it.

So we can agree that Annotated[T, *metadata] returns a type alias, based 
on T, but augmented with the

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

Thinking more about my example here:

On Tue, Dec 14, 2021 at 01:50:45AM +1100, Steven D'Aprano wrote:

> class MyClass:
> """Blah blah blah.
> 
> Variable annotations of the form Annotated[T, 'string', *args]
> always interpret the string in the second position as a 
> docstring. If you want to use Annotated but without a docstring,
> put None in the second position.
> """
> spam: Annotated[int, 'Yummy meat-like product']
> eggs: Annotated[float, 'Goes well with spam', Domain[0, 1]]
> cheese: Sequence[str]  # No docstring.

perhaps a better convention might be that the docstring is the *last* 
item in the Annotated metadata, rather than the second item (after the 
initial type).

The will allow people to use arbitrarily long augmented types, and 
follow it with the docstring:

spam: Annotated[int,
Positive,
GreaterThan[1],
Union[Odd|Literal[2]],
Prime,
Wibble,
Wobble,
"Doc string goes here..."
]=2,

That will also help ensure that the docstring stands out visually, as it 
will be last in the list, not stuck in the middle.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5KHCYOCDKJDIM3KU2FCLB3XSXRS6IEHM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-13 Thread Christopher Barker

On Mon, Dec 13, 2021 at 6:54 AM Steven D'Aprano  wrote:

>
> Okay, I'm sorry, I understand the statement to be referring to people
> who had no intention of using type-hints. For those people,
> no_type_hints is the right solution. But for those who want type hints
> as well as docstrings, it is not.

The consideration I'm making here is that any standard use of annotations
needs to be fully compatible with type hinting. That's even spelled out in
PEP 563.

"uses for annotations incompatible with the aforementioned PEPs should be
considered deprecated."

> > __attrdoc__ has been suggested -- but THAT requires some level of
> approval
> > from the SC.
>
> I don't think that needs a PEP or SC approval, I think that's a small
> enough change that any core developer who cares can just add that dunder
> to their class and start collecting docstrings in there.
>

OK -- but that brings us back to how I started this sub-thread -- where are
the docstrings stored to be retrieved at runtime. If that's a new dunder,
great, then future enhancements making it easier to add them would be
compatible, and folks could hand-populate the dunder in any number of ways,
maybe including the use of typing.Annotated.

Quote:
>
> Ultimately, the responsibility of how to interpret the annotations
> (if at all) is the responsibility of the tool or library
> encountering the Annotated type.

yes, i read that. This trick is that that's just kicking the can down the
road. - one level up from the original state of "annotations will be
stored, and you can use them however you want". But then it was decided to
standardize the use of annotations as type hints. ANd now we are changing
how annotations work in order to better accommodate type hinting (PEP 563).

In light of the lesson there, I don't think it's a good idea to
semi-standardize one use for typing.Annotations without being more explicit
in the intent.

Until such time that the Steering Council announce that type-hints are
> no longer merely the preferred use for annotations, but the *only* use
> for annotations, and any other usage is banned, using annotations to
> capture variable docstrings is perfectly legitimate.
>

but not a good idea at this point -- at least not in a way that's
incompatible with type hinting.

> The current SC has made it quite clear that they are happy with the
> status quo, that typing is the primary but not only use for annotations.
>

Actually, that's not the least bit clear right now -- that exact point is
being mulled over as we speak.

Using docstrings in annotations would seem to be a good way to establish
> a strong non-typing use-case.
>

Then someone needs to do it NOW. The SC has literally asked for use-cases
for annotations for them to consider.

> in an Annotated type is a docstring” -- adding a typing,Documented
wouldn't

> > be a much bigger lift ( I wonder if making it a subclass of Annotated
> would
> > help).
>

I think it might be doable -- I'm going to give that a try soon.

No, we can't be that gung-ho to say that bare strings have to be

> docstrings. Since typing is the primary use-case for annotations,
> including the metadata in Annotated, it would have to be opt-in.
>

exactly -- so someone develops a tool that looks for Annotated, and tries
to extract docstrings from it. And someone else uses an Annotated with a
string in it for some completely different reason. And then a third person
runs the docstring tool on the code using Annotated for that other reason.
That just seems like asking for a mess to me.

Any type-hint can be a string: either due to PEP 563, due to forward
> references, or just because the developer on a whim choses to stringify
> their type-hints.
>

yes, though that is getting cleaned up, and is already a bit cleaner if you
use get_type_hints or get_annotations -- that's one reason they exist.

> I only mention dataclasses because that seemed to be the primary
> use-case mentioned from the start of this thread. I don't speak for
> Eric, and I haven't verified for myself that dataclasses would benefit
> from attribute docstrings.
>

and dataclasses are the one use-case of annotations in the stdlib. (I'm
pretty sure).

However, on thinking about it, this is actually NOT a good use case.

The dataclasse decorator uses analysis of the decorated class to set its
Fields:

If the class has a class attribute that has an annotation, it is a Field.

However, those class attributes are NOT necessarily class attributes in the
resulting class. I haven't checked the code carefully, but it looks like if
they have an immutable default, they are kept as class attributes, and
otherwise are not.

In [48]: @dataclass
...: class Test:
...: x: int
...: y: float = 1.0
...: l: list = field(default_factory=list)
...:

In [49]: Test.x
---
AttributeErrorTraceback (most recent call last)
 in

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-13 Thread Christopher Barker

On Mon, Dec 13, 2021 at 7:14 AM Simão Afonso <
simao.afo...@powertools-tech.com> wrote:

> What about having a docstring but no typing information? In this case
> that's impossible, no?


spam: Annotated[Any, 'Yummy meat-like product']

or

spam: Annotated[None, 'Yummy meat-like product']

-CHB
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QMK3CSNPH4QZ4GTPFARYZDDWAWZKNWYA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-13 Thread Stephen J. Turnbull

Steven D'Aprano writes:
 > On Sun, Dec 12, 2021 at 08:44:25PM -0500, Ricky Teachey wrote:

 > > class C:
 > > x: Annotated [Any, "spam"]
 > > 
 > > help(C.x)
 > 
 > > And it seems reasonable to try and create a way for it to work.
 > 
 > By default, for arbitrary classes, no, it *shouldn't* work, because for 
 > arbitrary classes we cannot treat type hints as doc strings.

Why not?  In the sense that we establish a standard protocol for
attaching them, and if there isn't one there, then 'help' would ignore
that attribute or provide an empty docstring or whatever default
action seems appropriate (presumably it would at least tell us the
type! even if the hint is not an Annotated).

 > At least not without buy in from the core devs, including people like 
 > Guido who care very much about typing. And that will probably need a 
 > PEP.

It's true that changing help will require buy-in from the core devs,
as it's a built-in.  But:

 > But to change the language

Where's the language change?  What am I missing?  Or are you using
this loosely to include builtins or even the whole stdlib?

It needs one thing: a standard place to put the documentation.  I don't
think just stuffing a string in __metadata__ is a good idea; that will
be an annoyance to existing users of the __metadata__ attribute.

I think it probably should be an attribute on Annotated instances, to
avoid tromping on existing uses of the __metadata__ attribute, while
allowing existing users of Annotated to take advantage of the
documentation function in a gradual way.  While I agree this is going
to need input from core devs that would likely delay things by months
rather than weeks, I see no reason why it wouldn't be in 3.12.  In
fact, it seems likely that this change would be small and
straightforward enough that it could be prototyped, polished, and
wrapped in time for 3.11.  I don't think it would be that hard to get
buy-in to a plausible proposal, getting Guido's attention is probably
harder. ;-)  (If he kibozes this post I'll cry and then laugh.)

 > A *lightweight* change that doesn't actually change the language
 > could be accepted without a PEP. If you submitted a PR to update
 > help() so that it was smart enough to recognise strings as the
 > second parameter of Annotated annotations and treat them as
 > docstrings, I suspect that would be allowed as just a relatively
 > small new feature.

I don't think that will fly, as it would be rude to people who are
already using Annotated for something else (especially if they've
already transitioned from direct use of annotations!)  I don't think
Python the language or the stdlib should touch the __metadata__
attribute, it should be reserved to client code.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YDMZATBBUEXFGLU3WCCGSL2DELN3JQ54/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-13 Thread Simão Afonso

On 2021-12-14 01:50:45, Steven D'Aprano wrote:
> On Sun, Dec 12, 2021 at 12:23:54PM -0800, Christopher Barker wrote:
> > And I note that Annotated flattens nested Annotated types, so having both a
> > docstring and other use of Annotated could be a bit tricky.
> 
> class MyClass:
> """Blah blah blah.
> 
> Variable annotations of the form Annotated[T, 'string', *args]
> always interpret the string in the second position as a 
> docstring. If you want to use Annotated but without a docstring,
> put None in the second position.
> """
> spam: Annotated[int, 'Yummy meat-like product']
> eggs: Annotated[float, 'Goes well with spam', Domain[0, 1]]
> cheese: Sequence[str]  # No docstring.
> 
> 
> The metadata is *always* interpreted by the tool, this is no different.

What about having a docstring but no typing information? In this case
that's impossible, no?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FTG5TDUGTYEOQCMQVD4T2WFLXS5EMNYK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sun, Dec 12, 2021 at 12:23:54PM -0800, Christopher Barker wrote:

> > the tools will follow. Runtime tools will
> > look at the dunder, static tools will look at the annotation directly.
> >
> 
> I hope not. *maybe* inspect.get_annotations() though.

Sorry, I don't understand your position here.

Putting aside *how* we write attribute docstrings, isn't the point of 
them that they are available to runtime inspection? Say, by help(). So 
why do you hope that runtime tools don't inspect the dunder storing the 
attribute docstrings?

And that static tools *don't* look at the annotation? (Or whatever 
mechanism we use.) They can't use the dunder because it doesn't exist 
yet, you haven't run the code.

So I'm confused... we go to the trouble of deciding on a mechanism to 
write attribute docstrings, we write them in the code, and you hope that 
nobody uses them???

Have I misunderstood?

> > > x: "spam"
> > >
> > > ... isn't really an option.
> >
> > Of course it is. Just decorate your class with @no_type_hints.
> >
> 
> and then your class attribute isn't type hinted. Of course you can do it,
> but it can't be the standard way to  add documentation to class attributes
> — see above.

Okay, I'm sorry, I understand the statement to be referring to people 
who had no intention of using type-hints. For those people, 
no_type_hints is the right solution. But for those who want type hints 
as well as docstrings, it is not. Sorry for the confusion.

> > All it takes is literally one stdlib class to start doing it officially,
> > and people will pay attention.
> >
> 
> Sure -- but where do the docstrings extracted from a typing.Annoated object
> go?
> 
> __attrdoc__ has been suggested -- but THAT requires some level of approval
> from the SC.

I don't think that needs a PEP or SC approval, I think that's a small 
enough change that any core developer who cares can just add that dunder 
to their class and start collecting docstrings in there.

But at some point somebody will need to ask on the Python-Dev list and 
see if anyone objects and says yes, you need a PEP.

> > As far as I am concerned, this is exactly the sort of use-case that
> > Annotated was invented for.
> 
> 
> and yet the docs for typing.Annotated don't seem to say that. So what
> concerns me is that this may break other uses of Annotated, which are
> expecting types to be stored there.

Have you read the PEP? Particularly this section:

https://www.python.org/dev/peps/pep-0593/#consuming-annotations

Quote:

Ultimately, the responsibility of how to interpret the annotations 
(if at all) is the responsibility of the tool or library 
encountering the Annotated type. A tool or library encountering an 
Annotated type can scan through the annotations to determine if they 
are of interest (e.g., using isinstance()).

Unknown annotations: When a tool or a library does not support 
annotations or encounters an unknown annotation it should just 
ignore it and treat annotated type as the underlying type.

Similarly in the docs:

https://docs.python.org/3/library/typing.html#typing.Annotated

Until such time that the Steering Council announce that type-hints are 
no longer merely the preferred use for annotations, but the *only* use 
for annotations, and any other usage is banned, using annotations to 
capture variable docstrings is perfectly legitimate.

The current SC has made it quite clear that they are happy with the 
status quo, that typing is the primary but not only use for annotations.

Using docstrings in annotations would seem to be a good way to establish 
a strong non-typing use-case.

> One problem with a docstrings is that
> they are  "just a string".  isinstance checks, as mentioned in the docs,
> are not so helpful. So we'd be establishing a new standard: “a bare string
> in an Annotated type is a docstring” -- adding a typing,Documented wouldn't
> be a much bigger lift ( I wonder if making it a subclass of Annotated would
> help).

No, we can't be that gung-ho to say that bare strings have to be 
docstrings. Since typing is the primary use-case for annotations, 
including the metadata in Annotated, it would have to be opt-in.

Any type-hint can be a string: either due to PEP 563, due to forward 
references, or just because the developer on a whim choses to stringify 
their type-hints.

Opting-in would mean documenting your class' requirements and 
assumptions about the format of annotations, and then using something 
like a decorator or metaclass to extract the docstrings.

> And I note that Annotated flattens nested Annotated types, so having both a
> docstring and other use of Annotated could be a bit tricky.

class MyClass:
"""Blah blah blah.

Variable annotations of the form Annotated[T, 'string', *args]
always interpret the string in the second position as a 
docstring. If you want to use Annotated but without a docstring,
put None in the second

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sun, Dec 12, 2021 at 05:34:33PM -0800, Paul Bryan wrote:

> 1. While I agree that assigning the `Annotated` value to `SomeType`
> makes `SomeType` a type alias, what do you call the specific instance
> of `Annotated[...]` itself?

Its a type. The specific type is a private implementation detail:

>>> type(Annotated[int, 'something'])


I just call it a type alias. If you want to be precise, it is an alias 
to int, augmented with some extra metadata.


> To date, I've been referring to it as a
> type, but that's also muddying the waters here.

No no, it is absolutely a type! _AnnotatedAlias is an instance of type 
itself:

>>> type(typing._AnnotatedAlias)

>>> isinstance(typing._AnnotatedAlias, type)
True

although Python plays some undocumented(?) shenanigans to make 
issubclass fail:

>>> issubclass(Annotated[int, 'something'], type)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: issubclass() arg 1 must be a class


> 2. I've been flyng fast and loose with the term "annotate". No help
> from PEP 593, which convolutes the situation with passages such as:

I will have a lot more to say on that in another post, replying to 
Brendan. Later.


> Getting concrete:
> 
> Coordinate = Annotated[int, "a graph coordinate", ValueRange(-100, 100)]
> ...
> @dataclass
> class Point:
>   x: Annotated[Coordinate, "the horizontal coordinate"]
>   y: Annotated[Coordinate, "the vertical coordinate"]

PEP 593 is absolutely clear on the fact that the semantics of the 
metadata are up to the consumer, and that Python does not require it to 
be a type or to be treated as a type. It describes the metadata as

"an arbitrary list of Python values"

so we're fine to shove docstrings in there.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QZFRKG6A726AGVIFM7FSHIDJIV2ISBHY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sun, Dec 12, 2021 at 08:44:25PM -0500, Ricky Teachey wrote:

> I'll go ahead and throw this out there: I was talking to a friend about
> this and he pointed out something simple but astute.
> 
> Will this be expected to work?
> 
> class C:
> x: Annotated [Any, "spam"]
> 
> help(C.x)
> 
> Obviously it shouldn't as things stand now because that's an
> AttributeError.

You can fix that by giving x a default value.

> But if I'm mucking about in the interpreter and I want to see the docstring
> of a runtime defined (but not import time) attribute, it would be kind of
> nice if it would work. And it seems reasonable to try and create a way for
> it to work.

By default, for arbitrary classes, no, it *shouldn't* work, because for 
arbitrary classes we cannot treat type hints as doc strings.

(Actually, to be clear, I say "we" but I mean the language and the 
stdlib, including help(). "We" as individuals can do anything we like.)

At least not without buy in from the core devs, including people like 
Guido who care very much about typing. And that will probably need a 
PEP.

That's the beauty of a decorator. We don't need a PEP or Steering 
Council approval to write a decorator (although we might need one to 
make it a built-in). Just do it!

But to change the language means we have to go through a big cumbersome 
process that may take months or years.

A *lightweight* change that doesn't actually change the language could 
be accepted without a PEP. If you submitted a PR to update help() so 
that it was smart enough to recognise strings as the second parameter of 
Annotated annotations and treat them as docstrings, I suspect that would 
be allowed as just a relatively small new feature.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/X6BG5L3VW6VWRARLKYFFUP4KRJUHEZVH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Brendan Barnwell

On 2021-12-12 16:22, Steven D'Aprano wrote:

On Sun, Dec 12, 2021 at 03:38:23PM -0800, Paul Bryan wrote:

OK, so it's not the type, except it kind of is.

Except it isn't annotating the type, it is annotating the attribute.

We don't annotate *types*:

int: Sequence[str]

That would be a regular variable or attribute called "int" that just
shadowed the builtin type int. We annotate *variables* (including
function parameters and class attributes).

You're still missing what people have mentioned a few times. It is
true that this annotates an attribute:

class Foo:
x: int

The question is what does this annotate:

Annotated[int, "some text here"]

In other words what does the use of Annotated in itself annotate. As
Paul pointed out in his earlier message, you can create this Annotated
thing without attaching it to any attribute. It seems that Annotated
itself is annotated the type. In fact, there can be no debate about
what Annotated annotates, since the documentation even says it
explicitly (https://docs.python.org/3/library/typing.html#typing.Annotated):

> Specifically, a type T can be annotated with metadata x via the
typehint Annotated[T, x].

So yes, Annotated IS annotating the type. The text specified in
`Annotated[T, text]` is creating a sort of augmented type, which is like
"a type that is type T but additionally means such-and-such". It is
true that that can type can itself later be used to annotate an
attribute (or variable), but at that point it will be marking the
variable as being of a type that incorporates the annotating text, not
annotating the variable itself with that text.

As such, I don't think Annotated is really a good choice for this. We
really want the text of the annotation to be associated with the
attribute itself, not with the attribute's type. Moreover, using
Annotated requires the user to specify a type, but people should be able
to specify annotations for documentation even if they're not using
typing at all and not specifying any types. (They shouldn't have to
resort to an awkward workaround like always throwing in a dummy Any type.)

As an aside, seeing what the docs say about Annotated makes me think
that "Annotated" is a very bad name for this thing. It confuses the
idea of a type annotation (i.e., attached to a variable) with this
type-incorporating-a-label, where neither the label nor the type is
actually an annotation in the type-annotation sense (because they have
not been attached to a variable to annotate it). It seems it would have
been better to called Annotated "Tagged" or "Labeled" or some such thing
to make it clear that when you using it you are defining a new
special-purpose type for use in later annotations./

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."

--author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/P5QIDQLETFHOWNJG3YBWBCAAWZUOAO2S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Ricky Teachey

I'll go ahead and throw this out there: I was talking to a friend about
this and he pointed out something simple but astute.

Will this be expected to work?

class C:
x: Annotated [Any, "spam"]

help(C.x)

Obviously it shouldn't as things stand now because that's an
AttributeError.

But if I'm mucking about in the interpreter and I want to see the docstring
of a runtime defined (but not import time) attribute, it would be kind of
nice if it would work. And it seems reasonable to try and create a way for
it to work.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7OJISJNCVKIFNA5LYRBHL7C2RLK3B5FI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Paul Bryan

On Mon, 2021-12-13 at 11:22 +1100, Steven D'Aprano wrote:
> On Sun, Dec 12, 2021 at 03:38:23PM -0800, Paul Bryan wrote:
> 
> > OK, so it's not the type, except it kind of is.
> 
> Except it isn't annotating the type, it is annotating the attribute.
> 
> We don't annotate *types*:
> 
>     int: Sequence[str]
> 
> That would be a regular variable or attribute called "int" that just 
> shadowed the builtin type int. We annotate *variables* (including 
> function parameters and class attributes).
> 
> 
> > In my other example, I used `Annotated` to create what I guess is
> > (and 
> > what you suggested) a pseudo-type? When it was annotated it had 
> > nothing to do with any attribute. To reiterate:
> > 
> > SomeType = Annotated[str, "some type"]
> 
> That's not an annotation. That's a type alias.
> 
> Annotations follow a colon, not an equals sign.

1. While I agree that assigning the `Annotated` value to `SomeType`
makes `SomeType` a type alias, what do you call the specific instance
of `Annotated[...]` itself? To date, I've been referring to it as a
type, but that's also muddying the waters here.

2. I've been flyng fast and loose with the term "annotate". No help
from PEP 593, which convolutes the situation with passages such as:

> Annotated is parameterized with a type and an arbitrary list of
> Python values that represent the annotations.

To rephrase, if I add a string "some type" to an `Annotated` instance,
it would not be documenting the (non-existent) attribute, it would
arguably be documenting the type itself. 

Getting concrete:

Coordinate = Annotated[int, "a graph coordinate", ValueRange(-100, 100)]
...
@dataclass
class Point:
  x: Annotated[Coordinate, "the horizontal coordinate"]
  y: Annotated[Coordinate, "the vertical coordinate"]

I suggest that in these uses of strings in `Annotated`, the 1st case
would serve to document the "coordinate" (pseudo-)type, while the 2nd
and 3rd cases would serve to document the attributes themselves. In
some cases, a developer may very well be satisfied with the
documentation of the (pseudo-)type and not override it when annotating
the attribute.

> > `Annotated` certainly appears to be intended to provide hints about
> > the
> > type. In PEP 593, `MaxLen` is an example of prescribing constraints
> > on
> > value. It would apply to an attribute if that attribute was
> > annotated
> > with it as a type hint.
> 
> And until some variable or attribute was actually annotated with it,
> it 
> might as well not exist.

The use of `Annotated` is to decorate an existing type to provide
additional metadata about it. The `Coordinate` type alias in the
example above can be introspected at runtime, independently of its use
in an annotation. If such a type alias were in a module, I would want
its documentation string to be displayed to provide additional context
about it, and help in determining if it's appropriate to use as an
annotation of an variable, parameter, attribute.

> 
> Annotated allows us to create new types, by associating arbitrary 
> metadata with an existing type. The meaning of that metadata is 
> completely unspecified. So we can say:
> 
>     Vec = Annotated[List[Tuple[T, T]], MaxLen(10)]
> 
> but that has no real meaning until you annotate a variable or
> attribute, 
> in which case the Vec type is annotated onto the attribute.

So, I think we agree that an `Annotated` instance results in what's
tantamount to a new type. And I hope by my examples above, we could
agree that it would be reasonable to document such a type in such a way
that its usage in an annotation could either inherit it, or override
it.

> That use-case for Annotated is independent of the proposed use-case 
> here. Sure you can use it to create new types, or type-aliases. But 
> that's not the only thing we can use it for.
> 
> The meaning of MaxLen(10) is unspecified. To a human reader, we can 
> guess that it probably means that the list can have a length of no
> more 
> than 10. Any tooling that comes across it is supposed to ignore it if
> it 
> doesn't know how to interpret it.
> 
> So for all we know, MaxLen(10) is not enforced by any tool, and it is
> purely there as documentation to the reader: "Please don't make your 
> vector longer than ten items".
> 
> We're not limited to only using Annotated to create new types. In an 
> annotation, we can associate arbitrary metadata with the annotated 
> attribute or variable. The interpretation of that metadata is still
> up 
> to the consumer. Is it a type restriction? Is it a docstring? Is it 
> something else? It's entirely up to the tooling.

I agree with the above.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MVIS6C5H2FYF2ZKMBO6FD76WEEKCW65Q/
Code of Conduct:

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sun, Dec 12, 2021 at 03:46:16PM -0800, Christopher Barker wrote:

> I know what a wink means, but I have no idea what your actual point is.

My actual point is that the annotation:

attr: int

is annotating the attribute named "attr". That's pretty basic 
stuff: if you don't think that we've annotated the attribute 
attr, then what do you think we've done?

Changing int to Annotated[int, metadata]:

attr: Annotated[int, metadata]

is still annotating attr, not T. What else could it be? The syntax for 
an annotation is (roughly):

name colon object

and that's exactly what we have. It's an annotation, and the thing being 
annotated is attr.

I know the docs say:

"a type T can be annotated with metadata x via the typehint 
Annotated[T, x]"

but that is wrong and misleading. Better wording would be

"a type T can be *associated* with metadata x via the typehint 
Annotated[T, x]"

but that leaves the question, why is this feature called "Annotated"?

To me the answer is that it clearly refers to the fact that using this 
feature, we can annotate attr with *both* T and some additional metadata 
x, without any restriction on the nature of x.

A wrinkle in this is that we can also create new types, using type 
aliasing:

PositiveIntegers = Annotated[int, MinValue(1)]

(which, by the way, was one of the motivating use-cases for this 
feature, so it is important in the big picture of typing, not so 
important for this use-case though)

and in that sense we might say that the new type, PositiveIntegers, is 
kinda-sorta made from annotating the original type, int, with the 
metadata.

Except... there is no actual annotation there. Its an equals sign, not a 
colon, so it's a name binding, which the typing system interprets as a 
type alias. If you look at int, it has no annotations attached, so 
clearly we haven't annotated int.

Which brings us back to my claim that the description is misleading and 
it should talk about *associating* the type T with the metadata, not 
annotating it.

Semantics are important. Would we be having this argument now if the 
docs didn't make the (wrong!) claim that it is the type that is being 
annotated? I don't think so. It is attr being annotated, not the type.

And the interpretation of the metadata is entirely up to the consumer of 
the annotation, which means we are completely entitled to use it for 
docstrings.

> typing.Annotated can exist outside of __annotations__ it is not an
> annotation itself, it has special kind of type object (type descriptor?
> What the heck do we call things that are used for type hints but are not
> regular  Python  instantiable types?) it’s a container for a type and other
> extra information that pertains to that type, not necessarily to the
> parameter or whatever it gets attached to.

I don't think that there is any general term for types which are only 
intended for type hints and cannot be instantiated. If such a standard 
terminology exists, I don't know it.

I agree that there should be.

> >>> A.__annotations__
> > {'attr': typing.Annotated[int, 'Doc string']}
> >
> > It's the *class* that is annotated. But note that the mapping is between
> > the attribute name to annotation, so in the sense that attributes are
> > represented by their name, it is the attribute that is annotated.
> 
> 
> The attribute got annotated the typing.Annotated object, but what did the
> string ‘Doc string’ get attached to?

Indirectly to the attribute's name "attr".

The implementation uses a special type alias for that purpose. Using a 
tuple such as (int, 'Doc string') was considered by the PEP but rejected.

Nothing in the PEP or the existing docs for Annotated precludes us from 
using the metadata as a docstring. The interpretation of the metadata 
has always been up to the consumer of the metadata.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/N4NTXD5EFI75FP7ORSX6DQMIWXQ4U5DR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Wed, Dec 08, 2021 at 11:50:55AM -, tmkehrenb...@gmail.com wrote:
> A few weeks ago, I proposed on this mailing list to write docstrings for
> class attributes like this:
> 
> @dataclass
> class A:
> x: int
> """Docstring for x."""

Perhaps I missed it, but I haven't seen anyone point out that this 
proposal has already been suggested and rejected more than twenty years 
ago:

https://www.python.org/dev/peps/pep-0224/


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4QNMP4YCDNEROTVT3HRNRZVXMWAFXVGV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sun, Dec 12, 2021 at 03:38:23PM -0800, Paul Bryan wrote:

> OK, so it's not the type, except it kind of is.

Except it isn't annotating the type, it is annotating the attribute.

We don't annotate *types*:

int: Sequence[str]

That would be a regular variable or attribute called "int" that just 
shadowed the builtin type int. We annotate *variables* (including 
function parameters and class attributes).

> In my other example, I used `Annotated` to create what I guess is (and 
> what you suggested) a pseudo-type? When it was annotated it had 
> nothing to do with any attribute. To reiterate:
> 
> SomeType = Annotated[str, "some type"]

That's not an annotation. That's a type alias.

Annotations follow a colon, not an equals sign.

> No attribute here (yet). Only when this is applied as a type hint to an
> attribute would it then apply.
> 
> class A:
>     attr: SomeType

Right. Now you have an annotation, and sure enough, we would find

{'attr': typing.Annotated[str, "some type"]}

in A.__annotations__. It is still annotating the attribute (to be 
precise, the attribute name).

> `Annotated` certainly appears to be intended to provide hints about the
> type. In PEP 593, `MaxLen` is an example of prescribing constraints on
> value. It would apply to an attribute if that attribute was annotated
> with it as a type hint.

And until some variable or attribute was actually annotated with it, it 
might as well not exist.

Annotated allows us to create new types, by associating arbitrary 
metadata with an existing type. The meaning of that metadata is 
completely unspecified. So we can say:

Vec = Annotated[List[Tuple[T, T]], MaxLen(10)]

but that has no real meaning until you annotate a variable or attribute, 
in which case the Vec type is annotated onto the attribute.

That use-case for Annotated is independent of the proposed use-case 
here. Sure you can use it to create new types, or type-aliases. But 
that's not the only thing we can use it for.

The meaning of MaxLen(10) is unspecified. To a human reader, we can 
guess that it probably means that the list can have a length of no more 
than 10. Any tooling that comes across it is supposed to ignore it if it 
doesn't know how to interpret it.

So for all we know, MaxLen(10) is not enforced by any tool, and it is 
purely there as documentation to the reader: "Please don't make your 
vector longer than ten items".

We're not limited to only using Annotated to create new types. In an 
annotation, we can associate arbitrary metadata with the annotated 
attribute or variable. The interpretation of that metadata is still up 
to the consumer. Is it a type restriction? Is it a docstring? Is it 
something else? It's entirely up to the tooling.

By the way, the proposal to just follow attributes with a string has 
already been considered and rejected:

https://www.python.org/dev/peps/pep-0224/

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6UP3ZJBNXBXZS5QCIJQIDCX35HXQUZCI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Christopher Barker

On Sun, Dec 12, 2021 at 3:00 PM Steven D'Aprano  wrote:

> > But what's being annotated, the type or the attribute?
>
> That's easy to test:
>
> >>> class A:
> ... attr: Annotated[int, "Doc string"]
> ...
> >>> int.__annotations__
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: type object 'int' has no attribute '__annotations__'
>
>
> Okay, so it's not the type. *wink*

I know what a wink means, but I have no idea what your actual point is.
It’s very hard not to get tripped up by the overloading of the word
“annotation”, but

typing.Annotated can exist outside of __annotations__ it is not an
annotation itself, it has special kind of type object (type descriptor?
What the heck do we call things that are used for type hints but are not
regular  Python  instantiable types?) it’s a container for a type and other
extra information that pertains to that type, not necessarily to the
parameter or whatever it gets attached to.

As far as I know, no one in this conversation has used typing.Annotated in
a type checking system— we really should hear from those folks.

>>> A.__annotations__
> {'attr': typing.Annotated[int, 'Doc string']}
>
> It's the *class* that is annotated. But note that the mapping is between
> the attribute name to annotation, so in the sense that attributes are
> represented by their name, it is the attribute that is annotated.

The attribute got annotated the typing.Annotated object, but what did the
string ‘Doc string’ get attached to?

As long as we are winking:

class A:
attr = “a string”

A.attr.upper() exists, so obviously the class attribute has that method.

Of course not. The attribute has a value that is a string, and that has the
upper attribute.

Similarly, in your example, attr has an annotation, and the value of that
annotation has a string attached to it.

I’m not saying that a convention couldn’t be established to store
docstrings in annotations via the typing.Annotated type. But I’m not sure
that it wouldn’t get in the way of other uses, and it’s really not that
obvious or convenient either.

-CHB

>
> As it obviously has to be. After all, didn't we *literally* annotate the
> attribute?
>
> attr: annotation
>
> Don't be fooled by the use of the Annotated pseudo-type as the
> annotation. We're still annotating the attribute, just like the code
> shows, regardless of what type we annotate it as.
>
> If we had written:
>
> attr: Sequence[int]
>
> there would be no questions about what's being annotated, is it the type
> int or the attribute.
>
> It cannot be the type, because many different objects with many
> different annotations may share the same type.
>
> Unlike modules, classes and functions, the annotation cannot be on the
> attribute's *value*, because the attribute may not even have a value at
> this point. And even if it does, it might be immutable, or like the
> type, many different attributes, with different annotations, may be
> sharing the same value.
>
> This rules out putting the docstring on the attribute directly.
>
> The syntax shows us annotating the attribute, and that's exactly what
> got annotated.
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UBGM3KTTH3A7BJXOQCZAAGKNWJ6LBRN4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4QOJ3M7255XSO4L3YW7PU5EDZ5OJP33Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Paul Bryan

On Mon, 2021-12-13 at 09:57 +1100, Steven D'Aprano wrote:
> On Sun, Dec 12, 2021 at 12:48:36PM -0800, Paul Bryan wrote:
> 
> > But what's being annotated, the type or the attribute?
> 
> That's easy to test:
> 
> 
> > > > class A:
> ... attr: Annotated[int, "Doc string"]
> ... 
> > > > int.__annotations__
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: type object 'int' has no attribute '__annotations__'
> 
> 
> Okay, so it's not the type. *wink*

OK, so it's not the type, except it kind of is. In my other example, I
used `Annotated` to create what I guess is (and what you suggested) a
pseudo-type? When it was annotated it had nothing to do with any
attribute. To reiterate:

SomeType = Annotated[str, "some type"]

No attribute here (yet). Only when this is applied as a type hint to an
attribute would it then apply.

class A:
    attr: SomeType

`Annotated` certainly appears to be intended to provide hints about the
type. In PEP 593, `MaxLen` is an example of prescribing constraints on
value. It would apply to an attribute if that attribute was annotated
with it as a type hint.

By the way, to see real code implementing this convention see:
https://github.com/fondat/fondat-core/blob/main/fondat/validation.py#L41

Here's another place that defines annotations that can be used to
enhance API documentation (e.g. OpenAPI):
https://github.com/fondat/fondat-core/blob/main/fondat/annotation.py

(Disclosure: this is code I wrote.)

Being able to define a new type like this using Annotated has proven to
be very powerful. You can apply constraints and 

> It's the *class* that is annotated. But note that the mapping is
> between 
> the attribute name to annotation, so in the sense that attributes are
> represented by their name, it is the attribute that is annotated.

It's unfortunate that the name `Annotated` was selected, because it's
definitely a point that can result in confusion. An attribute can have
an annotation (i.e. has a type hint); however, is `Annotated`
annotating the attribute? It's an object that contains multiple values,
which can be applied to an attribute as an annotation (type hint).

> Unlike modules, classes and functions, the annotation cannot be on
> the 
> attribute's *value*, because the attribute may not even have a value
> at 
> this point. And even if it does, it might be immutable, or like the 
> type, many different attributes, with different annotations, may be 
> sharing the same value.

Agree.

> This rules out putting the docstring on the attribute directly.

Agree.

> The syntax shows us annotating the attribute, and that's exactly what
> got annotated.

I'm still not in aligned on this point. I think I agree that if you
annotate an attribute with `Annotated`, you're defining a pseudo type
that can serve to document the attribute. However, as I've
demonstrated, that can be inherited by defining that type in advance
and using it to annotate multiple attributes. As I suggested, I think a
reasonable approach could be to either take the last string in the
annotation, or combine multiple strings if present.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YAWO5EUWMC2HLY6XF55GW2NBOW3XTJ4B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sun, Dec 12, 2021 at 12:48:36PM -0800, Paul Bryan wrote:

> But what's being annotated, the type or the attribute?

That's easy to test:

>>> class A:
... attr: Annotated[int, "Doc string"]
... 
>>> int.__annotations__
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: type object 'int' has no attribute '__annotations__'

Okay, so it's not the type. *wink*

>>> A.__annotations__
{'attr': typing.Annotated[int, 'Doc string']}

It's the *class* that is annotated. But note that the mapping is between 
the attribute name to annotation, so in the sense that attributes are 
represented by their name, it is the attribute that is annotated.

As it obviously has to be. After all, didn't we *literally* annotate the 
attribute?

attr: annotation

Don't be fooled by the use of the Annotated pseudo-type as the 
annotation. We're still annotating the attribute, just like the code 
shows, regardless of what type we annotate it as.

If we had written:

attr: Sequence[int]

there would be no questions about what's being annotated, is it the type 
int or the attribute.

It cannot be the type, because many different objects with many 
different annotations may share the same type.

Unlike modules, classes and functions, the annotation cannot be on the 
attribute's *value*, because the attribute may not even have a value at 
this point. And even if it does, it might be immutable, or like the 
type, many different attributes, with different annotations, may be 
sharing the same value.

This rules out putting the docstring on the attribute directly.

The syntax shows us annotating the attribute, and that's exactly what 
got annotated.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UBGM3KTTH3A7BJXOQCZAAGKNWJ6LBRN4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Paul Bryan

On Sun, 2021-12-12 at 12:23 -0800, Christopher Barker wrote:

> As for Typing.Annotated: it has one advantage in that typing tools
> currently (presumably) already accommodate extracting the type
> information from it -- see what typing.get_type_hints() does.

Confirmed. I have tooling today using exactly that to dynamically
generate API documentation.

> yes, it's very clear that any standardized use of annotations needs
> to be compatible with type hints and type checkers, static and
> dynamic. Which i think is Steve's point -- typing.Annotated is indeed
> compatible with those uses.

But what's being annotated, the type or the attribute?

> and yet the docs for typing.Annotated don't seem to say that. So what
> concerns me is that this may break other uses of Annotated, which are
> expecting types to be stored there. One problem with a docstrings is
> that they are  "just a string".  isinstance checks, as mentioned in
> the docs, are not so helpful. So we'd be establishing a new standard:
> “a bare string in an Annotated type is a docstring” -- adding a
> typing,Documented wouldn't be a much bigger lift ( I wonder if making
> it a subclass of Annotated would help).

1. Docstrings today are attached to the things they're documenting by
way of an included `__doc__` attrbute.  `Annotated` is arguably
"attached" to the type, to the extent that the type is embedded in the
annotation itself. Illustration:

SomeType = Annotated[str, "some type"]

class A:
  x: SomeType
  y: SomeType

Clearly, above we documented the type, not the attribute. We could
solve this by allowing it to be re-annotated, which seems perfectly
valid:

SomeType = Annotated[str, "some type"]

class A:
  x: Annotated[SomeType, "x docstring"]
  y: Annotated[SomeType, "y docstring"]

The last bare string could "win" as the prevaling docstring:

>>> SomeType = Annotated[str, "some type"]
>>> U = Annotated[SomeType, "new docstring"]
>>> U
typing.Annotated[str, 'some type', 'new docstring']

Or, doc tools could even include them both to provide a more meaningful
context.

You could argue that since the type annotation of an attribute can be
extended in this manner, `Annotated` is an acceptable way of
documenting an attribute.

2. Adding a typing.Documented would definitely increase verbosity. I
think we may already be pushing the envelope with `Annotated[...]`;
`Annotated[..., Documented(...)] may be an annotation too far. Since
there is not yet a convention for a bare string in `Annotated`, I
suggest we reserve it the purpose of documenting the type or attribute.

> And I note that Annotated flattens nested Annotated types, so having
> both a docstring and other use of Annotated could be a bit tricky.

I think as long as strings are considered documentation—regardless of
where they appear in the annotation—we'd still be on solid ground.

> As for using dataclasses as a way to introduce a convention: that
> could be a good route — but I doubt Eric would go for it[*] and in
> any case, it would in fact be introducing a new convention, so
> getting some consensus on what that convention should be would be
> helpful.
> 
> [*] Eric, of course, can speak for himself, but so far, dataclasses
> deliberately don’t do anything with annotations other than pass them
> on, and there is active debate about whether they should do more, and
> if so, what? 

I don't think dataclasses have to do anything with them. I'm literally
embedding strings in Annotated today, for this exact purpose, with no
impact on dataclasses. I think the work would be in documentation tools
to interpret the annotations and produce something usable to the
developer.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AAYG4NR4O3L7HIPTO4JOVJ2I2HKKA2H4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Christopher Barker

On Sat, Dec 11, 2021 at 10:55 PM Steven D'Aprano 
wrote:

> If dataclasses, and other classes, move to using Annotated and
> __attrdoc__ for docstrings,

OK -- THAT is the point I brought up earlier, and you responded  with "this
is settled" -- it's the __attrdoc__ part that needs to be standardized.

As for Typing.Annotated: it has one advantage in that typing tools
currently (presumably) already accommodate extracting the type information
from it -- see what typing.get_type_hints() does.

>
> the tools will follow. Runtime tools will
> look at the dunder, static tools will look at the annotation directly.
>

I hope not. *maybe* inspect.get_annotations() though.

> **Which they will have to do anyway** if we add the convention that
> string literals following an assignment get recorded in __attrdoc__.
> Runtime tools will want to look in the dunder, not parse the source
> code. And static tools which aren't Sphinx will need to be adjusted to
> look at the string literals.
>

but they do that anyway, for all other docstrings.

> As far as using the lowercase-annotation for the docstring: in a world of

> > more and more type hinted python

yes, it's very clear that any standardized use of annotations needs to be
compatible with type hints and type checkers, static and dynamic. Which i
think is Steve's point -- typing.Annotated is indeed compatible with those
uses.

> > x: "spam"
> >
> > ... isn't really an option.
>
> Of course it is. Just decorate your class with @no_type_hints.
>

and then your class attribute isn't type hinted. Of course you can do it,
but it can't be the standard way to  add documentation to class attributes
— see above.

All it takes is literally one stdlib class to start doing it officially,
> and people will pay attention.
>

Sure -- but where do the docstrings extracted from a typing.Annoated object
go?

__attrdoc__ has been suggested -- but THAT requires some level of approval
from the SC.

As far as I am concerned, this is exactly the sort of use-case that
> Annotated was invented for.

and yet the docs for typing.Annotated don't seem to say that. So what
concerns me is that this may break other uses of Annotated, which are
expecting types to be stored there. One problem with a docstrings is that
they are  "just a string".  isinstance checks, as mentioned in the docs,
are not so helpful. So we'd be establishing a new standard: “a bare string
in an Annotated type is a docstring” -- adding a typing,Documented wouldn't
be a much bigger lift ( I wonder if making it a subclass of Annotated would
help).

And I note that Annotated flattens nested Annotated types, so having both a
docstring and other use of Annotated could be a bit tricky.

I'm not a type hints user -- but I think this should be run by the MyPy and
other folks to see if I’m right in my skepticism.

Docstrings are metadata. Annotated is for
> attaching metadata to an annotation that is associated with some
> variable. Done and done.

It’s not Done unless the community accepts it as standard practice. This is
why we have all the Typing PEPs — annotations have been there a long time,
but if various tools use them differently, they really aren’t that useful.

As for using dataclasses as a way to introduce a convention: that could be
a good route — but I doubt Eric would go for it[*] and in any case, it
would in fact be introducing a new convention, so getting some consensus on
what that convention should be would be helpful.

-CHB

[*] Eric, of course, can speak for himself, but so far, dataclasses
deliberately don’t do anything with annotations other than pass them on,
and there is active debate about whether they should do more, and if so,
what?

>
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/4SMDBNO2FUQCJECDD6ILYK6RAQZ25TNP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MMGWIW2MMS4ZNER4ZDUD2UBLUIE7VEXU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Ricky Teachey

On Sun, Dec 12, 2021, 1:57 AM Steven D'Aprano  wrote:

On Sun, Dec 12, 2021 at 12:38:06AM -0500, Ricky Teachey wrote:

> But Steve, since the most utilized documentation tool in the python
> universe, sphinx, doesn't look at Annotated

Yet.

> or in the lowercase-annotation part of an expression

Yet.

> for this and instead looks at a bare string below the
> member definition, and since I presume it had been doing this for a long
> time (before Annotated was added not long ago, and probably before the
> annotation too?), doesn't this mean that right now there are at least
three
> competing ways being used to provide attribute docstrings?

And before we had PEP 484 there were multiple competing and
incompatible conventions for putting type hints in docstrings.

https://www.python.org/dev/peps/pep-0484/

There are:

- Epytext (Javadoc style)
- ReST (Sphinx)
- Googledoc
- Numpydoc

although some of them may no longer be widely used.

https://stackoverflow.com/questions/3898572/what-are-the-most-common-python-docstring-formats

https://www.adrian.idv.hk/2018-02-18-docstring/

Static and runtime tools that wanted to get type hints had to guess the
format and parse the docstring to get the types. Who still does that?

If dataclasses, and other classes, move to using Annotated and
__attrdoc__ for docstrings, the tools will follow. Runtime tools will
look at the dunder, static tools will look at the annotation directly.

**Which they will have to do anyway** if we add the convention that
string literals following an assignment get recorded in __attrdoc__.
Runtime tools will want to look in the dunder, not parse the source
code. And static tools which aren't Sphinx will need to be adjusted to
look at the string literals.

So either way the tooling has to change.

Sphinx is not the decider here. I dare say that they will want to
continue supporting bare strings into the indefinite future, but new
tools, and those with weaker backwards-compatibility constaints, will
surely prefer to extract docstrings from __attrdoc__ than parsing the
source code.

> As far as using the lowercase-annotation for the docstring: in a world of
> more and more type hinted python (to be clear: I don't always use type
> hints but I do like this world), if you want to make you don't create a
> problem later, using the annotation slot for your docstring:
>
> x: "spam"
>
> ... isn't really an option.

Of course it is. Just decorate your class with @no_type_hints.

> - without an official blessed agreement that the existing Annotated
feature
> ought to be used for docstrings, there is little reason for other 3rd
party
> help/documentation tools to look in that location for these docstrings

All it takes is literally one stdlib class to start doing it officially,
and people will pay attention.

> I just think there's more to do here than add a decorator.

That depends on whether you mean, what do *we* need to do get the ball
rolling, or what does the entire Python ecosystem need to do to
transition to the day that every tool and library is actively using
attribute docstrings both statically and at runtime?

Of course the second part is a bigger job.

But for the first, we can either (1):

* bikeshed this for another six ~~months~~ weeks;
* get at least one core dev to agree to sponsor a PEP;
* write a PEP;
* bikeshed some more;
* wait for the Steering Council to hopefully accept the PEP;
* modify the interpreter;
* wait for the new version to be available;

or we can (2)

* implement and support the feature today;

to get to the point that external tools can start using it.

I'm not kidding, if somebody cared enough to do this, they could
probably have a patch for dataclasses to support this within a couple of
hours and a plugin for Sphinx within a day.

(Spoken with the supreme confidence of somebody who knows that he will
absolutely not have to do either.)

As far as I am concerned, this is exactly the sort of use-case that
Annotated was invented for. Docstrings are metadata. Annotated is for
attaching metadata to an annotation that is associated with some
variable. Done and done.

--
Steve

I agree. I think adding something like __attrdoc__ is a very good idea.
Would like to see it in all classes, personally, and then have help() look
there. But starting with dataclasses seems like a logical place and might
get us started on that goal.

And if we start this with dataclasses and it gets expanded, none of this
would slam the door on the idea of using a bare string after the definition
as sugar for populating __attr_doc__. In fact if there's any hope that
would ever happen, first people would need to find themselves using
Annotated far more often, and get tired of having to use the longish syntax
with an import and agree on such a syntax change.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-12 Thread Stephen J. Turnbull

Christopher Barker writes:

 > However, what I haven’t seen in this thread is discussion of what I think
 > is the key question:
 > 
 > Where/how should class attribute doc strings be stored?
 > 
 > Tacked on to the class __doc__ ?

-10.  They need to be accessible via the attribute name.  You could
insert the name before tacking on, but surely whatever format you
choose, there are projects where it would be the wrong style.

 > Another dict?
 > 
 > __attr_doc__

+0.4 for now.

 > Added to __annotaions__ ?

+0.6 for now.  That's where mypy puts type annotations, I suspect you
would have to use values of Annotated type, which might be annoying.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VVQPFC7O7GUZ5IDHQVC37X4DHCDYYMKY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

Okay, here we go:


```
import typing

def extract_docstrings(K):
"""Extract docstrings from the class K's annotations.

Class attribute docstrings are extracted from the second item
in Annotated[...] attributes. The original annotation is left
unchanged.

FIXME:
Handling of PEP 563 string annotations may not be best practice.

https://www.python.org/dev/peps/pep-0563/
"""
d = {}
for key, value in K.__annotations__.items():
if type(value) is str:
if value.startswith('Annotated[') and value.endswith(']'):
s = value[10: -1]
items = s.split(',')
if len(items) >= 2:
doc = items[1].strip()
d[key] = doc
elif typing.get_origin(value) is typing.Annotated:
d[key] = value.__metadata__[0]
K.__attrdocs__ = d
return K
```

And here's an example:

>>> @extract_docstrings
... class Lunch:
... spam: "Annotated[int, 'a delicious meat-like product']"
... eggs: Annotated[float, 'something that goes with spam']
... 
>>> Lunch.__attrdocs__
{'spam': "'a delicious meat-like product'", 'eggs': 'something that goes with 
spam'}



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/I4FZKLAVNMJ6OWICFU47JPZFII4DHRBY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sun, Dec 12, 2021 at 12:38:06AM -0500, Ricky Teachey wrote:

> But Steve, since the most utilized documentation tool in the python
> universe, sphinx, doesn't look at Annotated

Yet.

> or in the lowercase-annotation part of an expression

Yet.

> for this and instead looks at a bare string below the
> member definition, and since I presume it had been doing this for a long
> time (before Annotated was added not long ago, and probably before the
> annotation too?), doesn't this mean that right now there are at least three
> competing ways being used to provide attribute docstrings?

And before we had PEP 484 there were multiple competing and 
incompatible conventions for putting type hints in docstrings.

https://www.python.org/dev/peps/pep-0484/

There are:

- Epytext (Javadoc style)
- ReST (Sphinx)
- Googledoc
- Numpydoc

although some of them may no longer be widely used.

https://stackoverflow.com/questions/3898572/what-are-the-most-common-python-docstring-formats

https://www.adrian.idv.hk/2018-02-18-docstring/

Static and runtime tools that wanted to get type hints had to guess the 
format and parse the docstring to get the types. Who still does that?

If dataclasses, and other classes, move to using Annotated and 
__attrdoc__ for docstrings, the tools will follow. Runtime tools will 
look at the dunder, static tools will look at the annotation directly.

**Which they will have to do anyway** if we add the convention that 
string literals following an assignment get recorded in __attrdoc__. 
Runtime tools will want to look in the dunder, not parse the source 
code. And static tools which aren't Sphinx will need to be adjusted to 
look at the string literals.

So either way the tooling has to change.

Sphinx is not the decider here. I dare say that they will want to 
continue supporting bare strings into the indefinite future, but new 
tools, and those with weaker backwards-compatibility constaints, will 
surely prefer to extract docstrings from __attrdoc__ than parsing the 
source code.

> As far as using the lowercase-annotation for the docstring: in a world of
> more and more type hinted python (to be clear: I don't always use type
> hints but I do like this world), if you want to make you don't create a
> problem later, using the annotation slot for your docstring:
> 
> x: "spam"
> 
> ... isn't really an option.

Of course it is. Just decorate your class with @no_type_hints.

> - without an official blessed agreement that the existing Annotated feature
> ought to be used for docstrings, there is little reason for other 3rd party
> help/documentation tools to look in that location for these docstrings

All it takes is literally one stdlib class to start doing it officially, 
and people will pay attention.

> I just think there's more to do here than add a decorator.

That depends on whether you mean, what do *we* need to do get the ball 
rolling, or what does the entire Python ecosystem need to do to 
transition to the day that every tool and library is actively using 
attribute docstrings both statically and at runtime?

Of course the second part is a bigger job.

But for the first, we can either (1):

* bikeshed this for another six ~~months~~ weeks;
* get at least one core dev to agree to sponsor a PEP;
* write a PEP;
* bikeshed some more;
* wait for the Steering Council to hopefully accept the PEP;
* modify the interpreter;
* wait for the new version to be available;

or we can (2)

* implement and support the feature today;

to get to the point that external tools can start using it.

I'm not kidding, if somebody cared enough to do this, they could 
probably have a patch for dataclasses to support this within a couple of 
hours and a plugin for Sphinx within a day.

(Spoken with the supreme confidence of somebody who knows that he will 
absolutely not have to do either.)

As far as I am concerned, this is exactly the sort of use-case that 
Annotated was invented for. Docstrings are metadata. Annotated is for 
attaching metadata to an annotation that is associated with some 
variable. Done and done.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4SMDBNO2FUQCJECDD6ILYK6RAQZ25TNP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-11 Thread Ricky Teachey

On Sat, Dec 11, 2021, 10:58 PM Steven D'Aprano  wrote:

> On Sat, Dec 11, 2021 at 05:02:39PM -0800, Christopher Barker wrote:
> > On Sat, Dec 11, 2021 at 3:03 PM Steven D'Aprano 
> wrote:
>
> > > Didn't we decide there was an existing feature for this, no need for
> > > new syntax?
>
> > Well, no. In fact, you could always put anything you wanted into an
> > annotation:
>
> Right, and we want to put a documentation string in it! :-)
>
>
> > Anyway, take a look at the docs for Annotated:
> >
> > https://docs.python.org/3/library/typing.html#typing.Annotated
> >
> > It has a number of possible unspecified uses, but fundamentally, it's
> about
> > the adding information to the type, not to the attribute -- e.g. not
> really
> > intended for docstrings.
>
> I don't think that the docs argue against this. Annotated is documented
> as the way to put arbitrary metadata into an annotation. What you do
> with it is up to the consumer of the metadata.
>
> The docs talk about this associating the metadata with the type, but
> that's because this is the *typing* module and *type* hints are still
> the primary use-case for annotations.
>
> And despite what the docs say, the metadata is actually, literally,
> associated with the variable (or its name at least). The annotation that
> gets stored in __annotations__ is **not**
>
> {T: "metadata"}
>
> as a naive reading of the docs might trick you into believing, but
>
> {"var": Annotated[T, "metadata"]}
>
>
> In any case, we can still use annotations for anything we like, and
> while typing is definitely and certainly the primary use for them, there
> will always be a way to opt-out. (Just don't run mypy!)
>
> Don't need type hints at all? Great, put your doc string as the
> annotation.
>
> μ: 'Number of steps from x0 to the start of the cycle.'
> λ: 'Length (period) of the cycle.'
> y: 'Value of f at the start of the cycle.' = None
>
> You might want to annotate the class with `no_type_check` if there is
> any chance the user might want to run mypy over your module, but that's
> entirely up to you.
>
> You do want type hints? Great, use Annotated. If there's an attribute
> that you don't want to give a type hint to, just use Any. (A small price
> to pay for using an existing feature, instead of waiting until Python
> 3.11 or 3.27 when it gets added to the core language.)
>
> μ: Annotated[int,
>  'Number of steps from x0 to the start of the cycle.']
> λ: Annotated[int, 'Length (period) of the cycle.']
> y: Annotated[Any,
>  'Value of f at the start of the cycle.'
>  ] = None
>
> No new syntax required. No bare strings floating around. You can
> generate the doc strings dynamically, they don't have to be literals.
>
> lunch: Annotated[Meal, ' '.join(['spam']*SPAM_COUNT)]
>
>
> > Sure, typing.Annotated *could* be used that way, but I don't think we can
> > say that the problem is solved.
>
> I think that from a language standpoint, we absolutely can say that the
> problem is solved. We don't need new syntax.
>
> We *might* want to standardise on a decorator or other tool to extract
> those docstrings, possibly sticking them in a new __attrdoc__ dunder.
> Being a dunder, we should get the core dev's approval for this, even if
> it is a third-party tool that uses it.
>
> (I assume we don't already use __attrdoc__ for something?)
>
> If the first use of this is dataclasses, then that would be de facto
> core dev approval.
>
> So we have a tool that takes an annotation:
>
> var: Annotated[T, docstring, *args]
>
> extracts out the docstring to an __attrdoc__ dunder:
>
> # Class.__attrdoc__
> {'var': docstring}
>
> and removes the annotation from the annotation:
>
> # Class.__annotation__
> {'var': Annotated[T, *args]}  # or just T if *args is empty?
>
> Dataclasses can automatically apply this tool (subject to backwards
> compatibility constraints), or we can apply it as a decorator to any
> other class we want.
>
> So we've gone from the *hard* problem of making a language change (new
> syntax, build the functionality into the interpreter) to the *easy*
> problem of writing a decorator.
>
>
> > Heck you could just as easily provide a tuple for the annotation:
> >
> > In [17]: class C:
> > ...: x: ("Doc string", int) = 123
> > ...:
>
>
> Sure, annotations can be anything you like. But external static typing
> tools may not support this, and will be confused or break. Same for
> third-party runtime tools.
>
> Best practice is to stick as closely to the standard type-hinting
> behaviour as you can, or to completely opt-out with no_type_hints.
>
>
> > The use cases for annotations are a bit up in the air at the moment --
> see
> > the ongoing discussion on python-dev. But I don't think there's any doubt
> > that any future endorsed use of them will be compatible with static
> typing
> > (if not restricted to it), so extending the use of annotations for
> > docstrings

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sat, Dec 11, 2021 at 05:02:39PM -0800, Christopher Barker wrote:
> On Sat, Dec 11, 2021 at 3:03 PM Steven D'Aprano  wrote:

> > Didn't we decide there was an existing feature for this, no need for
> > new syntax?

> Well, no. In fact, you could always put anything you wanted into an
> annotation:

Right, and we want to put a documentation string in it! :-)

> Anyway, take a look at the docs for Annotated:
> 
> https://docs.python.org/3/library/typing.html#typing.Annotated
> 
> It has a number of possible unspecified uses, but fundamentally, it's about
> the adding information to the type, not to the attribute -- e.g. not really
> intended for docstrings.

I don't think that the docs argue against this. Annotated is documented 
as the way to put arbitrary metadata into an annotation. What you do 
with it is up to the consumer of the metadata.

The docs talk about this associating the metadata with the type, but 
that's because this is the *typing* module and *type* hints are still 
the primary use-case for annotations.

And despite what the docs say, the metadata is actually, literally, 
associated with the variable (or its name at least). The annotation that 
gets stored in __annotations__ is **not**

{T: "metadata"}

as a naive reading of the docs might trick you into believing, but

{"var": Annotated[T, "metadata"]}

In any case, we can still use annotations for anything we like, and 
while typing is definitely and certainly the primary use for them, there 
will always be a way to opt-out. (Just don't run mypy!)

Don't need type hints at all? Great, put your doc string as the 
annotation.

μ: 'Number of steps from x0 to the start of the cycle.'
λ: 'Length (period) of the cycle.'
y: 'Value of f at the start of the cycle.' = None

You might want to annotate the class with `no_type_check` if there is 
any chance the user might want to run mypy over your module, but that's 
entirely up to you.

You do want type hints? Great, use Annotated. If there's an attribute 
that you don't want to give a type hint to, just use Any. (A small price 
to pay for using an existing feature, instead of waiting until Python 
3.11 or 3.27 when it gets added to the core language.)

μ: Annotated[int,
 'Number of steps from x0 to the start of the cycle.']
λ: Annotated[int, 'Length (period) of the cycle.']
y: Annotated[Any, 
 'Value of f at the start of the cycle.'
 ] = None

No new syntax required. No bare strings floating around. You can 
generate the doc strings dynamically, they don't have to be literals.

lunch: Annotated[Meal, ' '.join(['spam']*SPAM_COUNT)]

> Sure, typing.Annotated *could* be used that way, but I don't think we can
> say that the problem is solved.

I think that from a language standpoint, we absolutely can say that the 
problem is solved. We don't need new syntax.

We *might* want to standardise on a decorator or other tool to extract 
those docstrings, possibly sticking them in a new __attrdoc__ dunder. 
Being a dunder, we should get the core dev's approval for this, even if 
it is a third-party tool that uses it.

(I assume we don't already use __attrdoc__ for something?)

If the first use of this is dataclasses, then that would be de facto 
core dev approval.

So we have a tool that takes an annotation:

var: Annotated[T, docstring, *args]

extracts out the docstring to an __attrdoc__ dunder:

# Class.__attrdoc__
{'var': docstring}

and removes the annotation from the annotation:

# Class.__annotation__
{'var': Annotated[T, *args]}  # or just T if *args is empty?

Dataclasses can automatically apply this tool (subject to backwards 
compatibility constraints), or we can apply it as a decorator to any 
other class we want.

So we've gone from the *hard* problem of making a language change (new 
syntax, build the functionality into the interpreter) to the *easy* 
problem of writing a decorator.

> Heck you could just as easily provide a tuple for the annotation:
> 
> In [17]: class C:
> ...: x: ("Doc string", int) = 123
> ...:

Sure, annotations can be anything you like. But external static typing 
tools may not support this, and will be confused or break. Same for 
third-party runtime tools.

Best practice is to stick as closely to the standard type-hinting 
behaviour as you can, or to completely opt-out with no_type_hints.

> The use cases for annotations are a bit up in the air at the moment -- see
> the ongoing discussion on python-dev. But I don't think there's any doubt
> that any future endorsed use of them will be compatible with static typing
> (if not restricted to it), so extending the use of annotations for
> docstrings would take a bit of work to define and accept that use.

Annotated is explicitly documented as *arbitrary* metadata that is 
entirely up to the application to interpret. There's no reason to think 
that will change.

> Perhaps a better way to do that than

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-11 Thread Paul Bryan

On Sat, 2021-12-11 at 17:02 -0800, Christopher Barker wrote:

> It [Annotated] has a number of possible unspecified uses, but
> fundamentally, it's about the adding information to the type, not to
> the attribute -- e.g. not really intended for docstrings.

Ah, good point. I've conflated the two because dataclass attributes are
used to define the schema of the dataclass. For general purpose
attribute docstrings (module, class), I agree Annotated is probably
inappropriate. 

> Perhaps a better way to do that than to use Annotated would be to
> introduce new "thing", maybe in teh typoing module, like
> "Documented":

I think this will suffer from the same problem as Annotated: it would
document the type, not the attribute itself.

> Anyway, what I'm getting at is that it needs to be determined whare
> to put the docstrings before we can get very far with this idea.

How is Sphinx is pulling the string below the attribute? Whatever
mechanism is to be used, I propose it must allow for runtime
introspection.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LG7NQUZ67XTVXUAEXRTNHTXFXIH4NDYC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-11 Thread Christopher Barker

On Sat, Dec 11, 2021 at 3:03 PM Steven D'Aprano  wrote:

> On Sat, Dec 11, 2021 at 10:07:50AM -0800, Christopher Barker wrote:
> > Where/how should class attribute doc strings be stored?
> >
> > Tacked on to the class __doc__ ?
> > Another dict?
> > __attr_doc__
> > Added to __annotaions__ ?
> > Something else?
>
> Didn't we decide there was an existing feature for this, no need for
> new syntax?
>
> >>> from typing import Annotated
> >>> class C:
> ... x: Annotated[int, "Doc string"] = 123
> ...
> >>> C.x
> 123
> >>> C.__annotations__
> {'x': typing.Annotated[int, 'Doc string']}
>

Well, no. In fact, you could always put anything you wanted into an
annotation:

In [10]: class C2:
...: x: "a docstring" = 123
...:

In [11]: inspect.get_annotations(C2)
Out[11]: {'x': 'a docstring'}

So the typing module contains a nifty thing they've called "Annotatated",
and, I imagine type checkers (like MyPy) do something sensible with them,
but there's nothing else going on here. Actually yes, there is, the
typing,get_type_hints function strips out the annotation:

In [13]: class C:
...: x: Annotated[int, "Doc string"] = 123
...:

In [14]: typing.get_type_hints(C)
Out[14]: {'x': int}

Anyway, take a look at the docs for Annotated:

https://docs.python.org/3/library/typing.html#typing.Annotated

It has a number of possible unspecified uses, but fundamentally, it's about
the adding information to the type, not to the attribute -- e.g. not really
intended for docstrings.

Sure, typing.Annotated *could* be used that way, but I don't think we can
say that the problem is solved.

Heck you could just as easily provide a tuple for the annotation:

In [17]: class C:
...: x: ("Doc string", int) = 123
...:

In [18]: inspect.get_annotations(C)
Out[18]: {'x': ('Doc string', int)}

The use cases for annotations are a bit up in the air at the moment -- see
the ongoing discussion on python-dev. But I don't think there's any doubt
that any future endorsed use of them will be compatible with static typing
(if not restricted to it), so extending the use of annotations for
docstrings would take a bit of work to define and accept that use.

Perhaps a better way to do that than to use Annotated would be to introduce
new "thing", maybe in teh typoing module, like "Documented":

class C:
x: Documented(doc_string="this is the doc string", type=int) = 123

and then inspect.get_annotations and typing,get_type_hints could be taught
to extract the type from the Documented object, and we could add an
inspect.get_docstrings() function, so that:

typing.get_type_hints(C)
{'x': int}

inspect.get_docstrings(C)
{'x': 'this is the doc string'}

And, of course, future syntax could automatically create a Documented()
object if there was a docstring in the appropriate place.

Or just have a new dunder for it, e.g. __attr_doc__

Anyway, what I'm getting at is that it needs to be determined whare to put
the docstrings before we can get very far with this idea.

-CHB

>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UQV3GZLVHDCHXKN4L2QCIHYLFTWBFEW3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LEFJD7BVN43E4EUNZOKTT2M7YOYU262R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-11 Thread Christopher Barker

On Sat, Dec 11, 2021 at 10:49 AM Ricky Teachey  wrote:

> The __annotations__ already exists. Is that a point in favor?
>

Yes and no. Right now, any object can be stored in annotations — having a
docstring tacked on would break who knows how much code.

However, there is the new inspect.get_annotations function, which could be
adapted to process doc strings.

https://docs.python.org/3/howto/annotations.html

If the syntax could become sugar for creating an Annotated object in
> __annotations__, this would be a pretty convenient location to find them.
>
> On the other hand, not every type hinted variable will have a docstring.
>

And not every attribute with a docstring would have an annotation.

 It seems like storing them in a so called __attr_doc__ might be the most
> straightforward thing to do.
>

Yes, as doc strings really are a different thing.

Though one might to also document other things like function parameters—
which would be an argument for extending what __annotations__ is used for.

If that were done, does it matter that the contents of __attr_doc__ and
> __annotations__ are not coupled?
>

I don’t think so — they are quite different concepts.

-CHB

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OTIC243UANQAVFSI3JAHQZATFRDKT7KJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-11 Thread Paul Bryan

I think that because Sphinx could interpret strings below attributes
for documentation, that perhaps we should go in that direction in
Python proper.

Personally, I find this more readable:

  class C:
x: Annotated[str, "Doc string"]
    y: Annotated[int, "Doc string"]

over:

  class C:
x: str
    "Doc string"
y: int
"Doc string"


On Sun, 2021-12-12 at 10:00 +1100, Steven D'Aprano wrote:
> On Sat, Dec 11, 2021 at 10:07:50AM -0800, Christopher Barker wrote:
> 
> > Where/how should class attribute doc strings be stored?
> > 
> > Tacked on to the class __doc__ ?
> > Another dict?
> > __attr_doc__
> > Added to __annotaions__ ?
> > Something else?
> 
> Didn't we decide there was an existing feature for this, no need for 
> new syntax?
> 
> > > > from typing import Annotated
> > > > class C:
> ... x: Annotated[int, "Doc string"] = 123
> ... 
> > > > C.x
> 123
> > > > C.__annotations__
> {'x': typing.Annotated[int, 'Doc string']}
> 
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MEFYE3FPMI7L4QPO4PRJ424IJ3HPXFEN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Sat, Dec 11, 2021 at 10:07:50AM -0800, Christopher Barker wrote:

> Where/how should class attribute doc strings be stored?
> 
> Tacked on to the class __doc__ ?
> Another dict?
> __attr_doc__
> Added to __annotaions__ ?
> Something else?

Didn't we decide there was an existing feature for this, no need for 
new syntax?

>>> from typing import Annotated
>>> class C:
... x: Annotated[int, "Doc string"] = 123
... 
>>> C.x
123
>>> C.__annotations__
{'x': typing.Annotated[int, 'Doc string']}


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UQV3GZLVHDCHXKN4L2QCIHYLFTWBFEW3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-11 Thread Ricky Teachey

On Sat, Dec 11, 2021, 1:19 PM Christopher Barker 
wrote:

> 
>
> Where/how should class attribute doc strings be stored?
>
> Tacked on to the class __doc__ ?
>
> Another dict?
>
> __attr_doc__
>
> Added to __annotaions__ ?
>
> Something else?
>
> If they are to be available at run time, they need to go somewhere…
>
> -CHB
>
>>
>> --
> Christopher Barker, PhD (Chris)
>

The __annotations__ already exists. Is that a point in favor? If the syntax
could become sugar for creating an Annotated object in __annotations__,
this would be a pretty convenient location to find them.

On the other hand, not every type hinted variable will have a docstring. It
might be inconvenient to have them there only. It seems like storing them
in a so called __attr_doc__ might be the most straightforward thing to do.

If that were done, does it matter that the contents of __attr_doc__ and
__annotations__ are not coupled? The strings might need to be modified.
Would be inconvenient to have to modify two places in such a situation.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7S4RM4APDP4FC7L6DBHJU7GAHES3ABPV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-11 Thread Ricky Teachey

On Sat, Dec 11, 2021, 12:57 AM Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

Simão Afonso writes:
 > On 2021-12-10 12:20:44, Ricky Teachey wrote:
 > > I meant to ask about a (global) module member, not the module docstring
 > > itself. Like MY_GLOBAL below:
 > >
 > >  """This is the module docs"""
 > >
 > >  MY_GLOBAL = None
 > >  """MY_GLOBAL docs"""
 > >
 > > Is this "global docstring" recognized by Sphinx as a docstring, too?
 >
 > My bad.
 >
 > Double checked and that's right, it is recognised as such.

To my mind Sphinx is sufficiently widely used that this settles the
"above or below" question.

Steve

I agree.

Even with this convention one could still accomplish separating the
documentation section from the definition section, like:

class Steel:
"""A simple material model of steel."""

# parameters
E_s: float
"the linear elastic modulus"

nu: float
"the poisson's ratio"

gamma: float
"the density"

# values
E_s = 29e6  # ksi
nu = 0.3
gamma = 490  # lbs per cu ft

As has already been observed, we already have something LIKE an attribute
docstring, which is an Annotated type hint object.

Should the bare docstring below the member become sugar for creating an
Annotated type hint?

x = 1
"spam"

Would then be equivalent to:

x: typing.Annotated[typing.Any, "spam"]

This seems really nice to me.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LO7KDBDFKKDI5XJJ7GE227PBTTM3D7NN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-11 Thread Christopher Barker

On Fri, Dec 10, 2021 at 9:57 PM Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

> To my mind Sphinx is sufficiently widely used


And the system used for the official Python docs.

that this settles the
> "above or below" question.


So yes :-)

However, what I haven’t seen in this thread is discussion of what I think
is the key question:

Where/how should class attribute doc strings be stored?

Tacked on to the class __doc__ ?

Another dict?

__attr_doc__

Added to __annotaions__ ?

Something else?

If they are to be available at run time, they need to go somewhere…

-CHB

>
> --
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RWTU2EJ2TIC62IU2WZZUPO3CEPLRLTRY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-10 Thread Stephen J. Turnbull

Simão Afonso writes:
 > On 2021-12-10 12:20:44, Ricky Teachey wrote:
 > > I meant to ask about a (global) module member, not the module docstring
 > > itself. Like MY_GLOBAL below:
 > > 
 > >  """This is the module docs"""
 > > 
 > >  MY_GLOBAL = None
 > >  """MY_GLOBAL docs"""
 > > 
 > > Is this "global docstring" recognized by Sphinx as a docstring, too?
 > 
 > My bad.
 > 
 > Double checked and that's right, it is recognised as such.

To my mind Sphinx is sufficiently widely used that this settles the
"above or below" question.

Steve


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EQKNZVG3NQBSTDYHLYWNW2A3U6E2MRXZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-10 Thread Simão Afonso

On 2021-12-10 12:20:44, Ricky Teachey wrote:
> I meant to ask about a (global) module member, not the module docstring
> itself. Like MY_GLOBAL below:
> 
>  """This is the module docs"""
> 
>  MY_GLOBAL = None
>  """MY_GLOBAL docs"""
> 
>  class CLS:
>"""CLS docs"""
> 
>attr: int
>"""CLS.attr doc"""
> 
> Is this "global docstring" recognized by Sphinx as a docstring, too?

My bad.

Double checked and that's right, it is recognised as such.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XWTA2CRUUUAGSBG3O6TIYDQNSOWLXGD5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-10 Thread Ricky Teachey

On Fri, Dec 10, 2021 at 11:46 AM Simão Afonso <
simao.afo...@powertools-tech.com> wrote:

> On 2021-12-10 08:20:25, Ricky Teachey wrote:
> > Very very interesting that Sphinx already treats a bare string under the
> > parameter as documentation! I had no idea. Does it do the same thing at
> the
> > module level?
>
> Yes.
>
> > $ cat module.py
> > """This is the module docs"""
> >
> > class CLS:
> >   """CLS docs"""
> >
> >   attr: int
> >   """CLS.attr doc"""


I meant to ask about a (global) module member, not the module docstring
itself. Like MY_GLOBAL below:

 """This is the module docs"""

 MY_GLOBAL = None
 """MY_GLOBAL docs"""

 class CLS:
   """CLS docs"""

   attr: int
   """CLS.attr doc"""

Is this "global docstring" recognized by Sphinx as a docstring, too?

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/643W2H5YJZN7G5AAQUE4HL7VCAERVEUF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-10 Thread Simão Afonso

On 2021-12-10 08:20:25, Ricky Teachey wrote:
> Very very interesting that Sphinx already treats a bare string under the
> parameter as documentation! I had no idea. Does it do the same thing at the
> module level?

Yes.

> $ cat module.py
> """This is the module docs"""
>
> class CLS:
>   """CLS docs"""
>
>   attr: int
>   """CLS.attr doc"""
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HTFID5VQYMO65JACAPDTQDRXJVYWKLL7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-10 Thread Ricky Teachey

On Fri, Dec 10, 2021, 6:08 AM Simão Afonso 
wrote:

> On 2021-12-09 12:47:28, Paul Bryan wrote:
> > On Thu, 2021-12-09 at 19:01 +, Simão Afonso wrote:
> > > I'm using docstrings bellow the attributes (analogous to functions
> > > and
> > > classes), I think it works well.
> > > It helps with distinguishing the class docstring from the arguments.
> >
> > I think you'll find on close inspection that those strings don't "go
> > anywhere" (i.e. they're not in help documentation, they're not
> > introspectable.)
>
> Wow, that's right, I stand corrected.
>
> They do appear on Sphinx auto-generated documentation, so I assumed it
> was a regular annotation, like classes and functions.
>

Very very interesting that Sphinx already treats a bare string under the
parameter as documentation! I had no idea. Does it do the same thing at the
module level?

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/U5D53FEEDHL6WXCQHTCOSTR3FTYQAVF6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-10 Thread Simão Afonso

On 2021-12-09 12:47:28, Paul Bryan wrote:
> On Thu, 2021-12-09 at 19:01 +, Simão Afonso wrote:
> > I'm using docstrings bellow the attributes (analogous to functions
> > and
> > classes), I think it works well.
> > It helps with distinguishing the class docstring from the arguments.
> 
> I think you'll find on close inspection that those strings don't "go
> anywhere" (i.e. they're not in help documentation, they're not
> introspectable.)

Wow, that's right, I stand corrected.

They do appear on Sphinx auto-generated documentation, so I assumed it
was a regular annotation, like classes and functions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KRMCFF7EH35X3SUKOAOLPAMWNCF2GHDJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-09 Thread Ricky Teachey

On Thu, Dec 9, 2021 at 3:23 PM Ricky Teachey  wrote:

> On Thu, Dec 9, 2021 at 3:08 PM Christopher Barker 
> wrote:
>
>> On Thu, Dec 9, 2021 at 11:08 AM Simão Afonso <
>> simao.afo...@powertools-tech.com> wrote:
>>
>>> Shouldn't typing be encouraged on dataclasses, at least?
>>
>>
>> Typing shouldn’t be encouraged for any particular class, nor the language
>> itself. Typing policy is a matter for the project/organization to decide.
>>
>> But Annotations are required by dataclasses in any case. That’s how the
>> decorator identifies fields.
>>
>> The contents of those annotations are up to the user.
>>
>> -CHB
>>
>>
> Since not everyone uses dataclasses and to avoid confusion: what you mean
> here is that type HINTS are required by dataclasses, not Annotations. An
> Annotated is a type of type hint that contains an annotation, which we're
> loosely calling a member docstring in this thread (currently, a "member
> docstring" is not really a thing that exists, unless you count a comment or
> a string nearby the statement creating the member).
>

A correction to my correction (Chris Barker very kindly replied to me
off-list about it; what a mensch): type hints aren't required in a
dataclass. All that is required is *something* in the "type hint slot" of
the expression, and that something is often (officially?) called an
annotation.

So we have at least 3 meanings of annotation/annotate/annotated possibly at
work in this thread (what fun):

   1. an annotation (the slot where type hints usually go these days, but
   really, you can put anything there even when constructing a dataclass),
   2. the Annotated type hint type, and
   3. the annotation that is the second argument of the Annotated brackets
   operator. How confusing!

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/26WOAIZF4Y57XTXRKYSOBBETHPKKG5H7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-09 Thread Paul Bryan

On Thu, 2021-12-09 at 19:01 +, Simão Afonso wrote:
> I'm using docstrings bellow the attributes (analogous to functions
> and
> classes), I think it works well.
> It helps with distinguishing the class docstring from the arguments.

I think you'll find on close inspection that those strings don't "go
anywhere" (i.e. they're not in help documentation, they're not
introspectable.)

> On 2021-12-08 13:25:55, Ricky Teachey wrote:
> > On Wed, Dec 8, 2021 at 1:20 PM Paul Bryan  wrote:
> > > I believe a Annotated[..., str] could become an attribute
> > > docstring if by
> > > consensus we deem it to be so. I can't see any disadvantages,
> > > perhaps save
> > > for the verbosity of `Annotated`. It certainly seems like an
> > > advantage to
> > > use an existing mechanism rather than define a new one that would
> > > appear to
> > > require changes to the interpreter.
> > 
> > I think this would be better than nothing. But it is a little
> > verbose. And
> > it requires you to supply the type hint.
> > 
> > I use type hinting, but I don't want to use it every time I provide
> > an
> > Annotation/docstring.
> 
> In this case, why not use the current behaviour of putting the
> docstring
> below the attribute?

There is no current behaviour to leverage.

> 
> Shouldn't typing be encouraged on dataclasses, at least?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/SG33WQJCHQUG23NKETD6FCPSYIWP5U4T/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H2MCTFUABNPYC2746G5D56W4DDRDORID/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-09 Thread Ricky Teachey

On Thu, Dec 9, 2021 at 3:08 PM Christopher Barker 
wrote:

> On Thu, Dec 9, 2021 at 11:08 AM Simão Afonso <
> simao.afo...@powertools-tech.com> wrote:
>
>> Shouldn't typing be encouraged on dataclasses, at least?
>
>
> Typing shouldn’t be encouraged for any particular class, nor the language
> itself. Typing policy is a matter for the project/organization to decide.
>
> But Annotations are required by dataclasses in any case. That’s how the
> decorator identifies fields.
>
> The contents of those annotations are up to the user.
>
> -CHB
>
>
Since not everyone uses dataclasses and to avoid confusion: what you mean
here is that type HINTS are required by dataclasses, not Annotations. An
Annotated is a type of type hint that contains an annotation, which we're
loosely calling a member docstring in this thread (currently, a "member
docstring" is not really a thing that exists, unless you count a comment or
a string nearby the statement creating the member).

On Thu, Dec 9, 2021 at 2:01 PM Simão Afonso <
simao.afo...@powertools-tech.com> wrote:

> I'm using docstrings bellow the attributes (analogous to functions and
> classes), I think it works well.
> It helps with distinguishing the class docstring from the arguments.
>
> On 2021-12-08 13:25:55, Ricky Teachey wrote:
> > On Wed, Dec 8, 2021 at 1:20 PM Paul Bryan  wrote:
> > > I believe a Annotated[..., str] could become an attribute docstring if
> by
> > > consensus we deem it to be so. I can't see any disadvantages, perhaps
> save
> > > for the verbosity of `Annotated`. It certainly seems like an advantage
> to
> > > use an existing mechanism rather than define a new one that would
> appear to
> > > require changes to the interpreter.
> >
> > I think this would be better than nothing. But it is a little verbose.
> And
> > it requires you to supply the type hint.
> >
> > I use type hinting, but I don't want to use it every time I provide an
> > Annotation/docstring.
>
> In this case, why not use the current behaviour of putting the docstring
> below the attribute?
>
> Shouldn't typing be encouraged on dataclasses, at least?


What is meant here by "the current behaviour"? Member docstrings are
currently not a feature supported by the language. If you like putting a
string under the member definition for reference, that sounds really nice.
But baking it into the language so that the docstring has to come right
after the member seems limiting to me.

This location requirement makes a lot of sense for modules and classes. But
for class members, I'd really like to be able to organize the docstring
location in a more flexible way, like (using Steven's proposed syntax):

class Steel:
"""A simple material model of steel."""

# parameters
modulus :: "the linear elastic modulus"
nu :: "the poisson's ratio"
gamma: "the density"

# values
modulus: float = 29e6  # ksi
nu: float = 0.3
gamma: float = 490  # lbs per cu ft



---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KZ5NW2XG2AJP5R5MEQNBIPWVA4OQGLTN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-09 Thread Christopher Barker

On Thu, Dec 9, 2021 at 11:08 AM Simão Afonso <
simao.afo...@powertools-tech.com> wrote:

> Shouldn't typing be encouraged on dataclasses, at least?


Typing shouldn’t be encouraged for any particular class, nor the language
itself. Typing policy is a matter for the project/organization to decide.

But Annotations are required by dataclasses in any case. That’s how the
decorator identifies fields.

The contents of those annotations are up to the user.

-CHB


> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/SG33WQJCHQUG23NKETD6FCPSYIWP5U4T/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CLICKMEZKFXPCPKI46GAIQD4S77VAEUR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-09 Thread Simão Afonso

I'm using docstrings bellow the attributes (analogous to functions and
classes), I think it works well.
It helps with distinguishing the class docstring from the arguments.

On 2021-12-08 13:25:55, Ricky Teachey wrote:
> On Wed, Dec 8, 2021 at 1:20 PM Paul Bryan  wrote:
> > I believe a Annotated[..., str] could become an attribute docstring if by
> > consensus we deem it to be so. I can't see any disadvantages, perhaps save
> > for the verbosity of `Annotated`. It certainly seems like an advantage to
> > use an existing mechanism rather than define a new one that would appear to
> > require changes to the interpreter.
> 
> I think this would be better than nothing. But it is a little verbose. And
> it requires you to supply the type hint.
> 
> I use type hinting, but I don't want to use it every time I provide an
> Annotation/docstring.

In this case, why not use the current behaviour of putting the docstring
below the attribute?

Shouldn't typing be encouraged on dataclasses, at least?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SG33WQJCHQUG23NKETD6FCPSYIWP5U4T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-09 Thread Ricky Teachey

On Thu, Dec 9, 2021 at 12:12 AM Paul Bryan  wrote:

> On Thu, 2021-12-09 at 12:32 +1100, Steven D'Aprano wrote:
>
> On Wed, Dec 08, 2021 at 09:45:35AM -0800, Paul Bryan wrote:
>
> I propose there is already a viable option in typing.Annotated.
> Example:
>
> @dataclass
> class InventoryItem:
> """Class for keeping track of an item in inventory."""
>
> name: Annotated[str, "Short name of the item."]
> unit_price: Annotated[float, "Price per unit in dollar."]
> ...
>
>
> Oh nice! Guido's time machine strikes again.
>
> So dataclasses could recognise Annotated and populate `__attrdoc__`.
>
>
> Indeed.
>
> Should all classes do the same? How about top level module variables?
>
>
> If we started with dataclass, it could be performed in the dataclass
> decorator and make_dataclass functions. I think this would be low-hanging
> fruit compared to classes and modules.
>

Again: better than nothing. But I'd really like to see this for all classes.

I suggest that if a proposal like this is accepted, the help() output for
classes and modules should have a new section added containing the member
docstring information; it essentially would insert the member information
to the end of the main docstring WHEN DISPLAYED (not suggested attaching it
permanently as part of the docstring). This new section would appear
directly beneath the module/class docstring when using help. Additionally,
other help()-ilke tools (such as the ipython ? operator) should be
encouraged to concatenate the member docstring information directly below
the class/module docstring when printing out help-like information.

To illustrate how things currently stand, here's a current, sort of
minimally complicated class definition (class B) with some annotations:

In [1]: from typing import Annotated

In [2]: class B:
   ...: """Docstring for the B class."""
   ...:
   ...: a: Annotated[int, "here is what i'd call a medium length
docstring"]
   ...: b: Annotated[str, "short docstring"]
   ...:

and here is the help(B) output that i get (Windows command line):

In [3]: help(B)
Help on class B in module __main__:

class B(builtins.object)
 |  Docstring for the B class.
 |
 |  Data descriptors defined here:
 |
 |  __dict__
 |  dictionary for instance variables (if defined)
 |
 |  __weakref__
 |  list of weak references to the object (if defined)
 |
 |  --
 |  Data and other attributes defined here:
 |
 |  __annotations__ = {'a': typing.Annotated[int, "here is what i'd call
a...

The member annotations aren't very discoverable this way... you can't even
see all of the first one. If it is agreed that Annotated should be The
Blessed Way to annotate members, this needs to be improved.

Here is the same thing for a dataclass (class C); here you CAN at least see
the whole of each docstring, but they are on one line together and it's not
all that great, and bits of the Annotated objects appear in this help
output at least 5 times by my count (maybe this can't be helped since it's
part of the Annotated object):

In [4]: from dataclasses import dataclass

In [5]: @dataclass
   ...: class C:
   ...: """Docstring for the C class."""
   ...:
   ...: a: Annotated[int, "here is what i'd call a medium length
docstring"]
   ...: b: Annotated[str, "short docstring"]
   ...:

In [6]: help(C)
Help on class C in module __main__:

class C(builtins.object)
 |  C(a: typing.Annotated[int, "here is what i'd call a medium length
docstring"], b: typing.Annotated[str, 'short docstring']) -> None
 |
 |  Docstring for the C class.
 |
 |  Methods defined here:
 |
 |  __eq__(self, other)
 |
 |  __init__(self, a: typing.Annotated[int, "here is what i'd call a medium
length docstring"], b: typing.Annotated[str, 'short docstring']) -> None
 |
 |  __repr__(self)
 |
 |  --
 |  Data descriptors defined here:
 |
 |  __dict__
 |  dictionary for instance variables (if defined)
 |
 |  __weakref__
 |  list of weak references to the object (if defined)
 |
 |  --
 |  Data and other attributes defined here:
 |
 |  __annotations__ = {'a': typing.Annotated[int, "here is what i'd call
a...
 |
 |  __dataclass_fields__ = {'a':
Field(name='a',type=typing.Annotated[int,...
 |
 |  __dataclass_params__ =
_DataclassParams(init=True,repr=True,eq=True,or...
 |
 |  __hash__ = None

Here is the ipython ? output for the dataclass, by the way-- it is FAR
better! But this is only so nice because dataclasses adds the typing
information to the __init__ signature. For regular classes (like class B
above), we would not get this nice output when using ?.

In [7]: C?
Init signature:
C(
a: typing.Annotated[int, "here is what i'd call a medium length
docstring"],
b: typing.Annotated[str, 'short docstring'],
) -> None
Docstring:  Docstring for the C class.

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-08 Thread Paul Bryan

On Thu, 2021-12-09 at 12:32 +1100, Steven D'Aprano wrote:
> On Wed, Dec 08, 2021 at 09:45:35AM -0800, Paul Bryan wrote:
> 
> > I propose there is already a viable option in typing.Annotated.
> > Example: 
> > 
> > @dataclass
> > class InventoryItem:
> >     """Class for keeping track of an item in inventory."""
> >     
> >     name: Annotated[str, "Short name of the item."]
> >     unit_price: Annotated[float, "Price per unit in dollar."]
> >     ...
> 
> Oh nice! Guido's time machine strikes again.
> 
> So dataclasses could recognise Annotated and populate `__attrdoc__`.

Indeed.

> Should all classes do the same? How about top level module variables?

If we started with dataclass, it could be performed in the dataclass
decorator and make_dataclass functions. I think this would be low-
hanging fruit compared to classes and modules.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XWSFSHUYMTUVYGVTZLS2GFBEZVQQEZ4C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-08 Thread Steven D'Aprano

On Wed, Dec 08, 2021 at 09:45:35AM -0800, Paul Bryan wrote:

> I propose there is already a viable option in typing.Annotated.
> Example: 
> 
> @dataclass
> class InventoryItem:
> """Class for keeping track of an item in inventory."""
> 
> name: Annotated[str, "Short name of the item."]
> unit_price: Annotated[float, "Price per unit in dollar."]
> ...

Oh nice! Guido's time machine strikes again.

So dataclasses could recognise Annotated and populate `__attrdoc__`.

Should all classes do the same? How about top level module variables?



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/23GMGHDEW6OMDASIXAPC6WQDJYQFJR5M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-08 Thread Greg Ewing


On 9/12/21 12:50 am, tmkehrenb...@gmail.com wrote:

The main criticism, I think, was that it is weird to have the docstring
*below* the attribute.


I don't think that's a problem. The docstring of a class or function
is also below the name of the class or function. I think it's
actually more consistent that way.

It seems excessive to invent a whole new type of string just for
this purpose. The neat thing about the docstring convention is that
it doesn't require any special syntax.

It would also raise some other questions, such as what happens if
you use a d-string in some other context? Does it just behave as
an ordinary string? Is it disallowed?


# in my_module.py:
d"Number of days in a week."
DAYS_PER_WEEK: Final = 7


Does this mean you could also use it on module-level functions?

d"This function is really important."
def f():
...

That would give you two different ways to document a function,
and it seems you could use both of them at the same time,
which could be confusing. If you're looking for documentation
of a function, there are two places you would have to look --
in the function itself, and in the module where it's defined.
Similarly for classes.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H5LQ5EHICVUOW75FYW6DMZAEYZBA2GAE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Wed, Dec 8, 2021 at 1:20 PM Paul Bryan  wrote:

> I believe a Annotated[..., str] could become an attribute docstring if by
> consensus we deem it to be so. I can't see any disadvantages, perhaps save
> for the verbosity of `Annotated`. It certainly seems like an advantage to
> use an existing mechanism rather than define a new one that would appear to
> require changes to the interpreter.
>

I think this would be better than nothing. But it is a little verbose. And
it requires you to supply the type hint.

I use type hinting, but I don't want to use it every time I provide an
Annotation/docstring.

Maybe we could do both: establish the 2nd argument of Annotated as the
docstring (and adjust the output of help() and other tools accordingly),
but also provide a new "shortcut syntax" as Steven proposed:

E_s : float : "the elastic modulus of steel"

... is sugar for:

E_s : typing.Annotated[float, "the elastic modulus of steel"]

...and:

x :: "spam"

...is sugar for:

x : typing.Annotated[typing.Any, "spam"]

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KTJ6BTJJJTGS42JLQWMATDLWXGWPZHZT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-08 Thread Paul Bryan

I believe a Annotated[..., str] could become an attribute docstring if
by consensus we deem it to be so. I can't see any disadvantages,
perhaps save for the verbosity of `Annotated`. It certainly seems like
an advantage to use an existing mechanism rather than define a new one
that would appear to require changes to the interpreter.

On Wed, 2021-12-08 at 13:04 -0500, Ricky Teachey wrote:
> On Wed, Dec 8, 2021 at 12:46 PM Paul Bryan  wrote:
> > I propose there is already a viable option in typing.Annotated.
> > Example: 
> > 
> > @dataclass
> > class InventoryItem:
> > """Class for keeping track of an item in inventory."""
> > 
> > name: Annotated[str, "Short name of the item."]
> > unit_price: Annotated[float, "Price per unit in dollar."]
> > ...
> > 
> > 
> > I've been using this in production code for about a year (with code
> > that generates OpenAPI document), with additional validation
> > constraints, and it's proving to be quite usable. 
> > 
> > Paul
> > 
> 
> 
> As I noted in the previous thread, a big downside IMO of using
> Annotated for docstrings is it really makes the help() output quite
> messy. The "docstrings" appear in at least 5 different places (as
> part of the Annotation objects).
> 
> This could be fixed, I guess. But that would skip over a question
> that gets raised here, which is: are a docstring and an annotation
> really the same thing? Are there disadvantages to assuming they are?
> 
> ---
> Ricky.
> 
> "I've never met a Kentucky man who wasn't either thinking about going
> home or actually going home." - Happy Chandler
> 
> > 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DCQNO4OS3COA5CAIB2FVXRQAIXDIOKCD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Wed, Dec 8, 2021 at 12:46 PM Paul Bryan  wrote:

> I propose there is already a viable option in typing.Annotated. Example:
>
> @dataclass
>
> class InventoryItem:
>
> """Class for keeping track of an item in inventory."""
>
> name: Annotated[str, "Short name of the item."]
>
> unit_price: Annotated[float, "Price per unit in dollar."]
>
> ...
>
>
> I've been using this in production code for about a year (with code that
> generates OpenAPI document), with additional validation constraints, and
> it's proving to be quite usable.
>
> Paul
>

As I noted in the previous thread, a big downside IMO of using Annotated
for docstrings is it really makes the help() output quite messy. The
"docstrings" appear in at least 5 different places (as part of the
Annotation objects).

This could be fixed, I guess. But that would skip over a question that gets
raised here, which is: are a docstring and an annotation really the same
thing? Are there disadvantages to assuming they are?

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JUGZ6UJHQQCCQNH7MCGMGJG6WZD4QAQJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-08 Thread David Lowry-Duda

On Wed, Dec 08, 2021 at 11:50:55AM -, tmkehrenb...@gmail.com wrote:
> A few weeks ago, I proposed on this mailing list to write docstrings for
> class attributes like this:
> 
> @dataclass
> class A:
> x: int
> """Docstring for x."""
> ... is a normal string, except that it is stored in __attrdoc__ when 
> used inside a class. It is stored with the name of the variable 
> *below* it as the key.

I like the idea of standardizing a name like __attrdoc__. Broadly, I 
think it is a good idea to enable attribute docstrings. But I think bare 
strings are probably not a good idea, and I definitely think adding more 
string prefixes is not a good idea.

Of the various ideas noted here and before, I think the one that 
resonates with me the most is to add a docstring parser (and possibly 
allow others to write their own if they write their docstrings in a 
different way). This would look like this.

# add a docstring parser decorator
@parse_attrdoc_from_docstring
class A:
"""
Class docstring.

x: Docstring for x
y: Docstring for y
"""
x: int
y: bool = False

And functionally, that would be equivalent to doing the following 
directly.

# or explicitly set __attrdoc__
class B:
"""Class docstring."""
__attrdoc__ = {
"x": "Docstring for x",
"y": "Docstring for y",
}
x: int
y: bool = False

If someone wanted to document an already written library and has a 
different (but consistent) convention in their class docstrings for 
class variables, then it shouldn't be too hard to modify the default 
parser. Maybe?

- DLD
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AUWFRUGDQC4DGSUURYAQSJOQ75NKXEQT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-08 Thread Paul Bryan

I propose there is already a viable option in typing.Annotated.
Example: 

@dataclass
class InventoryItem:
"""Class for keeping track of an item in inventory."""

name: Annotated[str, "Short name of the item."]
unit_price: Annotated[float, "Price per unit in dollar."]
...

I've been using this in production code for about a year (with code
that generates OpenAPI document), with additional validation
constraints, and it's proving to be quite usable. 

Paul

On Wed, 2021-12-08 at 11:50 +, tmkehrenb...@gmail.com wrote:
> A few weeks ago, I proposed on this mailing list to write docstrings
> for
> class attributes like this:
> 
> @dataclass
> class A:
>     x: int
>     """Docstring for x."""
> 
> The main criticism, I think, was that it is weird to have the
> docstring
> *below* the attribute.
> 
> To solve this problem, I propose to introduce a new kind of string: a
> d-string ('d' for 'docstring'; alternatively also 'v' because it
> looks a
> bit like a downward arrow, or 'a' for 'attribute docstring'). A d-
> string
> is a normal string, except that it is stored in __attrdoc__ when used
> inside a class. It is stored with the name of the variable *below*
> it as the key.
> 
> Examples:
> 
> @dataclass
> class InventoryItem:
>     """Class for keeping track of an item in inventory."""
>     
>     d"""Short name of the item."""
>     name: str
>     d"""Price per unit in dollar."""
>     unit_price: float
>     d"""Available quantity currently in the warehouse."""
>     quantity_on_hand: int = 0
> 
> 
> InventoryItem.__attrdoc__ == {
>     "name": "Short name of the item.",
>     "unit_price": "Price per unit in dollar.",
>     "quantity_on_hand": "Available quantity currently in the
> warehouse.",
> }
> 
> 
> 
> class HttpRequest(Enum):
>     """Types of HTTP requests."""
>     
>     d"""GET requests are used to retrieve data."""
>     GET = auto()
>     d"""POST requests are used to insert/update remote data."""
>     POST = auto()
> 
> 
> HttpRequest.__attrdoc__ == {
>     "GET": "GET requests are used to retrieve data.",
>     "POST": "POST requests are used to insert/update remote data.",
> }
> 
> d-strings can be combined with raw strings by using the prefix rd or
> dr.
> d-strings could also be used to document module-level constants:
> 
> # in my_module.py:
> d"Number of days in a week."
> DAYS_PER_WEEK: Final = 7
> 
> 
> my_module.__attrdoc__ == {"DAYS_PER_WEEK": "Number of days in a
> week."}
> 
> -Thomas
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/VV5MOSIRVSTRDTA5NFVVUXGD2JFBERRS/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IC7B7DMUO2WRF3IUSLN4GAJJ65E3MOSQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

On Wed, Dec 8, 2021 at 8:41 AM Steven D'Aprano  wrote:

> On Wed, Dec 08, 2021 at 11:50:55AM -, tmkehrenb...@gmail.com wrote:
> > A few weeks ago, I proposed on this mailing list to write docstrings for
> > class attributes like this:
> >
> > @dataclass
> > class A:
> > x: int
> > """Docstring for x."""
> >
> > The main criticism, I think, was that it is weird to have the docstring
> > *below* the attribute.
>
> ...
>
> Rather than implicitly linking a bare string to the following (or
> previous?) attribute, this would explicitly link them by putting them on
> the same line (at least for short docstrings).
>
> Full declaration syntax becomes:
>
> name : Type = value : docstring_expression
>
> with the various parts being optional. Example:
>
>
> @dataclass
> class InventoryItem:
> """Class for keeping track of an item in inventory."""
>
> name: str : "Short name of the item"
> unit_price: float : "Price per unit in dollar."
> quantity_on_hand: int = 0 : (
> "Available quantity currently in the warehouse.")
>
>
> The docstring expression could evaluate to any value, not necessarily a
> string, but conventionally one would use a string, and I expect style
> guides would discourage or ban non-strings. (But see below for a
> possible objection.)
>
> Aside from the name, each part of the syntax is optional:
>
> # Type hint only.
> name: Type
>
> # Assignment only.
> name = value
>
> # Declare a type hint, and a docstring.
> name: Type : docstring
>
> # Assign a value, and a docstring, with no type hint.
> name = value : docstring
>
> # And all three: type hint, value and docstring.
> name: Type = value : docstring
>
>
> If this makes sense, one might even have an attribute with no type hint
> or value, only a docstring:
>
> name : : docstring
>
> which would assign the docstring to `__attrdoc__` but otherwise have no
> effect.
>
> ...
>
>
> --
> Steve
>

IMO this is a much better idea than the OP's, or the similar d-string idea
I had come up with.

Here's an example of the kind of thing I imagine doing with this feature.

If I were doing a set of engineering calculations, I might get started by
writing pseudocode (but using the new docstrings) like this:

 cantilevered_beam.py #

"""Checking cantilevered beam designs."""

# material properties
E_s : float : "elastic modulus of steel"
F_y: float : "strength of steel"

# beam properties
I_beam : float : "moment of inertia of beam"
S_beam : float : "section modulus of beam"
L_beam : float : "cantilever length"
phi_M : float : "moment limit of beam"

# loads
P_u : float : "applied ultimate force"
M_u : float : "applied ultimate bending moment"

# deflections
delta_max : float : "maximum beam deflection"
delta_limit : float : "beam deflection limit"

check : bool : "Is the beam OK? - OK (true) or No Good (false)"

### end of module 

This code doesn't yet do anything, but there is a ton of information being
shared here- information that can be used to write code later, quickly. It
is a very nice module template that could be used over and over again
(writing out the details of actual projects later).

Of course you could write pseudocode containing all of this information
today with regular strings. But the difference is that, if this were to be
a language feature, the information is going to be available
programmatically BEFORE actually implementing anything at all.

If I were in a jupyter notebook, I could write something like this (you
could so similar things a REPL):

[CELL 1]
import cantilevered_beam
print("\n".join(f"{name}:\t{docstring}" for name,  docstring in
cantilevered_beam.__attrdoc__.items()))

...and it would nicely output all the documentation written in the template
module above for my reference, and in the next cell I would interactively
compose code like this (probably copying and pasting from above, and then
typing over the type-hints and docstrings):

[CELL 2]
 my_cantilevered_beam.py #
"""A check for a cantilevered beam design."""

# material properties
E_s = 29_000  # ksi
F_y = 50  # ksi

# beam properties
I_beam = 1_830  # in^4 for W 24x68 beam
S_beam = 154  # in^3 for W 24x68 beam
L_beam = 190.25  # inches
phi_M = 0.9 * F_y * S_beam  # kip-in

P_u = 4.947  # kips
M_u = P_u *  L_beam  # kip-in
delta_max = P_u * L_beam**3 / (48 * E_s * I_beam)  # inches
delta_limit = 1  # inches
check = True if M_u <= phi_M and delta_max <= delta_limit else False

Could I accomplish something similar today using regular strings? Sure. But
it seems to me that having an official way to document attributes/members
has been an oft requested feature, and I really like imaging the kinds of
things I would be inclined to do with it if it were supported in the
language (rather than something I was off doing on my own as a
low-experience programmer, and wondering if it was dumb or a mistake).

---
Ricky.

"I've never met a Kentucky man who

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2

2021-12-08 Thread Steven D'Aprano

On Wed, Dec 08, 2021 at 11:50:55AM -, tmkehrenb...@gmail.com wrote:
> A few weeks ago, I proposed on this mailing list to write docstrings for
> class attributes like this:
> 
> @dataclass
> class A:
> x: int
> """Docstring for x."""
> 
> The main criticism, I think, was that it is weird to have the docstring
> *below* the attribute.

Why would it be weird to put a docstring below the object that owns it? 
We already do that for class and function signatures.

def func():
"""Docstring is below the signature"""

I would strongly prefer to have the docstring follow the attribute, like 
function and class docstrings follow the signature, not lead it.

> To solve this problem, I propose to introduce a new kind of string: a
> d-string

Not another string prefix :-(

> A d-string is a normal string, except that it is stored in __attrdoc__ 
> when used inside a class. It is stored with the name of the variable 
> *below* it as the key.

I'd like to suggest an alternative syntax that doesn't need a new string 
prefix, although it does require new syntax. I think it is fully 
backwards compatible.

Rather than implicitly linking a bare string to the following (or 
previous?) attribute, this would explicitly link them by putting them on 
the same line (at least for short docstrings).

Full declaration syntax becomes:

name : Type = value : docstring_expression

with the various parts being optional. Example:

@dataclass
class InventoryItem:
"""Class for keeping track of an item in inventory."""

name: str : "Short name of the item"
unit_price: float : "Price per unit in dollar."
quantity_on_hand: int = 0 : (
"Available quantity currently in the warehouse.")

The docstring expression could evaluate to any value, not necessarily a 
string, but conventionally one would use a string, and I expect style 
guides would discourage or ban non-strings. (But see below for a 
possible objection.)

Aside from the name, each part of the syntax is optional:

# Type hint only.
name: Type

# Assignment only.
name = value

# Declare a type hint, and a docstring.
name: Type : docstring

# Assign a value, and a docstring, with no type hint.
name = value : docstring

# And all three: type hint, value and docstring.
name: Type = value : docstring

If this makes sense, one might even have an attribute with no type hint 
or value, only a docstring:

name : : docstring

which would assign the docstring to `__attrdoc__` but otherwise have no 
effect.

Formatting long lines could be done either with parentheses as in the 
quantity_on_hand example above, or backslashes for those who don't hate 
them:

quantity_on_hand: \
int = 0 : \
"Available quantity currently in the warehouse."

Strings of course can use any prefix, raw strings, f-strings, triple- 
quoted strings, whatever you like. Since the docstring part is just an 
expression, it doesn't need to be a string literal.

If people object to the "arbitrary expression" part as too YAGNI, we 
could limit the docstring part to be:

* a string literal (using any prefix);
* an f-string;
* or one of the above, parenthesed.

And maybe even drop the f-string and just allow an string literal, 
optionally with parentheses.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3KNFCJF2T6KOUHM6YMVXTCS6C2OHDQGG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Runtime-accessible attribute docstrings – take 2