On Sat, Dec 11, 2021, 10:58 PM Steven D'Aprano <st...@pearwood.info> wrote:
> On Sat, Dec 11, 2021 at 05:02:39PM -0800, Christopher Barker wrote: > > On Sat, Dec 11, 2021 at 3:03 PM Steven D'Aprano <st...@pearwood.info> > wrote: > > > > Didn't we decide there was an existing feature for this, no need for > > > new syntax? > > > Well, no. In fact, you could always put anything you wanted into an > > annotation: > > Right, and we want to put a documentation string in it! :-) > > > > Anyway, take a look at the docs for Annotated: > > > > https://docs.python.org/3/library/typing.html#typing.Annotated > > > > It has a number of possible unspecified uses, but fundamentally, it's > about > > the adding information to the type, not to the attribute -- e.g. not > really > > intended for docstrings. > > I don't think that the docs argue against this. Annotated is documented > as the way to put arbitrary metadata into an annotation. What you do > with it is up to the consumer of the metadata. > > The docs talk about this associating the metadata with the type, but > that's because this is the *typing* module and *type* hints are still > the primary use-case for annotations. > > And despite what the docs say, the metadata is actually, literally, > associated with the variable (or its name at least). The annotation that > gets stored in __annotations__ is **not** > > {T: "metadata"} > > as a naive reading of the docs might trick you into believing, but > > {"var": Annotated[T, "metadata"]} > > > In any case, we can still use annotations for anything we like, and > while typing is definitely and certainly the primary use for them, there > will always be a way to opt-out. (Just don't run mypy!) > > Don't need type hints at all? Great, put your doc string as the > annotation. > > μ: 'Number of steps from x0 to the start of the cycle.' > λ: 'Length (period) of the cycle.' > y: 'Value of f at the start of the cycle.' = None > > You might want to annotate the class with `no_type_check` if there is > any chance the user might want to run mypy over your module, but that's > entirely up to you. > > You do want type hints? Great, use Annotated. If there's an attribute > that you don't want to give a type hint to, just use Any. (A small price > to pay for using an existing feature, instead of waiting until Python > 3.11 or 3.27 when it gets added to the core language.) > > μ: Annotated[int, > 'Number of steps from x0 to the start of the cycle.'] > λ: Annotated[int, 'Length (period) of the cycle.'] > y: Annotated[Any, > 'Value of f at the start of the cycle.' > ] = None > > No new syntax required. No bare strings floating around. You can > generate the doc strings dynamically, they don't have to be literals. > > lunch: Annotated[Meal, ' '.join(['spam']*SPAM_COUNT)] > > > > Sure, typing.Annotated *could* be used that way, but I don't think we can > > say that the problem is solved. > > I think that from a language standpoint, we absolutely can say that the > problem is solved. We don't need new syntax. > > We *might* want to standardise on a decorator or other tool to extract > those docstrings, possibly sticking them in a new __attrdoc__ dunder. > Being a dunder, we should get the core dev's approval for this, even if > it is a third-party tool that uses it. > > (I assume we don't already use __attrdoc__ for something?) > > If the first use of this is dataclasses, then that would be de facto > core dev approval. > > So we have a tool that takes an annotation: > > var: Annotated[T, docstring, *args] > > extracts out the docstring to an __attrdoc__ dunder: > > # Class.__attrdoc__ > {'var': docstring} > > and removes the annotation from the annotation: > > # Class.__annotation__ > {'var': Annotated[T, *args]} # or just T if *args is empty? > > Dataclasses can automatically apply this tool (subject to backwards > compatibility constraints), or we can apply it as a decorator to any > other class we want. > > So we've gone from the *hard* problem of making a language change (new > syntax, build the functionality into the interpreter) to the *easy* > problem of writing a decorator. > > > > Heck you could just as easily provide a tuple for the annotation: > > > > In [17]: class C: > > ...: x: ("Doc string", int) = 123 > > ...: > > > Sure, annotations can be anything you like. But external static typing > tools may not support this, and will be confused or break. Same for > third-party runtime tools. > > Best practice is to stick as closely to the standard type-hinting > behaviour as you can, or to completely opt-out with no_type_hints. > > > > The use cases for annotations are a bit up in the air at the moment -- > see > > the ongoing discussion on python-dev. But I don't think there's any doubt > > that any future endorsed use of them will be compatible with static > typing > > (if not restricted to it), so extending the use of annotations for > > docstrings would take a bit of work to define and accept that use. > > Annotated is explicitly documented as *arbitrary* metadata that is > entirely up to the application to interpret. There's no reason to think > that will change. > > > > Perhaps a better way to do that than to use Annotated would be to > introduce > > new "thing", maybe in teh typoing module, like "Documented": > > > > class C: > > x: Documented(doc_string="this is the doc string", type=int) = 123 > > Now you're just arguing about the colour of the bike-shed :-) > > Fine, if we want to increase the complexity of *every* tool that > operates on annotations, to specially recognise this new pseudo-type, we > could make them recognise > > Documented["docstring", T] > > where T is optional and to be interpreted as Any if missing. > > But I suspect that the type checkers will say "Just use Annotated" but > what do I know? > > Having introduced that special Documented type-hint, we still need to > modify dataclasses to extract the docstring from it, and we still need > to make it available to other classes via a decorator. > > So apart from the spelling, which I don't think is a win, we gain > nothing. > > > -- > Steve But Steve, since the most utilized documentation tool in the python universe, sphinx, doesn't look at Annotated or in the lowercase-annotation part of an expression for this and instead looks at a bare string below the member definition, and since I presume it had been doing this for a long time (before Annotated was added not long ago, and probably before the annotation too?), doesn't this mean that right now there are at least three competing ways being used to provide attribute docstrings? It really seems like it would be good for the language to standardize this. Just like function or module docstrings were standardized long ago. As far as using the lowercase-annotation for the docstring: in a world of more and more type hinted python (to be clear: I don't always use type hints but I do like this world), if you want to make you don't create a problem later, using the annotation slot for your docstring: x: "spam" ... isn't really an option. So you have little choice but to use typing.Annotated right now (unless you want to take advantage of the sphinx way of doing it). But the problem with Annotated is Matt, imo: - it's a rather long spelling - you have to import it from typing - the help output isn't very useful when you use Annotated right now (this can be improved but probably only if Annotated is officially specified as the "right place") - without an official blessed agreement that the existing Annotated feature ought to be used for docstrings, there is little reason for other 3rd party help/documentation tools to look in that location for these docstrings I just think there's more to do here than add a decorator.
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MGNZ6JAWNI2RDII5DPWWMMSXWYYHTQLC/ Code of Conduct: http://python.org/psf/codeofconduct/
