> > Probably the best you can do is write your own recursive > isinstance-lookalike that has the behavior you need for validating JSON. >
This is already done in the form of pydantic <https://pydantic-docs.helpmanual.io/> which is the backbone of one of the fastest growing python web frameworks, fastAPI <https://fastapi.tiangolo.com/>. (Admission: I'm one of the authors of pydantic) Pydantic doesn't yet support TypedDict, but there's an issue <https://github.com/samuelcolvin/pydantic/issues/760> to implement it. --- More generally, I think runtime type checking was never the intention for type hints, in fact I think Guido specifically said somewhere that runtime type checking was not an intended use case (am I right?). However pydantic and a few other projects seem to me to have been very successful in using them for that anyway. I'd love it if python's developers could be a bit more supportive to runtime type inspection in future. Match statements look very interesting, thanks for alerting me to the PEP. Samuel -- Samuel Colvin On Sun, 22 Nov 2020 at 18:17, Guido van Rossum <gu...@python.org> wrote: > You nerd-sniped me there. :-) > > I think this is perhaps too complicated to attempt to make it all work. > > - We intentionally don't support things like `isinstance(x, List[str])` > because that would require checking all the items with `isinstance(item, > str)`, and that seems a speed trap. Reverting this decision would be hard > work. > > - `isinstance(3, float)` returns False, because at runtime int and float > are distinct types, even though static type checkers regard int as a > subtype of float (with some caveats, at least in mypy's case, but that's > basically how it works). Changing this behavior at runtime will break tons > of existing code that tries to distinguish ints from floats but checks for > float first (since everyone "knows" they are distinct types), so this would > be even harder to get through than the previous bullet. > > - Currently static type checkers don't allow defining any methods (even > class methods) in TypedDict instances, so you can't manually an > `__instancecheck__` method to a TypedDict class. (But we could add it to > typing.TypedDict of course.) > > - There's also the issue that bool is a subtype of int. Again, very hard > to change that without breaking code. > > - Some static type checkers (mypy, but not pyright -- haven't tried others > yet) disallow `isinstance(x, SomeTypedDict)` -- presumably because they are > aware of the problems above. > > Probably the best you can do is write your own recursive > isinstance-lookalike that has the behavior you need for validating JSON. > But then you're no better off than any other JSON validation framework (and > I expect there already to be some that introspect TypedDict subclasses). > > I suppose you could come up with some mechanism whereby you can create a > parallel hierarchy of classes that do support isinstance(), so you could > write e.g. > > ICircle = make_interface(Circle) > IRect = make_interface(Rect) > # etc. > > def draw_shape(): > match request.json: > case ICircle(center, radius): ... > case IRect(x, y, width, height): ... > ... > > but this loses much of the original attractiveness. > > --Guido > > On Sat, Nov 21, 2020 at 10:46 PM David Foster <davidf...@gmail.com> wrote: > >> I am excited about the potential of the new PEP 634-636 "match" statement >> to >> match JSON data received by Python web applications. Often this JSON data >> is >> in the form of structured dictionaries (TypedDicts) containing Lists and >> other primitives (str, float, bool, None). >> >> PEP 634-636 already contain the ability to match all of those underlying >> data >> types except for TypedDicts, so I'd like to explore what it might look >> like to >> match a TypedDict... >> >> Consider an example web application that wants to provide a service to >> draw >> shapes, perhaps on a connected physical billboard. >> >> The service has a '/draw_shape' endpoint which takes a JSON object >> (a Shape TypedDict) describing a shape to draw: >> >> from bottle import HTTPResponse, request, route >> from typing import Literal, TypedDict, Union >> >> class Point2D(TypedDict): >> x: float >> y: float >> >> class Circle(TypedDict): >> type: Literal['circle'] >> center: Point2D # has a nested TypedDict! >> radius: float >> >> class Rect(TypedDict): >> type: Literal['rect'] >> x: float >> y: float >> width: float >> height: float >> >> Shape = Union[Circle, Rect] # a Tagged Union / Discriminated Union >> >> @route('/draw_shape') >> def draw_shape() -> None: >> match request.json: # a Shape? >> ... >> case _: >> return HTTPResponse(status=400) # Bad Request >> >> Now, what syntax could we have at the ... inside the "match" statement to >> effectively pull apart a Shape? >> >> The current version of PEP 634-636 would require duplicating all the keys >> and value types that are defined in Shape's underlying Circle and Rect >> types: >> >> match request.json: # a Shape? >> case {'type': 'circle', 'center': {'x': float(), 'y': >> float()}, \ >> radius: float()} as circle: >> draw_circle(circle) # type is inferred as Circle >> case {'type': 'rect', 'x': float(), 'y': float(), \ >> 'width': float(), 'height': float()} as rect: >> draw_rect(rect) # type is inferred as Rect >> case _: >> return HTTPResponse(status=400) # Bad Request >> >> Wouldn't it be nicer if we could use class patterns instead? >> >> match request.json: # a Shape? >> case Circle() as circle: >> draw_circle(circle) >> case Rect() as rect: >> draw_rect(rect) >> case _: >> return HTTPResponse(status=400) # Bad Request >> >> Now that syntax almost works except that Circle and Rect, being >> TypedDicts, >> do not support isinstance() checks. PEP 589 ("TypedDict") did not define >> how >> such isinstance() checks should work initially because it's somewhat >> complex >> to specify. From the PEP: >> >> > In particular, TypedDict type objects cannot be used in isinstance() >> tests >> > such as isinstance(d, Movie). The reason is that there is no existing >> > support for checking types of dictionary item values, since >> isinstance() >> > does not work with many PEP 484 types, including common ones like >> List[str]. >> > [...] >> > This is consistent with how isinstance() is not supported for >> List[str]. >> >> Well, what if we (or I) took the time to specify how isinstance() worked >> with >> TypedDict? Then the match syntax above with TypedDict as a class pattern >> would work! >> >> Refining the example above even further, it would be nice if we didn't >> have to >> enumerate all the different types of Shapes directly in the >> match-statement. >> What if we could match on a Shape directly? >> >> match request.json: # a Shape? >> case Shape() as shape: >> draw_shape(shape) >> case _: >> return HTTPResponse(status=400) # Bad Request >> >> Now for that syntax to work it must be possible for an isinstance() check >> to >> work on a Shape, which is defined to be a Union[Circle, Rect], and >> isinstance() >> checks also aren't currently defined for Union types. So it would be >> useful >> to define isinstance() for Union types as well. >> >> Of course that match-statement is now simple enough to just be rewriten >> as an >> if-statement: >> >> if isinstance(shape := request.json, Shape): >> draw_shape(shape) >> else: >> return HTTPResponse(status=400) # Bad Request >> >> Now *that* is a wonderfully short bit of parsing code that results in >> well-typed objects as output. 🎉 >> >> So to summarize, I believe it's possible to support really powerful >> matching >> on JSON objects if we just define how isinstance() should work with a >> handful >> of new types. >> >> In particular the above example would work if isinstance() was defined >> for: >> * Union[T1, T2], Optional[T] >> * T extends TypedDict >> * Literal['foo'] >> >> For arbitrary JSON beyond the example, we'd also want to support >> isinstance() for: >> * List[T] >> >> We already support isinstance() for the other JSON primitive types: >> * str >> * float >> * bool >> * type(None) >> >> So what do folks think? If I were to start writing a PEP to extend >> isinstance() >> to cover at least the above cases, would that be welcome? >> >> -- >> David Foster | Seattle, WA, USA >> Contributor to TypedDict support for mypy >> _______________________________________________ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/Y2EJEZXYRKCXH7SP5MDF3PT2TYIB7SJS/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > --Guido van Rossum (python.org/~guido) > *Pronouns: he/him **(why is my pronoun here?)* > <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> > _______________________________________________ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/AZ2JBLZGMA4ZIDTTC5J3F55XZTV43HOS/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ME3PWLDBYLQBA7KQJF44NKDP6NYVE3S7/ Code of Conduct: http://python.org/psf/codeofconduct/
