[
https://issues.apache.org/jira/browse/BEAM-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846309#comment-16846309
]
Chad Dombrova commented on BEAM-7060:
-------------------------------------
Hi all, I'm quite interested in this topic and I have quite a bit of experience
with python typing and mypy.
Firstly, I think it's definitely the right choice to move from the in-house
typehints to the official typing module.
Do you plan on keeping runtime type checks, or to switch over entirely to
static type checks, e.g. using mypy? The advantage of the latter is that users
who are using annotations throughout their code get consistency between the
type validation applied at the interfaces to their transforms (currently
handled by beam typehints) and within their transform functions/classes. Also,
as you're probably well aware, type analysis is fraught with corner cases that
are really hard to get right, so it's nice to be able to lean on a project with
dedicated experts in this area.
If you go with the static approach, you should probably start by adding
annotations to the beam python sdk first, either as .pyi stubs of the public
API or inline. I recommend the latter, so that you can get the benefits in
your own development process. Then, if necessary, you can write a plugin for
mypy to give it insight into any dynamic/magic code that it cannot properly
inspect. As an example, mypy ships with [a
plugin|[https://github.com/python/mypy/blob/master/mypy/plugins/attrs.py]] for
[attrs|[https://github.com/python-attrs/attrs]]. It is also possible for mypy
plugins to be bundled with third-party code and registered with mypy rather
than added directly to the mypy project.
I'm happy to help with the addition of type annotations into the current code
base and add mypy tests, if you all agree that's the direction you'd like to go!
> Design Py3-compatible typehints annotation support in Beam 3.
> -------------------------------------------------------------
>
> Key: BEAM-7060
> URL: https://issues.apache.org/jira/browse/BEAM-7060
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Udi Meiri
> Priority: Major
>
> Existing [Typehints implementaiton in
> Beam|[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/typehints/
> ] heavily relies on internal details of CPython implementation, and some of
> the assumptions of this implementation broke as of Python 3.6, see for
> example: https://issues.apache.org/jira/browse/BEAM-6877, which makes
> typehints support unusable on Python 3.6 as of now. [Python 3 Kanban
> Board|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245&view=detail]
> lists several specific typehints-related breakages, prefixed with "TypeHints
> Py3 Error".
> We need to decide whether to:
> - Deprecate in-house typehints implementation.
> - Continue to support in-house implementation, which at this point is a stale
> code and has other known issues.
> - Attempt to use some off-the-shelf libraries for supporting
> type-annotations, like Pytype, Mypy, PyAnnotate.
> WRT to this decision we also need to plan on immediate next steps to unblock
> adoption of Beam for Python 3.6+ users. One potential option may be to have
> Beam SDK ignore any typehint annotations on Py 3.6+.
> cc: [~udim], [~altay], [~robertwb].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
