[
https://issues.apache.org/jira/browse/BEAM-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16847151#comment-16847151
]
Udi Meiri commented on BEAM-7060:
---------------------------------
I was thinking of using Beam runtime checks (which only happen during pipeline
construction) and supporting static checks. By supporting I mean adding adding
annotations to Beam (note that Beam still support Python 2 so inline
annotations aren't possible yet).
We will support Beam-style type hints (on pipeline objects such as DoFn), but
also use PEP 484 annotations if provided (such as on DoFn.process()).
I do want to get static type checks working on Beam and Beam user code.
I'm not convinced that it'd be simple to switch over entirely to static checks.
Some things to consider:
- Beam uses type hints to determine which coders to use for pipeline elements.
- Beam constructs pipelines using the pipe symbol/apply() method. The operands
are classes (PTransforms), which don't get type hints in Python (AFAIK).
- Some PTransforms are composite transforms that expand() into sub-pipelines.
Would mypy handle this case? It seems it would have to simulate the pipeline
construction process.
> Design Py3-compatible typehints annotation support in Beam 3.
> -------------------------------------------------------------
>
> Key: BEAM-7060
> URL: https://issues.apache.org/jira/browse/BEAM-7060
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Udi Meiri
> Priority: Major
>
> Existing [Typehints implementaiton in
> Beam|[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/typehints/
> ] heavily relies on internal details of CPython implementation, and some of
> the assumptions of this implementation broke as of Python 3.6, see for
> example: https://issues.apache.org/jira/browse/BEAM-6877, which makes
> typehints support unusable on Python 3.6 as of now. [Python 3 Kanban
> Board|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245&view=detail]
> lists several specific typehints-related breakages, prefixed with "TypeHints
> Py3 Error".
> We need to decide whether to:
> - Deprecate in-house typehints implementation.
> - Continue to support in-house implementation, which at this point is a stale
> code and has other known issues.
> - Attempt to use some off-the-shelf libraries for supporting
> type-annotations, like Pytype, Mypy, PyAnnotate.
> WRT to this decision we also need to plan on immediate next steps to unblock
> adoption of Beam for Python 3.6+ users. One potential option may be to have
> Beam SDK ignore any typehint annotations on Py 3.6+.
> cc: [~udim], [~altay], [~robertwb].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
