[
https://issues.apache.org/jira/browse/BEAM-7060?focusedWorklogId=276844&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-276844
]
ASF GitHub Bot logged work on BEAM-7060:
----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Jul/19 17:03
Start Date: 15/Jul/19 17:03
Worklog Time Spent: 10m
Work Description: chadrik commented on pull request #9056: [BEAM-7060]
Add python type hints
URL: https://github.com/apache/beam/pull/9056#discussion_r303539155
##########
File path: sdks/python/apache_beam/examples/wordcount.py
##########
@@ -85,31 +98,66 @@ def run(argv=None):
pipeline_options.view_as(SetupOptions).save_main_session = True
p = beam.Pipeline(options=pipeline_options)
+ reveal_type(p)
+
+ _read = ReadFromText(known_args.input)
+ reveal_type(_read)
+
+ read = 'read' >> _read
+ reveal_type(read)
+
# Read the text file[pattern] into a PCollection.
- lines = p | 'read' >> ReadFromText(known_args.input)
+ lines = p | read
+ reveal_type(lines) # PCollection[unicode*]
+
+ def make_ones(x):
+ # type: (T) -> Tuple[T, int]
Review comment:
Can you explain what you mean?
Let me try to answer what I think you're asking about.
First, even for a simple function like `make_ones`, mypy does not guess at
the arguments and return type.
Next, as a user annotating pipelines I have a few of choices (all of them
are valid):
1. annotate functions, and let mypy infer the types of the `PTransforms`
they are passed to
2. annotate `PTransforms`
3. annotate both, as a way to guard against unintentional incompatibilities
Option 2 would look like this:
```python
def make_ones(x):
return (x, 1)
makemap = beam.Map(make_ones) # type: beam.Map[T, Tuple[T, int]]
```
To be clear, in option 2, mypy is not using the type of `beam.Map` to infer
the type of the `make_ones`, it's simply assuming it's correct, since in our
thought experiment it's unannotated. In other words, the following would not
generate an error because `make_ones` is untyped:
```python
def make_ones(x):
return (x + 1, 1) # x can only be a numeric type!
makemap = beam.Map(make_ones) # type: beam.Map[str, Tuple[str, int]]
```
This module is very messy right now, so I apologize for any confusion that
it's generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 276844)
Time Spent: 1h 40m (was: 1.5h)
> Design Py3-compatible typehints annotation support in Beam 3.
> -------------------------------------------------------------
>
> Key: BEAM-7060
> URL: https://issues.apache.org/jira/browse/BEAM-7060
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Udi Meiri
> Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Existing [Typehints implementaiton in
> Beam|[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/typehints/
> ] heavily relies on internal details of CPython implementation, and some of
> the assumptions of this implementation broke as of Python 3.6, see for
> example: https://issues.apache.org/jira/browse/BEAM-6877, which makes
> typehints support unusable on Python 3.6 as of now. [Python 3 Kanban
> Board|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245&view=detail]
> lists several specific typehints-related breakages, prefixed with "TypeHints
> Py3 Error".
> We need to decide whether to:
> - Deprecate in-house typehints implementation.
> - Continue to support in-house implementation, which at this point is a stale
> code and has other known issues.
> - Attempt to use some off-the-shelf libraries for supporting
> type-annotations, like Pytype, Mypy, PyAnnotate.
> WRT to this decision we also need to plan on immediate next steps to unblock
> adoption of Beam for Python 3.6+ users. One potential option may be to have
> Beam SDK ignore any typehint annotations on Py 3.6+.
> cc: [~udim], [~altay], [~robertwb].
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
