[
https://issues.apache.org/jira/browse/BEAM-7746?focusedWorklogId=339116&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339116
]
ASF GitHub Bot logged work on BEAM-7746:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Nov/19 01:40
Start Date: 06/Nov/19 01:40
Worklog Time Spent: 10m
Work Description: chadrik commented on pull request #9915: [BEAM-7746]
Add python type hints (part 1)
URL: https://github.com/apache/beam/pull/9915#discussion_r342882155
##########
File path: sdks/python/apache_beam/coders/coder_impl.py
##########
@@ -74,53 +86,66 @@
else:
is_compiled = False
fits_in_64_bits = lambda x: -(1 << 63) <= x <= (1 << 63) - 1
+
# pylint: enable=wrong-import-order, wrong-import-position, ungrouped-imports
_TIME_SHIFT = 1 << 63
MIN_TIMESTAMP_micros = MIN_TIMESTAMP.micros
MAX_TIMESTAMP_micros = MAX_TIMESTAMP.micros
+IterableStateReader = Callable[[bytes, 'CoderImpl'], Iterable]
+IterableStateWriter = Callable[[Iterable, 'CoderImpl'], bytes]
+Observables = List[Tuple[observable.ObservableMixin, 'CoderImpl']]
class CoderImpl(object):
"""For internal use only; no backwards-compatibility guarantees."""
def encode_to_stream(self, value, stream, nested):
+ # type: (Any, create_OutputStream, bool) -> None
Review comment:
My intent was that mypy would use the slow_stream implementation of these,
which _are_ types. But I realized when looking at this that
`create_InputStream` is being treated as `Any` because mypy can't inspect the
`stream` c-extension module (mypy always uses the first import location of an
object, unless `TYPE_CHECKING` is used).
The simple/obvious approach works, but is pretty verbose:
```python
if TYPE_CHECKING:
from apache_beam.transforms.window import IntervalWindow
from .slow_stream import InputStream as create_InputStream
from .slow_stream import OutputStream as create_OutputStream
from .slow_stream import ByteCountingOutputStream
from .slow_stream import get_varint_size
else:
# pylint: disable=wrong-import-order, wrong-import-position,
ungrouped-imports
try:
from .stream import InputStream as create_InputStream
from .stream import OutputStream as create_OutputStream
from .stream import ByteCountingOutputStream
from .stream import get_varint_size
# Make it possible to import create_InputStream and other cdef-classes
# from apache_beam.coders.coder_impl when Cython codepath is used.
globals()['create_InputStream'] = create_InputStream
globals()['create_OutputStream'] = create_OutputStream
globals()['ByteCountingOutputStream'] = ByteCountingOutputStream
except ImportError:
from .slow_stream import InputStream as create_InputStream
from .slow_stream import OutputStream as create_OutputStream
from .slow_stream import ByteCountingOutputStream
from .slow_stream import get_varint_size
if False: # pylint: disable=using-constant-test
# This clause is interpreted by the compiler.
from cython import compiled as is_compiled
else:
is_compiled = False
fits_in_64_bits = lambda x: -(1 << 63) <= x <= (1 << 63) - 1
```
Here's another approach (still verbose but less redundant):
```python
try:
from . import stream
except ImportError:
SLOW_STREAM = True
else:
SLOW_STREAM = False
if TYPE_CHECKING or SLOW_STREAM:
from .slow_stream import InputStream as create_InputStream
from .slow_stream import OutputStream as create_OutputStream
from .slow_stream import ByteCountingOutputStream
from .slow_stream import get_varint_size
if False: # pylint: disable=using-constant-test
# This clause is interpreted by the compiler.
from cython import compiled as is_compiled
else:
is_compiled = False
fits_in_64_bits = lambda x: -(1 << 63) <= x <= (1 << 63) - 1
else:
# pylint: disable=wrong-import-order, wrong-import-position,
ungrouped-imports
from .stream import InputStream as create_InputStream
from .stream import OutputStream as create_OutputStream
from .stream import ByteCountingOutputStream
from .stream import get_varint_size
# Make it possible to import create_InputStream and other cdef-classes
# from apache_beam.coders.coder_impl when Cython codepath is used.
globals()['create_InputStream'] = create_InputStream
globals()['create_OutputStream'] = create_OutputStream
globals()['ByteCountingOutputStream'] = ByteCountingOutputStream
# pylint: enable=wrong-import-order, wrong-import-position,
ungrouped-imports
```
Opinion?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 339116)
Time Spent: 19h 20m (was: 19h 10m)
> Add type hints to python code
> -----------------------------
>
> Key: BEAM-7746
> URL: https://issues.apache.org/jira/browse/BEAM-7746
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py-core
> Reporter: Chad Dombrova
> Assignee: Chad Dombrova
> Priority: Major
> Time Spent: 19h 20m
> Remaining Estimate: 0h
>
> As a developer of the beam source code, I would like the code to use pep484
> type hints so that I can clearly see what types are required, get completion
> in my IDE, and enforce code correctness via a static analyzer like mypy.
> This may be considered a precursor to BEAM-7060
> Work has been started here: [https://github.com/apache/beam/pull/9056]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
