[
https://issues.apache.org/jira/browse/BEAM-13439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Chapman updated BEAM-13439:
---------------------------------
Description:
Because the return type of the ptransform_fn is not specified, [certain
typecheckers|https://github.com/microsoft/pyright/issues/2688#issuecomment-990238720]
will assume the type signature remains unchanged. However, ptransform_fn
specifically adjusts the type signature such that the function no longer takes
a pcoll.
This can be corrected by changing the the definition of ptransform_fn from:
{code:java}
def ptransform_fn(fn):
{code}
to:
{code:java}
def ptransform_fn(fn) -> Callable[..., _PTransformFnPTransform]: {code}
This allows the typechecker to understand the return type of a function
decorated with ptransform_fn will instead be a _PTransformFnPTransform type.
Additionally, for Python 3.10,
[ParamSpec|https://www.python.org/dev/peps/pep-0612/#parameter-specification-variables]
could be used to preserve the signature of the internal function as well.
{code:java}
from typing import ParamSpec, TypeVar, Concatenate
P = ParamSpec("P")
T = TypeVar("T")
R = TypeVar("R")
def ptransform_fn(fn: Callable[Concatenate[PCollection[T], P], R]) ->
Callable[[P], _PTransformFnPTransform]: ...{code}
was:
Because the return type of the ptransform_fn is not specified, [certain
typecheckers|https://github.com/microsoft/pyright/issues/2688#issuecomment-990238720]
will assume the type signature remains unchanged. However, ptransform_fn
specifically adjusts the type signature such that the function no longer takes
a pcoll.
This can be corrected by changing the the definition of ptransform_fn from:
{code:java}
def ptransform_fn(fn):
{code}
to:
{code:java}
def ptransform_fn(fn) -> Callable[..., _PTransformFnPTransform]: {code}
This allows the typechecker to understand the return type of a function
decorated with ptransform_fn will instead be a _PTransformFnPTransform type.
Additionally, for Python 3.10, ParamSpec could be used to preserve the
signature of the internal function as well.
{code:java}
from typing import ParamSpec, TypeVar, Concatenate
P = ParamSpec("P")
T = TypeVar("T")
R = TypeVar("R")
def ptransform_fn(fn: Callable[Concatenate[PCollection[T], P], R]) ->
Callable[[P], _PTransformFnPTransform]: ...{code}
> Correct type signature on ptransform_fn to better support typecheckers
> ----------------------------------------------------------------------
>
> Key: BEAM-13439
> URL: https://issues.apache.org/jira/browse/BEAM-13439
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Affects Versions: 2.34.0
> Reporter: Sandy Chapman
> Priority: P2
> Labels: transform, typecheck
>
> Because the return type of the ptransform_fn is not specified, [certain
> typecheckers|https://github.com/microsoft/pyright/issues/2688#issuecomment-990238720]
> will assume the type signature remains unchanged. However, ptransform_fn
> specifically adjusts the type signature such that the function no longer
> takes a pcoll.
> This can be corrected by changing the the definition of ptransform_fn from:
> {code:java}
> def ptransform_fn(fn):
> {code}
> to:
> {code:java}
> def ptransform_fn(fn) -> Callable[..., _PTransformFnPTransform]: {code}
> This allows the typechecker to understand the return type of a function
> decorated with ptransform_fn will instead be a _PTransformFnPTransform type.
> Additionally, for Python 3.10,
> [ParamSpec|https://www.python.org/dev/peps/pep-0612/#parameter-specification-variables]
> could be used to preserve the signature of the internal function as well.
> {code:java}
> from typing import ParamSpec, TypeVar, Concatenate
> P = ParamSpec("P")
> T = TypeVar("T")
> R = TypeVar("R")
> def ptransform_fn(fn: Callable[Concatenate[PCollection[T], P], R]) ->
> Callable[[P], _PTransformFnPTransform]: ...{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
