Yifan Mai created BEAM-10781:
--------------------------------
Summary: Allow PTransformOverride to use AppliedPTransform when
constructing replacement transform
Key: BEAM-10781
URL: https://issues.apache.org/jira/browse/BEAM-10781
Project: Beam
Issue Type: New Feature
Components: sdk-py-core
Reporter: Yifan Mai
In the Python SDK, {{PTransformOverride.get_replacement_transform}} takes in a
[{{PTransform}}|https://github.com/apache/beam/blob/cfa448d121297398312d09c531258a72b413488b/sdks/python/apache_beam/pipeline.py#L1321]
rather than an {{AppliedPTransform}}. This is inconvenient because the
{{PTransformOverride}} may want to inspect the properties of the
{{AppliedPTransform}} when constructing the replacement {{PTransform}}. This is
already [allowed in
Java|https://github.com/apache/beam/blob/cfa448d121297398312d09c531258a72b413488b/sdks/java/core/src/main/java/org/apache/beam/sdk/runners/PTransformOverrideFactory.java#L49].
The proposal to do this in a backward compatible way is as follows:
* Introduce
{{PTransformOverride.get_replacement_transform_for_applied_ptransform(self,
applied_ptransform}} and call it instead of
{{PTransformOverride.get_replacement_transform}}. The user can override this to
provide a replacement {{PTransform}} using the input {{AppliedPTransform}}.
* Provide a default implementation of
{{PTransformOverride.get_replacement_transform_for_applied_ptransform()}} that
calls {{PTransformOverride.get_replacement_transform()}} for backwards
compatibility
* Deprecate {{PTransformOverride.get_replacement_transform()}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
