[
https://issues.apache.org/jira/browse/BEAM-7850?focusedWorklogId=360610&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-360610
]
ASF GitHub Bot logged work on BEAM-7850:
----------------------------------------
Author: ASF GitHub Bot
Created on: 17/Dec/19 00:09
Start Date: 17/Dec/19 00:09
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #10183: [BEAM-7850]
Makes environment ID a top level attribute of PTransform.
URL: https://github.com/apache/beam/pull/10183#discussion_r358529091
##########
File path: sdks/python/apache_beam/pipeline.py
##########
@@ -972,15 +986,27 @@ def transform_to_runner_api(transform, # type:
Optional[ptransform.PTransform]
return transform.to_runner_api(context, has_parts=bool(self.parts))
# Iterate over inputs and outputs by sorted key order, so that ids are
# consistently generated for multiple runs of the same pipeline.
+ transform_spec = transform_to_runner_api(self.transform, context)
+ environment_id = self.environment_id
+ transform_urn = transform_spec.urn if transform_spec else None
+ if (not environment_id and transform_urn and
+ (transform_urn in KNOWN_SDK_TRANSFORMS)):
+ environment_id = context.default_environment_id()
+
+ if (not environment_id and
Review comment:
Why is this needed given the above?
(Also, if it is needed, should it be nested for clarity, i.e.
```
if not environment_id:
if transform_urn and transform_urn in KNOWN_SDK_TRANSFORMS:
environment_id = context.default_environment_id()
elif ...:
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 360610)
Time Spent: 2h 10m (was: 2h)
> Make Environment a top level attribute of PTransform
> ----------------------------------------------------
>
> Key: BEAM-7850
> URL: https://issues.apache.org/jira/browse/BEAM-7850
> Project: Beam
> Issue Type: Sub-task
> Components: beam-model
> Reporter: Chamikara Madhusanka Jayalath
> Assignee: Chamikara Madhusanka Jayalath
> Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Currently Environment is not a top level attribute of the PTransform (of
> runner API proto).
> [https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/beam_runner_api.proto#L99]
> Instead it is hidden inside various payload objects. For example, for ParDo,
> environment will be inside SdkFunctionSpec of ParDoPayload.
> [https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/beam_runner_api.proto#L99]
>
> This makes tracking environment of different types of PTransforms harder and
> we have to fork code (on the type of PTransform) to extract the Environment
> where the PTransform should be executed. It will probably be simpler to just
> make Environment a top level attribute of PTransform.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
