[
https://issues.apache.org/jira/browse/BEAM-11614?focusedWorklogId=535554&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-535554
]
ASF GitHub Bot logged work on BEAM-11614:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Jan/21 17:05
Start Date: 13/Jan/21 17:05
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#13741:
URL: https://github.com/apache/beam/pull/13741#discussion_r556686296
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -443,7 +443,8 @@ def _normalize(coder_proto):
self._external_namespace), (id, self._external_namespace)
new_proto = beam_runner_api_pb2.PTransform(
unique_name=proto.unique_name,
- spec=proto.spec,
+ # If URN is not set this is an empty spec.
+ spec=proto.spec if proto.spec.urn else None,
Review comment:
Unfortunately with Proto3 you cannot really check whether a field is set
or not it seems and Python will give a non zero byte value even for an empty
"spec" field. You can just check the values of the fields.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 535554)
Time Spent: 1h 20m (was: 1h 10m)
> sql_taxi examples fails on HEAD due to "transform unexpectedly with no env id"
> ------------------------------------------------------------------------------
>
> Key: BEAM-11614
> URL: https://issues.apache.org/jira/browse/BEAM-11614
> Project: Beam
> Issue Type: Bug
> Components: cross-language
> Reporter: Chamikara Madhusanka Jayalath
> Assignee: Brian Hulette
> Priority: P1
> Fix For: 2.28.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> For example,
> [https://pantheon.corp.google.com/dataflow/jobs/us-central1/2021-01-11_21_50_55-8290792779966349848?project=google.com:clouddfe]
>
> transform unexpectedly with no env id:
> external_1SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_34/Group.CombineFieldsByFields/ToKvs/toRow:
> spec \{ } inputs \{ key:
> "org.apache.beam.sdk.values.PCollection.<init>:402#3d93cb799b3970be" value:
> "external_1SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_33/ParDo(Calc)/ParMultiDo(Calc).output"
> } outputs \{ key:
> "org.apache.beam.sdk.values.PCollection.<init>:402#3d93cb799b3970be" value:
> "external_1SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_33/ParDo(Calc)/ParMultiDo(Calc).output"
> } unique_name:
> "SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_34/Group.CombineFieldsByFields/ToKvs/toRow"
>
> Seems like somehow Sql expansion produces a sub-transform that does not have
> environment ID set.
> This passes for Beam 2.27.0 so should be due to a recent change.
>
> Brian, are you aware of any recent change that could have resulted in this
> breakage ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
