[
https://issues.apache.org/jira/browse/BEAM-11614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263597#comment-17263597
]
Chamikara Madhusanka Jayalath commented on BEAM-11614:
------------------------------------------------------
Actually, ignore last comment. This is just a GBK which is handled by the
runner (hence no need for an SDDK environment). All non-composite transforms
are expected to be handled by an SDK have to have the environment set. Looking
at error seems like some of the external ParDos did not have environment set.
> sql_taxi examples fails on HEAD due to "transform unexpectedly with no env id"
> ------------------------------------------------------------------------------
>
> Key: BEAM-11614
> URL: https://issues.apache.org/jira/browse/BEAM-11614
> Project: Beam
> Issue Type: Bug
> Components: cross-language
> Reporter: Chamikara Madhusanka Jayalath
> Assignee: Brian Hulette
> Priority: P1
> Fix For: 2.28.0
>
>
> For example,
> [https://pantheon.corp.google.com/dataflow/jobs/us-central1/2021-01-11_21_50_55-8290792779966349848?project=google.com:clouddfe]
>
> transform unexpectedly with no env id:
> external_1SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_34/Group.CombineFieldsByFields/ToKvs/toRow:
> spec \{ } inputs \{ key:
> "org.apache.beam.sdk.values.PCollection.<init>:402#3d93cb799b3970be" value:
> "external_1SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_33/ParDo(Calc)/ParMultiDo(Calc).output"
> } outputs \{ key:
> "org.apache.beam.sdk.values.PCollection.<init>:402#3d93cb799b3970be" value:
> "external_1SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_33/ParDo(Calc)/ParMultiDo(Calc).output"
> } unique_name:
> "SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_34/Group.CombineFieldsByFields/ToKvs/toRow"
>
> Seems like somehow Sql expansion produces a sub-transform that does not have
> environment ID set.
> This passes for Beam 2.27.0 so should be due to a recent change.
>
> Brian, are you aware of any recent change that could have resulted in this
> breakage ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
