Daniel Oliveira created BEAM-11574:
--------------------------------------
Summary: Enable x-lang on Dataflow side for integration tests
Key: BEAM-11574
URL: https://issues.apache.org/jira/browse/BEAM-11574
Project: Beam
Issue Type: Bug
Components: cross-language, sdk-go
Reporter: Daniel Oliveira
Assignee: Daniel Oliveira
Dataflow x-lang ValidatesRunner tests are failing with the following error:
{noformat}
2021/01/05 21:57:59 Staged model pipeline:
gs://clouddfe-danoliveira/temp-storage-go-end-to-end-tests/staging-validatesrunner-test/go-1-1609912673288691694/model
ptest.go:97: Failed to execute job: invalid Combine payload , missing
Accumulator Coder unique_name:"PjLfVbFvtsExternal"
spec:{urn:"beam:transform:combine_per_key:v1"
payload:"\n\xe1\x01\n\x19beam:combinefn:javasdk:v1\x1a\xc3\x01\x82SNAPPY\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\xaf\x9c\x02\xf0C\xac\xed\x00\x05sr\x00,org.apache.beam.sdk.transforms.Sum$SumLongFn]\x12ܾ\"M\x80\xd5\x02\x00\x00xr\x00:ov;\x004Combine$Binary\r\x0e\x00L\x05I
-\r\xda\x12\x13\r\x9b\xdd\x02\x05I\x000z\x84\x00\r;\x00$\r\x08$Fn\x1f4³>֧\x94\t?\x00D\x96?\x00PFnBase$AbstractGlobal\x15S0\xa6BY\xe0V\xebн\x02\x00\x00xp\x12\x17PjLfVbFvtsDelegateCoder"}
subtransforms:"PjLfVbFvtsExternal/GroupByKey"
subtransforms:"PjLfVbFvtsExternal/Combine.GroupedValues"
inputs:{key:"org.apache.beam.sdk.values.PCollection.<init>:402#bb20b45fd4d95138"
value:"n3"} outputs:{key:"output"
value:"PjLfVbFvtsExternal/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anonymous).output"}
display_data:{urn:"beam:display_data:labelled_string:v1"
payload:"\n\x08Combiner\x12,org.apache.beam.sdk.transforms.Sum$SumLongFn"}
environment_id:"PjLfVbFvtsbeam:env:docker:v1"
caused by:
unmarshalling coder PjLfVbFvtsDelegateCoder
could not unmarshal coder from spec:{urn:"beam:coders:javasdk:0.1"
payload:"\x82SNAPPY\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x01\x82\x96\x05\xf0>\xac\xed\x00\x05sr\x00(org.apache.beam.sdk.coders.DelegateCoder~\x10P\xea\xdbb4\xdc\x02\x00\x04L\x00\x05c\x01\"
t\x00\"Lorg/a\x05?\x00/\x01?\x10/sdk/\t?\x00/\x0570;L\x00\x06fromFnt\x009n.\x002m\x00\xc0$CodingFunction;L\x00\x04toFnq\x00~\x00\x02L\x00\x0etypeDescriptort\x00+LN\x87\x00\x1cvalues/T2-\x00\x10;xr\x00&j\xf4\x00\x14Custom\x05\xbb8j\xb0\x08\x9d\x0b;\x1d\x0b\x02\x00\x00xr\x00
n5\x00!\x0e\x1cC\xddՉ\xae\xbc~\xf8\x01/\x10psr\x00'j1\x00\x18VarLong\x058\x1c\\l\xd6\xc3M\xfeMi\x018\x08r\x00*j6\x00$Structured\x059\x1cs\xbf\x12\x0e\xd5\xd46\x11\x019!\x1a!\xcf\x00QN?\x00`transforms.Combine$Binary\r\x0e\x01\x8b\x18Fn$From\x01\x0b\x00C2|\x01\x1c\x1f\xb1\x85\x04\x1a\xae\xc7J\x01f\x01\xd7\x00O\xeab\x00\x04ToF`\x004\xa1\x9f\xc1\xd98\xdc\x1d\xcb\x02\x00\x00xpp"},
unknown URN beam:coders:javasdk:0.1
{noformat}
This seems to imply that the bundles intended to be sent to the Java SDK are
still being sent to the Go SDK (these are Go SDK errors), so it seems that
cross-language functionality still needs to be enabled for the integration
tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
