[
https://issues.apache.org/jira/browse/BEAM-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136100#comment-17136100
]
Robert Burke edited comment on BEAM-7009 at 6/15/20, 6:56 PM:
--------------------------------------------------------------
Just like with proto files, external files are a bit variable and I'd rather
not have these tests be brittle based on where the user's test code is running.
In particular, the package tests shouldn't fail if the code is being used
elsewhere.
As such, we'll likely need to generate the tests from the YAML file, and check
in that test. This runs into the same problem with protos, with the risk of the
tests getting out of date, but coder formats change sparingly, so it's likely
fine with clear enough documentation around the generated tests.
We have other things that call go-generate, so this should work out ok. In
particular, I think I'll generate a package that the actual tests will use,
with the tests being manually written and configured, rather than being clever
in the generator.
Edit,re-reading the JIRA description, I already had the clever idea of putting
it with the integration tests instead, which would resolve the weirdness I
previously described.
was (Author: lostluck):
Just like with proto files, external files are a bit variable and I'd rather
not have these tests be brittle based on where the user's test code is running.
In particular, the package tests shouldn't fail if the code is being used
elsewhere.
As such, we'll likely need to generate the tests from the YAML file, and check
in that test. This runs into the same problem with protos, with the risk of the
tests getting out of date, but coder formats change sparingly, so it's likely
fine with clear enough documentation around the generated tests.
We have other things that call go-generate, so this should work out ok. In
particular, I think I'll generate a package that the actual tests will use,
with the tests being manually written and configured, rather than being clever
in the generator.
> Add Go SDK test for Standard Coders using the yaml data.
> --------------------------------------------------------
>
> Key: BEAM-7009
> URL: https://issues.apache.org/jira/browse/BEAM-7009
> Project: Beam
> Issue Type: Bug
> Components: beam-model, sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: P1
> Labels: stale-P2
>
> The Go SDK doesn't currently do this validation, against the standard yaml
> file. [1]
> The Java and Python equivalents of the test can be found from here [2].
>
> Care would need to be taken so that Beam Go SDK users (such as they are)
> aren't forced to run them, and not have the yaml file to read. I'd suggest
> putting it with the integration tests [3].
> The other thing of note is that the Go SDK has no notion of "nested" vs
> "unnested" coders. All coders are "nested" in the Go SDK, and should have
> their lengths prefixed to them as appropriate.
> 1:
> [https://github.com/apache/beam/blob/master/model/fn-execution/src/main/resources/org/apache/beam/model/fnexecution/v1/standard_coders.yaml]
> 2:
> [https://github.com/apache/beam/search?q=standard_coders.yaml&unscoped_q=standard_coders.yaml]
> 3: [https://github.com/apache/beam/tree/master/sdks/go/test
> |https://github.com/apache/beam/tree/master/sdks/go/test]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
