[
https://issues.apache.org/jira/browse/BEAM-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822240#comment-16822240
]
Kyle Weaver edited comment on BEAM-6959 at 4/19/19 10:45 PM:
-------------------------------------------------------------
I was able to get the Go integration tests running successfully against a Flink
job server that I manually started in a separate terminal.
The tricky thing here is how the job server will be created and shut down in
the actual test environment. In Java portable validatesRunner tests, a GRPC
server is created wrapping the job service [1]. In Python, the job server is
brought up in a container via some bespoke code [2]. In both cases, the job
server is actually brought up in the native SDK itself. The advantage here is
that information about the job endpoint, such as its dynamically assigned port,
is readily available to the SDK.
As far as I can tell, the Go SDK does not have any such code allowing us to
start and stop a job server. The alternatives here seem to be a) doing
something like what Python did with containerizing it or b) managing the job
server from Gradle or bash instead.
[1]
[https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/jobsubmission/JobServerDriver.java]
[2]
[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/job_server.py]
was (Author: ibzib):
I was able to get the Go integration tests running successfully against a Flink
job server that I manually started in a separate terminal.
The tricky thing here is how the job server will be created and shut down in
the actual test environment. In Java portable validatesRunner tests, a GRPC
server is created wrapping the job service [1]. In Python, the job server is
containerized via some bespoke code [2]. In both cases, the job server is
actually brought up in the native SDK itself. The advantage here is that
information about the job endpoint, such as its dynamically assigned port, is
readily available to the SDK.
As far as I can tell, the Go SDK does not have any such code allowing us to
start and stop a job server. The alternatives here seem to be a) doing
something like what Python did with containerizing it or b) managing the job
server from Gradle or bash instead.
[1]
[https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/jobsubmission/JobServerDriver.java]
[2]
[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/job_server.py]
> Run Go SDK Post Commit tests against the Flink Runner.
> -------------------------------------------------------
>
> Key: BEAM-6959
> URL: https://issues.apache.org/jira/browse/BEAM-6959
> Project: Beam
> Issue Type: Sub-task
> Components: runner-flink, sdk-go, testing
> Reporter: Robert Burke
> Assignee: Kyle Weaver
> Priority: Minor
>
> See parent task BEAM-6958
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
