[
https://issues.apache.org/jira/browse/BEAM-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758188#comment-16758188
]
Maximilian Michels commented on BEAM-6549:
------------------------------------------
That's understandable. PipelineOptions are all types of options which are
available in the Pipeline object/struct. The general problem we were trying to
solve is that only "known" Runner-related options will be passed on from the
Python SDK to the Runner. The rest of the options were discarded in the past.
We didn't want to replicate the constantly changing Runner-related options in
all the SDKs. In BEAM-5442, we first implemented an approach to always pass all
options which was problematic because it violated the contract for pipeline
options having to be explicitly defined through interfaces. Thomas' PR
introduced the DescribePipelineOptionsRequest which is implemented by the
Python SDK to retrieve all known options from the JobServer. The assumption is
that the JobServer maintains a list of valid options.
The Go SDK should do something similar if it wants to validate Runner options
provided by the user. For an example of Runner options see
{{FlinkPipelineOptions}} or {{PortablePipelineOptions}}.
> Make use of DescribePipelineOptionsRequest in Go SDK
> ----------------------------------------------------
>
> Key: BEAM-6549
> URL: https://issues.apache.org/jira/browse/BEAM-6549
> Project: Beam
> Issue Type: New Feature
> Components: sdk-go
> Reporter: Maximilian Michels
> Priority: Major
>
> The Python SDK retrieves Runner pipeline options from the JobService:
> https://github.com/apache/beam/pull/7597
> The Go SDK should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
