[
https://issues.apache.org/jira/browse/BEAM-13024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Burke updated BEAM-13024:
--------------------------------
Description:
Recently the Go SDK dataflow runner code began adding explicitly set flags to
the pipeline options. This made them available to worker code at execution
time, if queried with `beam.PipelineOptions.Get`.
The proposal is to generalize this code and make use of it in all runners
(including the direct runner). This would make PipelineOptions a useful
concept in the Go SDK.
Flag filtering and adding here:
[https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/dataflow/dataflow.go#L281]
Once implemented for all runners, it should be possible to test use of this in
the integration tests to ensure continued function.
Danny wrote a quick Doc:
https://docs.google.com/document/d/1AyO5SDEd_DzyOyrz_TkyDLUv19eLsKI97vKM8I7fQ9o/edit#
was:
Recently the Go SDK dataflow runner code began adding explicitly set flags to
the pipeline options. This made them available to worker code at execution
time, if queried with `beam.PipelineOptions.Get`.
The proposal is to generalize this code and make use of it in all runners
(including the direct runner). This would make PipelineOptions a useful
concept in the Go SDK.
Flag filtering and adding here:
[https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/dataflow/dataflow.go#L281]
Once implemented for all runners, it should be possible to test use of this in
the integration tests to ensure continued function.
> [Go SDK] Export flags to PipelineOptions on non-dataflow runners.
> -----------------------------------------------------------------
>
> Key: BEAM-13024
> URL: https://issues.apache.org/jira/browse/BEAM-13024
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Danny McCormick
> Priority: P2
>
> Recently the Go SDK dataflow runner code began adding explicitly set flags to
> the pipeline options. This made them available to worker code at execution
> time, if queried with `beam.PipelineOptions.Get`.
> The proposal is to generalize this code and make use of it in all runners
> (including the direct runner). This would make PipelineOptions a useful
> concept in the Go SDK.
> Flag filtering and adding here:
>
> [https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/dataflow/dataflow.go#L281]
> Once implemented for all runners, it should be possible to test use of this
> in the integration tests to ensure continued function.
> Danny wrote a quick Doc:
> https://docs.google.com/document/d/1AyO5SDEd_DzyOyrz_TkyDLUv19eLsKI97vKM8I7fQ9o/edit#
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
