This is an automated email from the ASF dual-hosted git repository. yhu pushed a commit to branch revert-30490-users/damccorm/yamlOptions in repository https://gitbox.apache.org/repos/asf/beam.git
commit b82603eb01d9ca70dfe1c7454d1aaae4a3768a8a Author: Yi Hu <[email protected]> AuthorDate: Tue Mar 5 23:13:08 2024 -0500 Revert "Document yaml pipeline options (#30490)" This reverts commit 3c46415a3cebc7337ff5fd0d7969836dd2e58694. --- .../www/site/content/en/documentation/sdks/yaml.md | 40 ---------------------- 1 file changed, 40 deletions(-) diff --git a/website/www/site/content/en/documentation/sdks/yaml.md b/website/www/site/content/en/documentation/sdks/yaml.md index d9c3b8db51b..ca87d302380 100644 --- a/website/www/site/content/en/documentation/sdks/yaml.md +++ b/website/www/site/content/en/documentation/sdks/yaml.md @@ -366,8 +366,6 @@ pipeline: config: topic: anotherPubSubTopic format: json -options: - streaming: true ``` Rather than using an explicit `WindowInto` operation, one may instead tag a @@ -394,8 +392,6 @@ pipeline: config: topic: anotherPubSubTopic format: json -options: - streaming: true ``` Note that the `Sql` operation itself is often a from of aggregation, and @@ -421,8 +417,6 @@ pipeline: config: topic: anotherPubSubTopic format: json -options: - streaming: true ``` The specified windowing is applied to all inputs, in this case resulting in @@ -454,8 +448,6 @@ pipeline: windowing: type: fixed size: 60s -options: - streaming: true ``` For a transform with no inputs, the specified windowing is instead applied to @@ -481,8 +473,6 @@ pipeline: config: topic: anotherPubSubTopic format: json -options: - streaming: true ``` One can also specify windowing at the top level of a pipeline (or composite), @@ -509,8 +499,6 @@ pipeline: windowing: type: fixed size: 60 -options: - streaming: true ``` Note that all these windowing specifications are compatible with the `source` @@ -542,9 +530,6 @@ pipeline: windowing: type: fixed size: 5m - -options: - streaming: true ``` @@ -599,31 +584,6 @@ providers: MyCustomTransform: "pkg.subpkg.PTransformClassOrCallable" ``` -## Pipeline Options - -[Pipeline options](https://beam.apache.org/documentation/programming-guide/#configuring-pipeline-options) -are used to configure different aspects of your pipeline, such as the pipeline runner that will execute -your pipeline and any runner-specific configuration required by the chosen runner. To set pipeline options, -append an options block at the end of your yaml file. For example: - -``` -pipeline: - type: chain - transforms: - - type: ReadFromPubSub - config: - topic: myPubSubTopic - format: ... - schema: ... - ... - - type: WriteToPubSub - config: - topic: anotherPubSubTopic - format: json -options: - streaming: true -``` - ## Other Resources * [Example pipelines](https://gist.github.com/robertwb/2cb26973f1b1203e8f5f8f88c5764da0)
