Polber commented on code in PR #30368:
URL: https://github.com/apache/beam/pull/30368#discussion_r1518259633
##########
website/www/site/content/en/documentation/programming-guide.md:
##########
@@ -2415,6 +2423,14 @@ properties in your `ParDo` operation and follow this
operation with a `Split`
to break it into multiple `PCollection`s.
{{< /paragraph >}}
+{{< paragraph class="language-yaml">}}
+In Beam YAML one obtains multiple outputs by emitting all outputs to a single
+`PCollection`, possibly with an extra field, and then using `Partition` to
+split this single `PCollection` into multiple distinct `PCollections`
+outputs.
Review Comment:
```suggestion
In Beam YAML, one obtains multiple outputs by emitting all outputs to a
single
`PCollection`, possibly with an extra field, and then using `Partition` to
split this single `PCollection` into multiple distinct `PCollection`
outputs.
```
##########
website/www/site/content/en/documentation/sdks/yaml-udf.md:
##########
@@ -207,6 +207,74 @@ criteria. This can be accomplished with a `Filter`
transform, e.g.
keep: "col2 > 0"
```
+## Partitioning
+
+It can also be useful to send different elements to different places
+(similar to what is done with side outputs in other SDKs).
+While this can be done with a set of `Filter` operations, if every
+element has a single destination it can be more natural to use a `Split`
+transform instead which send every element to a unique output.
+For example, this will send all elements where `col1` is equal to `"a"` to the
+output `Split.a`.
Review Comment:
```suggestion
element has a single destination it can be more natural to use a `Partition`
transform instead which sends every element to a unique output.
For example, this will send all elements where `col1` is equal to `"a"` to
the
output `Partition.a`.
```
##########
website/www/site/content/en/documentation/programming-guide.md:
##########
@@ -2153,6 +2153,14 @@ students = ...
{{< code_sample "sdks/typescript/test/docs/programming_guide.ts"
model_multiple_pcollections_partition >}}
{{< /highlight >}}
+{{< highlight yaml >}}
+{{< code_sample "sdks/python/apache_beam/yaml/programming_guide_test.py"
model_multiple_pcollections_partition >}}
+{{< /highlight >}}
+
+{{< paragraph class="language-yaml">}}
+Note that in Beam YAML `PCollections` are partitioned via string rather than
integer values.
Review Comment:
```suggestion
Note that in Beam YAML, `PCollections` are partitioned via string rather
than integer values.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
