rosetn commented on a change in pull request #13160:
URL: https://github.com/apache/beam/pull/13160#discussion_r510487900
##########
File path: website/www/site/content/en/documentation/programming-guide.md
##########
@@ -5143,3 +5143,281 @@ perUser.apply(ParDo.of(new DoFn<KV<String, ValueT>,
OutputT>() {
}
}));
{{< /highlight >}}
+
+## 12. Splittable `DoFns` {#splittable-dofns}
+
+A Splittable `DoFn` (SDF) enables users to create modular components
containing I/Os (and some advanced
+[non I/O use
cases](https://s.apache.org/splittable-do-fn#heading=h.5cep9s8k4fxv)). Having
modular
+I/O components that can be connected to each other simplify typical patterns
that users want.
+For example, a popular use case is to read filenames from a message queue
followed by parsing those
+files. Traditionally, users were required to either write a single I/O
connector that contained the
+logic for the message queue and the file reader (increased complexity) or
choose to reuse a message
+queue I/O followed by a regular `DoFn` that read the file (decreased
performance). With SDF,
+we bring the richness of Apache Beam’s I/O APIs to a `DoFn` enabling
modularity while maintaining the
+performance of traditional I/O connectors.
+
+### 12.1. SDF basics {#sdf-basics}
+
+At a high level, a SDF is responsible for processing element and restriction
pairs. A
Review comment:
Let's make these "an SDF"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
