[
https://issues.apache.org/jira/browse/BEAM-11078?focusedWorklogId=503918&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503918
]
ASF GitHub Bot logged work on BEAM-11078:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Oct/20 22:13
Start Date: 22/Oct/20 22:13
Worklog Time Spent: 10m
Work Description: rosetn commented on a change in pull request #13160:
URL: https://github.com/apache/beam/pull/13160#discussion_r510487900
##########
File path: website/www/site/content/en/documentation/programming-guide.md
##########
@@ -5143,3 +5143,281 @@ perUser.apply(ParDo.of(new DoFn<KV<String, ValueT>,
OutputT>() {
}
}));
{{< /highlight >}}
+
+## 12. Splittable `DoFns` {#splittable-dofns}
+
+A Splittable `DoFn` (SDF) enables users to create modular components
containing I/Os (and some advanced
+[non I/O use
cases](https://s.apache.org/splittable-do-fn#heading=h.5cep9s8k4fxv)). Having
modular
+I/O components that can be connected to each other simplify typical patterns
that users want.
+For example, a popular use case is to read filenames from a message queue
followed by parsing those
+files. Traditionally, users were required to either write a single I/O
connector that contained the
+logic for the message queue and the file reader (increased complexity) or
choose to reuse a message
+queue I/O followed by a regular `DoFn` that read the file (decreased
performance). With SDF,
+we bring the richness of Apache Beam’s I/O APIs to a `DoFn` enabling
modularity while maintaining the
+performance of traditional I/O connectors.
+
+### 12.1. SDF basics {#sdf-basics}
+
+At a high level, a SDF is responsible for processing element and restriction
pairs. A
Review comment:
Let's make these "an SDF"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 503918)
Time Spent: 2h 50m (was: 2h 40m)
> Add splittable DoFn programming guide documentation
> ---------------------------------------------------
>
> Key: BEAM-11078
> URL: https://issues.apache.org/jira/browse/BEAM-11078
> Project: Beam
> Issue Type: Improvement
> Components: website
> Reporter: Luke Cwik
> Assignee: Luke Cwik
> Priority: P2
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Convert
> https://docs.google.com/document/d/1kpn0RxqZaoacUPVSMYhhnfmlo8fGT-p50fEblaFr2HE/edit#
> to documentation for the website.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
