This is an automated email from the ASF dual-hosted git repository. mergebot-role pushed a commit to branch mergebot in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit b50a4a99faffff88c8c14c7f555be984c134b7bf Author: Eugene Kirpichov <kirpic...@google.com> AuthorDate: Thu Sep 7 13:35:57 2017 -0700 Fixes SDF blog post after rename of Match to FileIO.match --- src/_posts/2017-08-04-splittable-do-fn.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/src/_posts/2017-08-04-splittable-do-fn.md b/src/_posts/2017-08-04-splittable-do-fn.md index d5379fd..156a6ad 100644 --- a/src/_posts/2017-08-04-splittable-do-fn.md +++ b/src/_posts/2017-08-04-splittable-do-fn.md @@ -441,17 +441,17 @@ class AvroReader(DoFn): This hypothetical `DoFn` reads records from a single Avro file. Notably missing is the code for expanding a filepattern: it no longer needs to be part of this `DoFn`! Instead, the SDK includes a -[Match.filepatterns()](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Match.java) +[FileIO.matchAll()](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileIO.java) transform for expanding a filepattern into a `PCollection` of filenames, and different file format IOs can reuse the same transform, reading the files with different `DoFn`s. This example demonstrates the benefits of increased modularity allowed by SDF: -`Match` supports continuous ingestion of new files in streaming pipelines using -`.continuously()`, and this functionality becomes automatically available to -various file format IOs. For example, `TextIO.read().watchForNewFiles()` [uses -`Match` under the -hood)](https://github.com/apache/beam/blob/f7e8f886c91ea9d0b51e00331eeb4484e2f6e000/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextIO.java#L480). +`FileIO.matchAll()` supports continuous ingestion of new files in streaming +pipelines using `.continuously()`, and this functionality becomes automatically +available to various file format IOs. For example, +`TextIO.read().watchForNewFiles()` [uses `FileIO.matchAll()` under the +hood)](https://github.com/apache/beam/blob/3bd68ecfd7d576d78e02deb0476e549f11e1b5ef/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextIO.java#L486). ## Current status @@ -471,9 +471,10 @@ Several SDF-based transforms and IO connectors are available for Beam users at HEAD and will be included in Beam 2.2.0. `TextIO` and `AvroIO` finally provide continuous ingestion of files (one of the most frequently requested features) via `.watchForNewFiles()` which is backed by the utility transforms -`Match.filepatterns().continuously()` and the more general +`FileIO.matchAll().continuously()` and the more general [`Watch.growthOf()`](https://github.com/apache/beam/blob/f7e8f886c91ea9d0b51e00331eeb4484e2f6e000/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Watch.java). -These utility transforms are also independently useful for "power user" use cases. +These utility transforms are also independently useful for "power user" use +cases. To enable more flexible use cases for IOs currently based on the Source API, we will change them to use SDF. This transition is [pioneered by -- To stop receiving notification emails like this one, please contact "commits@beam.apache.org" <commits@beam.apache.org>.