This is an automated email from the ASF dual-hosted git repository.
jkff pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from b059664 Merge pull request #4218: Build output redirection option
new 90402e4 enforce fixed sharding
new 5795c32 Merges Writer.openWindowed/Unwindowed and removes result of
close()
new b2d0671 non-null window/pane in FileResult
new 54eacf4 remove ShardAssignment
new 97df5e7 consolidates windowed/unwindowed finalize fns somewhat
new c615438 Unifies windowed and unwindowed finalize.
new 2f73a95 Refactors WriteFiles into sub-transforms
new 5b600da Converts WriteFiles to AutoValue
new 3ecf13b Makes checkstyle and findbugs happy
new 83837eb Renames spilled back to unwritten
new 060f05c Fixes tests
new d314339 Reintroduces dynamic sharding with windowed writes for
bounded collections
new 761ec1a This closes #4145: Many simplifications to WriteFiles
The 13 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/beam/examples/WindowedWordCount.java | 5 +-
.../examples/common/WriteOneFilePerWindow.java | 12 +-
.../apache/beam/examples/WindowedWordCountIT.java | 8 +
.../beam/runners/apex/examples/WordCountTest.java | 2 +-
.../core/construction/WriteFilesTranslation.java | 7 +-
.../construction/WriteFilesTranslationTest.java | 13 +-
.../beam/runners/dataflow/DataflowRunnerTest.java | 2 +-
.../beam/runners/spark/io/AvroPipelineTest.java | 5 +-
.../java/org/apache/beam/sdk/io/FileBasedSink.java | 281 +++---
.../java/org/apache/beam/sdk/io/WriteFiles.java | 1061 ++++++++------------
.../java/org/apache/beam/sdk/transforms/Reify.java | 73 +-
.../apache/beam/sdk/values/TypeDescriptors.java | 4 +
.../org/apache/beam/sdk/io/FileBasedSinkTest.java | 66 +-
.../org/apache/beam/sdk/io/WriteFilesTest.java | 27 +-
14 files changed, 745 insertions(+), 821 deletions(-)
--
To stop receiving notification emails like this one, please contact
['"[email protected]" <[email protected]>'].
