This closes #2147: Explicitly GBK before stateful ParDo in Dataflow batch Upgrade Dataflow container version to beam-master-20170307 Explicitly GBK before stateful ParDo in Dataflow batch
Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/1d8be974 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/1d8be974 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/1d8be974 Branch: refs/heads/master Commit: 1d8be97439051ae770f99dcb49c7e8486962b118 Parents: 37c17f2 c7a06a6 Author: Kenneth Knowles <[email protected]> Authored: Wed Mar 8 10:38:12 2017 -0800 Committer: Kenneth Knowles <[email protected]> Committed: Wed Mar 8 10:38:12 2017 -0800 ---------------------------------------------------------------------- runners/google-cloud-dataflow-java/pom.xml | 2 +- .../dataflow/BatchStatefulParDoOverrides.java | 283 +++++++++++++++++++ .../dataflow/DataflowPipelineTranslator.java | 5 +- .../beam/runners/dataflow/DataflowRunner.java | 6 + .../beam/runners/dataflow/util/DoFnInfo.java | 9 + .../BatchStatefulParDoOverridesTest.java | 169 +++++++++++ .../DataflowPipelineTranslatorTest.java | 72 +++++ .../apache/beam/sdk/transforms/ParDoTest.java | 117 ++++++++ 8 files changed, 661 insertions(+), 2 deletions(-) ----------------------------------------------------------------------
