This closes #1678: Refactor Dataflow translator to decouple steps Reduce visibility of many Dataflow runner internals Move some pieces of Dataflow translator to top level Add explicit translation builder for a Step to in Dataflow translator
Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a6caa82a Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/a6caa82a Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/a6caa82a Branch: refs/heads/master Commit: a6caa82a621a0a9c1cd2cd92c26304e9f79546bb Parents: b4d8702 33907f8 Author: Kenneth Knowles <[email protected]> Authored: Fri Jan 6 11:37:07 2017 -0800 Committer: Kenneth Knowles <[email protected]> Committed: Fri Jan 6 11:37:07 2017 -0800 ---------------------------------------------------------------------- .../beam/runners/dataflow/AssignWindows.java | 89 +++ .../dataflow/DataflowAggregatorTransforms.java | 79 +++ .../dataflow/DataflowMetricUpdateExtractor.java | 109 ++++ .../runners/dataflow/DataflowPipelineJob.java | 2 - .../dataflow/DataflowPipelineTranslator.java | 424 ++++++-------- .../beam/runners/dataflow/DataflowRunner.java | 68 +-- .../DataflowUnboundedReadFromBoundedSource.java | 547 +++++++++++++++++++ .../beam/runners/dataflow/ReadTranslator.java | 102 ++++ .../runners/dataflow/TransformTranslator.java | 123 +++++ .../dataflow/internal/AssignWindows.java | 89 --- .../internal/DataflowAggregatorTransforms.java | 79 --- .../internal/DataflowMetricUpdateExtractor.java | 109 ---- .../DataflowUnboundedReadFromBoundedSource.java | 547 ------------------- .../dataflow/internal/ReadTranslator.java | 104 ---- .../dataflow/DataflowPipelineJobTest.java | 1 - .../DataflowPipelineTranslatorTest.java | 3 +- .../runners/dataflow/DataflowRunnerTest.java | 8 +- ...aflowUnboundedReadFromBoundedSourceTest.java | 79 +++ ...aflowUnboundedReadFromBoundedSourceTest.java | 79 --- 19 files changed, 1316 insertions(+), 1325 deletions(-) ----------------------------------------------------------------------
