This is an automated email from the ASF dual-hosted git repository.
jooykim pushed a change to branch maven-artifact
in repository https://gitbox.apache.org/repos/asf/incubator-nemo.git.
from 49266b5 Merge branch 'master' into maven-artifact
add 04d6eb4 [NEMO-230] Emit collected data when receiving watermark in
GroupByKeyAndWindowTransform (#135)
add 53522c6 [NEMO-237] Refactor ParentTaskDataFetcher to emit streaming
data and watermark (#138)
add 7858051 [NEMO-232] Implement InputWatermarkManager (#137)
new 85d2507 Merge branch 'master' into maven-artifact
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../GroupByKeyAndWindowDoFnTransform.java | 165 ++++++++++---
.../frontend/beam/transform/DoFnTransformTest.java | 35 ---
.../GroupByKeyAndWindowDoFnTransformTest.java | 176 +++++++++++++
.../beam/transform/TestOutputCollector.java | 69 ++++++
pom.xml | 16 +-
.../runtime/executor/data/PipeManagerWorker.java | 8 +-
.../datatransfer/DataFetcherOutputCollector.java | 1 +
.../datatransfer/InputWatermarkManager.java | 46 ++++
.../datatransfer/MultiInputWatermarkManager.java | 90 +++++++
.../datatransfer/NextIntraTaskOperatorInfo.java | 59 +++++
.../OperatorVertexOutputCollector.java | 35 ++-
...eader.java => SingleInputWatermarkManager.java} | 35 +--
.../task/MultiThreadParentTaskDataFetcher.java | 150 +++++++++++
.../executor/task/ParentTaskDataFetcher.java | 1 -
.../nemo/runtime/executor/task/TaskExecutor.java | 111 +++++++--
.../datatransfer/InputWatermarkManagerTest.java | 88 +++++++
.../runtime/executor/task/TaskExecutorTest.java | 274 +++++++++++++++------
17 files changed, 1145 insertions(+), 214 deletions(-)
create mode 100644
compiler/test/src/test/java/org/apache/nemo/compiler/frontend/beam/transform/GroupByKeyAndWindowDoFnTransformTest.java
create mode 100644
compiler/test/src/test/java/org/apache/nemo/compiler/frontend/beam/transform/TestOutputCollector.java
create mode 100644
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/InputWatermarkManager.java
create mode 100644
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/MultiInputWatermarkManager.java
create mode 100644
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/NextIntraTaskOperatorInfo.java
copy
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/{InputReader.java
=> SingleInputWatermarkManager.java} (53%)
create mode 100644
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/task/MultiThreadParentTaskDataFetcher.java
create mode 100644
runtime/executor/src/test/java/org/apache/nemo/runtime/executor/datatransfer/InputWatermarkManagerTest.java
