[GitHub] jeongyooneo opened a new pull request #2: [NEMO-7] Intra-TaskGroup pipelining

[GitBox]

jeongyooneo opened a new pull request #2: [NEMO-7] Intra-TaskGroup pipelining
URL: https://github.com/apache/incubator-nemo/pull/2
 
 
   * **Refactored `TaskGroupExecutor` to a pipelined execution model(Thanks to 
@sanha)**
      * Per-`Task` `InputReader`/`OutputWriter` and `TaskState` management are 
removed.
      * Per-`Task` `Pipe` acts as a local writer for this `Task` to where it 
emits output, and a local reader for its children `Task`s from where they 
receives input.
      * Input is deserialized element-wise, trickle down to the end of 
`TaskGroup` DAG in a recursive manner. 
          * `iteratorIdToDstTasksMap` maintains source input(data from bounded 
source or other `TaskGroup`s) to the `Task` DAG that processes this input.
          * `pipeToDstTasksMap` maintains intra-TaskGroup input to the `Task` 
DAG that processes this input.
   
   <p align="center">
   <img 
src="https://user-images.githubusercontent.com/6097406/36969167-11bdbb90-20a8-11e8-89f8-d3d1c459154c.png";
 width="400">
   </p>
   
   <p align="center">
   <img 
src="https://user-images.githubusercontent.com/6097406/36969232-4517e236-20a8-11e8-93e2-f4e8d8ae8f84.png";
 width="400">
   </p>
   
   JIRA: [NEMO-7](https://issues.apache.org/jira/browse/NEMO-7)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services