Kangji opened a new pull request #319: URL: https://github.com/apache/incubator-nemo/pull/319
JIRA: [NEMO-472: Fix and Implement Hierarchical Aggregation](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-472) **Major changes:** - Implemented stage execution property(TaskIndexToExecutorIDProperty) that stores task allocation information(stored in runtime) - Implemented data transfer logic on partial shuffle communication channel - Implemented constraint when scheduling intermediate combine stage(the stage that performs intermediate combine) **Minor changes to note:** - None **Tests for the changes:** - Tested on my Mac and ubuntu machine **Other comments:** - Also see #318, which implemented optimization pass that inserts intermediate accumulator vertex. - Currently the implemented scheduling constraint simply distributes tasks in the intermediate combine stage evenly to each shuffle executor groups. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
