ilovemesomeramen opened a new pull request #1351: URL: https://github.com/apache/systemds/pull/1351
This PR adds a Framework for dependent Task execution consisting of the `DependencyTask` and `DependencyThreadPool` as well as the implementation of build and apply in said Framework. It is now possible for different encoders to provide tasks which best fit the given data characteristics by overriding the `getApplyTasks` and `getBuildTasks` functions. These functions can return a DAG of Tasks which will be integrated into the DAG of the whole Matrix/Frame. Subsequently this DAG is then executed in the `DependencyThreadPool`. This approach enables the possibility of trying different execution orders without the need to make major changes to the code. At the moment only the Dummycoders implement a different apply method compared to the default. This will be changed in future PRs. Currently there are two approaches implemented staged and non staged. During staged execution the building and applying steps take place one after another. Which means the building will be done for the whole Matrix/Frame and then the apply takes place. Non staged execution on the other hand constructs one large DAG for building and applying. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
