tobiasrieger opened a new pull request #1113: URL: https://github.com/apache/systemds/pull/1113
This adds the shuffle data partitioner the the federated parameter server and a helpful way of running federated unit tests. See federatedBalancedAndWriteWithMTD in the AutomatedTestBase. The federated parameter server unit test was optimised and the scripts TwoNN.dml and CNN.dml updated. Also .getFedMapping() of FederationMap was refactored to getFRangeFDataMap to avoid confusion with the method of MatrixObject with the same name. **Current issues/ questions?** - The hybrid federated parameter server unit tests fail stating "Shuffle" scheme is not supported. This is because the read somehow pulls in the matrix and presents it as a dense block. This means the parameter server is running in regular mode thus failing on the shuffle scheme. - The Unit test input directory is currently not cleaned up. Should this be done after every test? - How to generate the ID of a newly created FederationMap correctly? Currently set to 1 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
