arnobock opened a new pull request, #2427: URL: https://github.com/apache/systemds/pull/2427
This PR implements a first version of a builtin realising independent subnet training ([https://www.vldb.org/pvldb/vol15/p1581-wolfe.pdf]) as DML script. The implementation provides: - a training routine for IST-based subnet optimization - a disjoint masking/partitioning function to construct independent subnet models - test infrastructure using LeNet + MNIST to validate correctness The builtin script consists of two functions: the training itself and a disjoint masking function for a provided model. Furthermore test infrastructure has been build to train independent subnet training on Lenet architecture and MNIST dataset. As hinted this version needs further runtime optimisation to enhance performance. A current determined bottleneck is the gradient computation, which (even though executed through a parfor-loop) remains rather slow, could be the sparse masking of disjoint subnets since dimensionalities of the model matrices are effectively not reduced. The code contains some TODO's which hint at potential performance boosts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
