phaniarnab opened a new pull request, #2010: URL: https://github.com/apache/systemds/pull/2010
These commits add the full integration of ResNets. The forward passes for the basic residual block architecture (ResNet18 & 34) have been previously added. These commits add the forward passes for the bottleneck architecture (ResNet50, 101 & 152) and the backward passes for all architectures. Additionally, since the iteration through the model parameters, especially for the bigger models, gets quite complex, I implemented utiltiy functions for the updating the parameters with differen optimizers. For each model and for each optimizer, a uitlity method to initialize the optimizer state parameters and to update the model parameters have been added. There is also an example added which uses random data to showcase the full training loop of the ResNets (forward, backward, param updating). The example uses ResNet18 but can changed by simply modifying the import statement to another model. Also, the ADAM optimizer is used but I also added all the other optimizers in the example and can be used by uncommenting the respective lines. The example was tested for all ResNets (18, 34, 50, 101, 152) and for all optimizers. Every combination trains correctly (i.e. the loss shrinks). About testing, I've added comprehensive testing for the forward and backward pass of the residual blocks (basic and bottleneck architectures) and for the residual layer's forward pass (multiple residual blocks) but not for the residual layer backward pass and the full networks because the example data would take up to much space. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@systemds.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
