eric-haibin-lin commented on issue #17237: Data imbalance handling in MXNet Gluon URL: https://github.com/apache/incubator-mxnet/issues/17237#issuecomment-571816022 Hi @chandana1332 Thanks for posting the question here. If your current mini-batch is small and GPU 3 & 4 does not even have 1 sample, the gradient on GPU 3 & 4 will remain the same as what they were for the previous iteration. Therefore, for this case, the allreduced_gradient will be based on refresh gradients from GPU 1 & 2, and stale gradients from GPU 3 & 4.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
