piiswrong commented on a change in pull request #8766: NDArray Indexing tutorial and Gradient Compression FAQ URL: https://github.com/apache/incubator-mxnet/pull/8766#discussion_r152674943
########## File path: docs/faq/gradient_compression.md ########## @@ -0,0 +1,107 @@ +# Gradient Compression + +Gradient Compression reduces communication bandwidth to make distributed training with GPUs more scalable and efficient without significant loss in convergence rate or accuracy. + + +## Benefits + +**Increased Speed** + +For architectures with fully connected components, the gradient compression capability is observed to speedup training by about 2x, depending on the size of the model and the network bandwidth of the instance. Bigger models see larger speedup with gradient compression. Review comment: fully connected components -> fully connected layers ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services