ptrendx commented on a change in pull request #13346: Aggregate SGD URL: https://github.com/apache/incubator-mxnet/pull/13346#discussion_r244892807
########## File path: docs/faq/env_var.md ########## @@ -218,6 +222,11 @@ When USE_PROFILER is enabled in Makefile or CMake, the following environments ca - When the array size is bigger than or equal to this threshold, NDArray::Copy(from, to) is implemented by OpenMP with the Recommended OMP Thread Count. - When the array size is less than this threshold, NDArray::Copy(from , to)) is implemented by memcpy in single thread. +* MXNET_OPTIMIZER_AGGREGATION_SIZE + - Values: Int ```(default=4)``` + - Maximum value is 60. Review comment: Maximum is here because the kernel cannot accept more than 60 pairs at the same time. This could be lifted by calling the kernel multiple times, although that's not ideal because setting it to a value higher than 60 would not have any performance benefit and it could actually slightly hurt performance because the operator would need to wait for more gradients to be ready before starting. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
