swfearp opened a new issue #17881: Solved Bug: contrib/amp/loss_scalar.py not initializing variables URL: https://github.com/apache/incubator-mxnet/issues/17881 Not sure if this has been raised before, but I couldn't find it. When training with multiple gpus using amp I get the following error message: ``` Traceback (most recent call last): File "_ctypes/callbacks.c", line 234, in 'calling callback function' File "/home/sam/envs/devenv/lib/python3.6/site-packages/mxnet/kvstore.py", line 84, in updater_handle updater(key, lhs, rhs) File "/home/sam/envs/devenv/lib/python3.6/site-packages/mxnet/optimizer/optimizer.py", line 1699, in __call__ states) File "/home/sam/envs/devenv/lib/python3.6/site-packages/mxnet/contrib/amp/amp.py", line 305, in new_update_multi_precision if not skip_update(): File "/home/sam/envs/devenv/lib/python3.6/site-packages/mxnet/contrib/amp/loss_scaler.py", line 63, in wait_and_update if self._wait_for_outputs: AttributeError: 'LossScaler' object has no attribute '_wait_for_outputs' ``` Can be fixed by just initializing the following variables in contrib/amp/loss_scalar.py: ``` self._wait_for_outputs = True self.output = nd.array([]) ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
