chrishkchris commented on issue #646: SINGA-487 Test case for distributed module URL: https://github.com/apache/singa/pull/646#issuecomment-607628829 > > > @XJDKC I also added `self.fused_all_reduce([p.data], send=False)` in the function `backward_and_partial_update` > > > to adopt the change last time in the `fused_all_reduce` function > > > > > > You can pass the whole plist to the fused_all_reduce function, no need to pass tensors one by one. > > done, it becomes > > ``` > self.fused_all_reduce(plist, send=False) > self.fused_all_reduce(plist) > ``` The asychronous training is also okay now: ``` root@71ac539cda77:~/dcsysh/singa/examples/autograd# python3 cifar10_multiprocess.py Starting Epoch 0: Training loss = 2910.528564, training accuracy = 0.335027 Evaluation accuracy = 0.438902, Elapsed Time = 148.023807s Starting Epoch 1: Training loss = 2098.108887, training accuracy = 0.512984 Evaluation accuracy = 0.558393, Elapsed Time = 147.847368s Starting Epoch 2: Training loss = 1696.253052, training accuracy = 0.612596 Evaluation accuracy = 0.634615, Elapsed Time = 145.889916s Starting Epoch 3: Training loss = 1424.486328, training accuracy = 0.678877 Evaluation accuracy = 0.725461, Elapsed Time = 144.518893s Starting Epoch 4: Training loss = 1202.804688, training accuracy = 0.732714 Evaluation accuracy = 0.731070, Elapsed Time = 145.818738s ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
