[GitHub] [incubator-mxnet] sxjscience commented on issue #17989: [Gradient Addto] Very serious bug of grad_req='add'

[GitBox]

sxjscience commented on issue #17989: [Gradient Addto] Very serious bug of 
grad_req='add'
URL: 
https://github.com/apache/incubator-mxnet/issues/17989#issuecomment-610591835
 
 
   @ptrendx @szha @zhreshold I find that the bug also exists in 1.5.0, 1.4.0, 
1.3.1, 1.2.1. In fact, results on both CPU and GPU are wrong in these versions. 
Reproducible script is given as follows (I used the legacy mx.nd).
   
   ```python
   import mxnet as mx
   import numpy as np
   
   
   for ctx in [mx.cpu(), mx.gpu()]:
       for nrepeat in range(1, 10):
           stored_grad = dict()
           for grad_req in ['write', 'add']:
               a = mx.nd.array([1], ctx=ctx)
               b = mx.nd.array([2], ctx=ctx)
               if grad_req == 'write':
                   a.attach_grad(grad_req='write')
               elif grad_req == 'add':
                   a.attach_grad(grad_req='add')
               a.grad[:] = 0
               with mx.autograd.record():
                   for _ in range(nrepeat):
                       b = b * a
                   b.backward()
               stored_grad[grad_req] = a.grad.asscalar()
           print('ctx={}, nrepeat={}, write={}, add={}'.format(ctx, nrepeat, 
stored_grad['write'], stored_grad['add']))
   ```
   
   For MXNet 1.5.0, I used `pip install mxnet-cu101==1.5.0`
   For MXNet 1.4.0, I used `pip install mxnet-cu92==1.4.0`
   For MXNet 1.3.1, I used `pip install mxnet-cu92==1.3.1`
   For MXNet 1.2.1, I used `pip install mxnet-cu92==1.2.1`
   
   Output
   ```
   ctx=cpu(0), nrepeat=1, write=2.0, add=2.0
   ctx=cpu(0), nrepeat=2, write=4.0, add=4.0
   ctx=cpu(0), nrepeat=3, write=6.0, add=6.0
   ctx=cpu(0), nrepeat=4, write=8.0, add=8.0
   ctx=cpu(0), nrepeat=5, write=10.0, add=62.0
   ctx=cpu(0), nrepeat=6, write=12.0, add=126.0
   ctx=cpu(0), nrepeat=7, write=14.0, add=254.0
   ctx=cpu(0), nrepeat=8, write=16.0, add=16.0
   ctx=cpu(0), nrepeat=9, write=18.0, add=18.0
   ctx=gpu(0), nrepeat=1, write=2.0, add=2.0
   ctx=gpu(0), nrepeat=2, write=4.0, add=4.0
   ctx=gpu(0), nrepeat=3, write=6.0, add=6.0
   ctx=gpu(0), nrepeat=4, write=8.0, add=8.0
   ctx=gpu(0), nrepeat=5, write=10.0, add=62.0
   ctx=gpu(0), nrepeat=6, write=12.0, add=126.0
   ctx=gpu(0), nrepeat=7, write=14.0, add=254.0
   ctx=gpu(0), nrepeat=8, write=16.0, add=16.0
   ctx=gpu(0), nrepeat=9, write=18.0, add=18.0
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services