larroy edited a comment on issue #14991: Second order gradient wrt inputs, expected behaviour. URL: https://github.com/apache/incubator-mxnet/issues/14991#issuecomment-494201045 This example works for me. I'm able to call grad two times, the second gradient has the correct value, given than the second function is 3 * 2 *x, so the grad is 6 for all elements. When using ag.grad the gradient is not stored in x though. One key issue here is that create_graph in the second call to grad has to be set to false, otherwise we are re-creating the graph and having the problem I had before that the graph only contains backward nodes and NOT the original nodes. ``` def test_ag_grad(): x = mx.nd.array([[2,2,2],[2,2,2],[2,2,4]]) y = mx.nd.array([[2,2,2], [3,3,3], [4,4,4]]) x.attach_grad() y.attach_grad() with mx.autograd.record(): z = nd.elemwise_add(nd.elemwise_mul(x,x),y) x_grad_y_grad = mx.autograd.grad(z, x, create_graph=True, retain_graph=True)[0] print("dz/dx |x") print(type(x_grad_y_grad)) print(x_grad_y_grad) fg_f = 3 * x_grad_y_grad second_grad = mx.autograd.grad(fg_f, [x], create_graph=False, retain_graph=True) print("second grad") print(second_grad) print("x.grad") print(x.grad) #fg_f.backward() print("x.grad") print(x.grad) ``` ``` test_autograd.test_ag_grad ... variables [[2. 2. 2.] [2. 2. 2.] [2. 2. 4.]] <NDArray 3x3 @cpu(0)> var_handles: <mxnet.base.c_void_p_Array_1 object at 0x7f0b9e536a60> dz/dx |x: [[4. 4. 4.] [4. 4. 4.] [4. 4. 8.]] <NDArray 3x3 @cpu(0)> variables [[2. 2. 2.] [2. 2. 2.] [2. 2. 4.]] <NDArray 3x3 @cpu(0)> var_handles: <mxnet.base.c_void_p_Array_1 object at 0x7f0b9e536b70> second grad: [[6. 6. 6.] [6. 6. 6.] [6. 6. 6.]] <NDArray 3x3 @cpu(0)> x.grad: [[0. 0. 0.] [0. 0. 0.] [0. 0. 0.]] <NDArray 3x3 @cpu(0)> x.grad: [[0. 0. 0.] [0. 0. 0.] [0. 0. 0.]] <NDArray 3x3 @cpu(0)> ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
