anirudhacharya edited a comment on issue #11865: attach_grad of intermediate variables causes the gradient graph to be lost URL: https://github.com/apache/incubator-mxnet/issues/11865#issuecomment-488424194 To understand this right, in scenarios such as the following ```python x = mx.nd.array([0, 7], ctx = mx.cpu()) x.attach_grad() with mx.autograd.record(): y = ((5 * (x**2)) + (13 * x) + 10) y.attach_grad() z = 2 * y z.backward() print(x.grad) ``` what you want is we should be able to get `x.grad` as non-zero values even though the intermediate variable `y` has been marked with `attach_grad`. In the above example would you also want the result of `y.grad` to be retained? Because that would be a bit different and more like a feature request than a bug. It will probably involve storing intermediate gradients of non-leaf variables in some sort of a buffer by providing a hook/function call to the user to enable storing the gradients of non-leaf variables because storing it every time by default might be a waste of memory. Have I understood this right? Which of the two above situations is your issue pointing to?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
