[GitHub] kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices
kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices URL: https://github.com/apache/incubator-mxnet/issues/8337#issuecomment-379471987 @roywei To be honest, over the last months I've already lost the context and don't remember what I was using the slicing for. You're correct by saying it may not be an ideal way. But as a developer I know how a seemingly insignificant issue can help to discover really big bugs in a software project. So I decided "If I discovered the bug anyway, I might as well go on and report it. Perhaps it would help the guys to improve autograd" :) About symbolic gradient via NNVM: I'm trying to do exactly that now. Unfortunately both NNVM and TVM have rather thin documentation, and to understand some things requires digging through C++ code, so the progress is slow. Simply don't have much time. Hence the attempt to use mxnet's autograd as a shortcut. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices
kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices URL: https://github.com/apache/incubator-mxnet/issues/8337#issuecomment-338047855 @piiswrong Is there any better way to get the graph/symbol from autograd? Because the method I use seems logical to me: I did forward propagation, then I launched back-propagation, then I saved the graph, so now I can re-use it again without the need of recording it. I know, it's not very flexible, but I have a reason for that. I'm trying to get the graph explicitly because I want to use nnvm to compile and run this graph on an OpenCL device. Currently nnvm-mxnet frontend supports only forward-propagating `HybridBlock`s or mxnet.symbol -based graphs. That's why I'm trying to get the last one. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices
kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices URL: https://github.com/apache/incubator-mxnet/issues/8337#issuecomment-337796013 @ZiyueHuang Yes, I think the empty line is indeed the argument `''`. But when I replace `mx.nd.ones_like(b)` with `mx.nd.ones((1,))` I still get the same error. Are you sure that when it worked for you, you actually did use slicing? Just to be on the same page, here's the full code that fails, even though I believe it shouldn't: ``` import mxnet as mx from mxnet import nd, autograd ctx = mx.cpu() a = mx.nd.array([1, 2, 3, 4], ctx=ctx) a.attach_grad() with autograd.record(): b = nd.sum(2 * (a[0:4] ** 2)) # works without slicing grads = autograd.grad(b, [a], create_graph=True, retain_graph=True) da_sym = autograd.get_symbol(grads[0]) executor = da_sym.bind(ctx=ctx, args=[nd.ones_like(b), a]) executor.forward() print(executor.outputs[0]) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices
kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices URL: https://github.com/apache/incubator-mxnet/issues/8337#issuecomment-337794186 @ZiyueHuang `a`s first dimension is 4, and slicing it like `a[0:4]` is absolutely valid and I didn't care about effectiveness here. But after you asked, I tried different expressions. I tried different sizes of `a` (for example, `a = mx.nd.array([ [ 1, 2, 3, 4] ])` and slicing it in the expression as `a[0]`). None of that has worked. I still see the same error every time I use slicing. `da_sym.list_arguments()` returns `['', 'var0']`. One must be the head gradient for the chain rule and another one is a placeholder the for variable `a`. That's why I used such arguments. Which is which I determined experimentally, since both have different shapes, and I could easilly check the result of `executor.forward()` knowing derivative `db / da = 4 * a`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices
kpot commented on issue #8337: mx.autograd.grad works or fails depending on use of slices URL: https://github.com/apache/incubator-mxnet/issues/8337#issuecomment-337678517 I'm afraid you miss the point. The failing code works fine being run using ordinary `.backward()` call: ``` import mxnet as mx from mxnet import nd, autograd ctx = mx.cpu() a = mx.nd.array([1, 2, 3, 4], ctx=ctx) a.attach_grad() with autograd.record(): b = nd.sum(2 * (a[0:4] ** 2)) b.backward() print(a.grad) ``` Why does it fail when I retain the graph by using `autograd.grad` and then re-use this graph again directly via `.bind`? And such re-use fails *only* when the slicing is used. I deliberately made slicing `a[0:4]` because it doesn't change the shape of the expression and technically should be equal to just `a`. Without slicing both `autograd.grad` and `.bind` do their jobs perfectly well. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
