This is an automated email from the ASF dual-hosted git repository.
jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new bd845aa Drafted documentation for autograd. (#7395)
bd845aa is described below
commit bd845aa58c08329192fc142f598d045275f43342
Author: Zack Chase Lipton <zachary.ch...@gmail.com>
AuthorDate: Wed Aug 9 13:07:57 2017 -0700
Drafted documentation for autograd. (#7395)
* added autograd documentation
* adding output for snippet
* adding output for snippet
* typo fix
* typo fix
* revised the autograd doc
* typo
* added sentence about autograd.pause():
* typo
* further deflation
* Update autograd.md
---
docs/api/python/autograd.md | 58 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 58 insertions(+)
diff --git a/docs/api/python/autograd.md b/docs/api/python/autograd.md
index d204a2c..5c84964 100644
--- a/docs/api/python/autograd.md
+++ b/docs/api/python/autograd.md
@@ -9,6 +9,64 @@
.. warning:: This package is currently experimental and may change in the near
future.
```
+## Overview
+
+The `autograd` package enables automatic
+differentiation of NDArray operations.
+In machine learning applications,
+`autograd` is often used to calculate the gradients
+of loss functions with respect to parameters.
+
+
+### Record vs Pause
+
+`autograd` records computation history on the fly to calculate gradients later.
+This is only enabled inside a `with autograd.record():` block.
+A `with auto_grad.pause()` block can be used inside a `record()` block
+to temporarily disable recording.
+
+To compute gradient with respect to an `NDArray` `x`, first call
`x.attach_grad()`
+to allocate space for the gradient. Then, start a `with autograd.record()`
block,
+and do some computation. Finally, call `backward()` on the result:
+
+```python
+>>> x = mx.nd.array([1,2,3,4])
+>>> x.attach_grad()
+>>> with mx.autograd.record():
+... y = x * x + 1
+>>> print(x.grad)
+[ 2. 4. 6. 8.]
+<NDArray 4 @cpu(0)>
+```
+
+
+## Train mode and Predict Mode
+
+Some operators (Dropout, BatchNorm, etc) behave differently in
+when training and when making predictions.
+This can be controled with `train_mode` and `predict_mode` scope.
+
+By default, MXNet is in `predict_mode`.
+A `with autograd.record()` block by default turns on `train_mode`
+(equivalent to ``with autograd.record(train_mode=True)``).
+To compute a gradient in prediction mode (as when generating adversarial
examples),
+call record with `train_mode=False` and then call `backward(train_mode=False)`
+
+Although training usually coincides with recording,
+this isn't always the case.
+To control *training* vs *predict_mode* without changing
+*recording* vs *not recording*,
+Use a `with autograd.train_mode():`
+or `with autograd.predict_mode():` block.
+
+Detailed tutorials are available in Part 1 of
+[the MXNet gluon book](http://gluon.mxnet.io/).
+
+
+
+
+
+
<script type="text/javascript"
src='../../_static/js/auto_module_index.js'></script>
## Autograd
--
To stop receiving notification emails like this one, please contact
['"comm...@mxnet.apache.org" <comm...@mxnet.apache.org>'].
