bhavinthaker commented on a change in pull request #7921: Add three sparse tutorials URL: https://github.com/apache/incubator-mxnet/pull/7921#discussion_r140679971
########## File path: docs/tutorials/sparse/train.md ########## @@ -0,0 +1,256 @@ + +# Train a Linear Regression Model with Sparse Symbols +In previous tutorials, we introduced `CSRNDArray` and `RowSparseNDArray`, +the basic data structures for manipulating sparse data. +MXNet also provides `Sparse Symbol` API, which enables symbolic expressions that handle sparse arrays. +In this tutorial, we first focus on how to compose a symbolic graph with sparse operators, +then train a linear regression model using sparse symbols with the Module API. + +## Prerequisites + +To complete this tutorial, we need: + +- MXNet. See the instructions for your operating system in [Setup and Installation](http://mxnet.io/get_started/install.html). + +- [Jupyter Notebook](http://jupyter.org/index.html) and [Python Requests](http://docs.python-requests.org/en/master/) packages. +``` +pip install jupyter requests +``` + +- Basic knowledge of Symbol in MXNet. See the detailed tutorial for Symbol in [Symbol - Neural network graphs and auto-differentiation](https://mxnet.incubator.apache.org/tutorials/basic/symbol.html). + +- Basic knowledge of CSRNDArray in MXNet. See the detailed tutorial for CSRNDArray in [TODO(haibin) Add Link Here](http://ec2-54-187-32-207.us-west-2.compute.amazonaws.com/tutorials/sparse/csr.html). + +- Basic knowledge of RowSparseNDArray in MXNet. See the detailed tutorial for RowSparseNDArray in [TODO(haibin) Add Link Here](http://ec2-54-187-32-207.us-west-2.compute.amazonaws.com/tutorials/sparse/rowsparse.html). + +## Variables + +Variables are placeholder for arrays. We can use them to hold sparse arrays, too. + +### Variable Storage Types + +The `stype` attribute of a variable is used to indicate the storage type of the array. +By default, the `stype` of a variable is "default" which indicates the default dense storage format. +We can specify the `stype` of a variable as "csr" or "row_sparse" to hold sparse arrays. + + +```python +import mxnet as mx +# Create a variable to hold an NDArray +a = mx.sym.Variable('a') +# Create a variable to hold a CSRNDArray +b = mx.sym.Variable('b', stype='csr') +# Create a variable to hold a RowSparseNDArray +c = mx.sym.Variable('c', stype='row_sparse') +(a, b, c) +``` + +### Bind with Sparse Arrays + +The sparse symbols constructed above declare storage types of the arrays to hold. +To evaluate them, we need to feed the free variables with sparse data. + +You can instantiate an executor from a sparse symbol by using the `simple_bind` method, +which allocate zeros to all free variables according to their storage types. +The executor provides `forward` method for evaluation and an attribute +`outputs` to get all the results. Later, we will show the use of the `backward` method and other methods computing the gradients and updating parameters. A simple example first: + + +```python +shape = (2,2) +# Instantiate an executor from sparse symbols +b_exec = b.simple_bind(ctx=mx.cpu(), b=shape) +c_exec = c.simple_bind(ctx=mx.cpu(), c=shape) +b_exec.forward() +c_exec.forward() +# Sparse arrays of zeros are bound to b and c +print(b_exec.outputs, c_exec.outputs) +``` + +You can update the array held by the variable by accessing executor's `arg_dict` and assigning new values. + + +```python +var_exec.arg_dict['b'][:] = mx.nd.ones(shape).tostype('csr') +var_exec.forward() +# The array `b` holds are updated to be ones +eval_b = var_exec.outputs[0] +{'eval_b': eval_b, 'eval_b.asnumpy()': eval_b.asnumpy()} +``` + +## Symbol Composition and Storage Type Inference + +### Basic Symbol Composition + +The following example builds a simple element-wise addition expression with different storage types. +The sparse symbols are available in the `mx.sym.sparse` package. + + +```python +# Element-wise addition of variables with "default" stype +d = mx.sym.elemwise_add(a, a) +# Element-wise addition of variables with "csr" stype +e = mx.sym.sparse.negative(b) +# Element-wise addition of variables with "row_sparse" stype +f = mx.sym.sparse.elemwise_add(c, c) +{'d':d, 'e':e, 'f':f} +``` + +### Storage Type Inference + +What will be the output storage types of sparse symbols? In MXNet, for any sparse symbol, the result storage types are inferred based on storage types of inputs. +You can read the [Sparse Symbol API](mxnet.io/api/python/symbol/sparse.html) documentation to find what output storage types are. In the example below we will try out the storage types introduced in the Row Sparse and Compressed Sparse Row tutorials: `default` (dense), `csr`, and `row_sparse`. Review comment: Add http:// to the URL. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
