This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git
The following commit(s) were added to refs/heads/master by this push:
new 468c88f polish tensor implementation section
new 12068c9 Merge pull request #12 from dcslin/tensor-impl
468c88f is described below
commit 468c88f354b508c401c78304a7c04f6719846ade
Author: Shicong <[email protected]>
AuthorDate: Thu Apr 2 22:58:48 2020 +0800
polish tensor implementation section
---
docs-site/docs/tensor.md | 40 ++++++++++++++++++++++++++++++++++++++--
1 file changed, 38 insertions(+), 2 deletions(-)
diff --git a/docs-site/docs/tensor.md b/docs-site/docs/tensor.md
index ca9c21e..5240e67 100644
--- a/docs-site/docs/tensor.md
+++ b/docs-site/docs/tensor.md
@@ -138,9 +138,28 @@ training completed
```
## Tensor implementation
+Previous section shows users the general usage of `Tensor`, the implementation
under the hood will be covered below. First, the design of python and C++
tensors will be introduced. Later part will talk about how the frontend(Python)
and backend(C++) are connected and how to extend them.
-SINGA has three different sets of implmentations of Tensor functions, one for
-each type of Device.
+### Python Tensor:
+
+Python class `Tensor`, defined in `python/singa/tensor.py`, provides high
level tensor manipulation access for implementing deep learning
architecture(autograd and models), as well as data management by end users.
+
+It primarily works by simply wrapping around C++ tensor methods, both
arithmetic (e.g. `sum`) and non arithmetic methods (e.g. `reshape`). Some
advanced arithmetic operations are later introduced and implemented in pure
python tensor API, e.g. `tensordot`. Python Tensor API could be used to
implement complex operations easily with the flexible methods available.
+
+API are grouped into `Tensor` methods, and functions that take `Tensor` as
inputs.
+
+### C++ Tensor:
+C++ class `Tensor`, defined in `include/singa/core/tensor.h`, primarily
manages the memory that holds the data, and provides low level APIs for tensor
manipulation. Also, it provides various arithmetic methods(e.g. `matmul`) by
wrapping different backends.
+
+#### Execution context and Memory Block
+Two important concepts or data structures for `Tensor` are execution context
`device`, and memory block `Block`.
+
+Each `Tensor` is linked to one device, representing the execution context
(CPU, GPU). Tensor math calculations are asynchronously delayed to be executed
on the linked device.
+
+Tensor data are stored in class `Block`, defined in
`include/singa/core/common.h`. `Block` owns the underlying data, while tensors
take ownership on the metadata describing the tensor, like `shape`, `strides`.
+
+#### Tensor Math backends:
+To leverage on the efficient math library provided by different backend
hardwards, SINGA has three different sets of implementations of Tensor
functions, one for each type of Device.
- 'tensor_math_cpp.h' implements operations using Cpp (with CBLAS) for CppGPU
devices.
@@ -149,6 +168,23 @@ each type of Device.
- 'tensor_math_opencl.h' implements operations using OpenCL for OpenclGPU
devices.
+### Connecting C++ Tensor and Python - SWIG
+
+While C++ Tensor could be used standalone as a Tensor library, it is further
extended to bridge Python code by [SWIG](http://www.swig.org/). SWIG can
automatically compile C++ API into Python modules.
+
+When compiling from source, several files are generated by SWIG, including
`python/singa/singa_wrap.py`. The Python `Tensor` class imports this module and
could use C++ API with ease.
+
+### Create new Tensor functions:
+With the groundwork set by the previous description, extending tensor
functions could be done easily in bottom up manner. For math operations, the
steps are:
+- Add new API to `tensor.h`
+- Add the code generation by predefined macro in `tensor.cc`, refer to
`GenUnaryTensorFn(Abs);` as an example.
+- Add template method/function placeholder in `tensor_math.h`
+- Fill up implementation at least for CPU(`tensor_math_cpp.h`) and
GPU(`tensor_math_cuda.h`)
+- Extend the API exposed to SWIG for translation in `src/api/core_tensor.i`
+- Wrap generated python API `python/singa/singa_wrap.py` and make it
consistent with python `Tensor` API.
+- Write unit tests where appropriate
+
+
## Python API
_work in progress_
