SINGA-237 New documentation files for SINGA v1.0 1. copy the 'examples' folder into docs/ to generate htmls files using the README.md files 2. add software_stack.md to describe the major data structures of v1.0 3. add device.rst to introduce the Device APIs
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/3299b0c2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/3299b0c2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/3299b0c2 Branch: refs/heads/master Commit: 3299b0c29e62f61dc7f74918fb01a0fa2c93a4a4 Parents: 6b2ff3c Author: Wei Wang <[email protected]> Authored: Thu Aug 11 23:13:12 2016 +0800 Committer: wangwei <[email protected]> Committed: Fri Aug 12 00:04:20 2016 +0800 ---------------------------------------------------------------------- doc/Makefile | 1 + doc/_static/images/singav1-sw.png | Bin 0 -> 24326 bytes doc/conf.py | 9 +-- doc/docs.rst | 6 +- doc/docs/device.rst | 47 ++++++++++++++++ doc/docs/examples.rst | 6 ++ doc/docs/index.rst | 15 ++--- doc/docs/jp/index.md | 23 -------- doc/docs/kr/index.md | 23 -------- doc/docs/software_stack.md | 99 +++++++++++++++++++++++++++++++++ doc/docs/tensor.md | 7 +++ doc/docs/zh/index.md | 10 ++-- 12 files changed, 177 insertions(+), 69 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/Makefile ---------------------------------------------------------------------- diff --git a/doc/Makefile b/doc/Makefile index 62a2236..c6eddf1 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -50,6 +50,7 @@ clean: .PHONY: html html: + cp -rf ../examples docs/ $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/_static/images/singav1-sw.png ---------------------------------------------------------------------- diff --git a/doc/_static/images/singav1-sw.png b/doc/_static/images/singav1-sw.png new file mode 100644 index 0000000..e443c6e Binary files /dev/null and b/doc/_static/images/singav1-sw.png differ http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/conf.py ---------------------------------------------------------------------- diff --git a/doc/conf.py b/doc/conf.py index 86dc031..20ba51a 100755 --- a/doc/conf.py +++ b/doc/conf.py @@ -16,9 +16,10 @@ # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # -# import os -# import sys -# sys.path.insert(0, os.path.abspath('.')) +import os +import sys +sys.path.insert(0, os.path.abspath('.')) +sys.path.insert(1, '../src/python/singa/') # -- General configuration ------------------------------------------------ from recommonmark.parser import CommonMarkParser @@ -35,7 +36,7 @@ source_parsers = { # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ - +'sphinx.ext.autodoc' ] # Add any paths that contain templates here, relative to this directory. http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs.rst ---------------------------------------------------------------------- diff --git a/doc/docs.rst b/doc/docs.rst index 2ebea60..400b12a 100644 --- a/doc/docs.rst +++ b/doc/docs.rst @@ -2,7 +2,5 @@ Documentation ============= .. toctree:: - docs/index - docs/zh/index - docs/jp/index - docs/kr/index + docs/index + docs/zh/index http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs/device.rst ---------------------------------------------------------------------- diff --git a/doc/docs/device.rst b/doc/docs/device.rst new file mode 100644 index 0000000..aa5defb --- /dev/null +++ b/doc/docs/device.rst @@ -0,0 +1,47 @@ +Device +======= + + +The Device abstract represent a hardware device with memory and compuation units. + +Specific devices +---------------- +Currently, SINGA has three Device implmentations, + +1. CudaGPU for an Nvidia GPU card which runs Cuda code +2. CppCPU for a CPU which runs Cpp code +3. OpenclGPU for a GPU card which runs OpenCL code + + +Create devices +--------------- + +Python API +~~~~~~~~~~ + +.. autofunction:: device.create_cuda_gpus + +.. autofunction:: device.create_cuda_gpus_on + +.. autofunction:: device.create_cuda_gpu_on + +.. autofunction:: device.get_default_device + + +The following code shows how to create devices, + +.. code:: python + + from singa import device + cuda = device.create_cuda_gpu_on(0) # use GPU card of ID 0 + host = device.get_default_device() # get the default host device (a CppCPU) + ary1 = device.create_cuda_gpus(2) # create 2 devices, starting from ID 0 + ary2 = device.create_cuda_gpus([0,2]) # create 2 devices on ID 0 and 2 + + + +CPP API +~~~~~~~ + + + http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs/examples.rst ---------------------------------------------------------------------- diff --git a/doc/docs/examples.rst b/doc/docs/examples.rst new file mode 100644 index 0000000..b0b2af8 --- /dev/null +++ b/doc/docs/examples.rst @@ -0,0 +1,6 @@ +Examples +======== + +.. toctree:: + + examples/index http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs/index.rst ---------------------------------------------------------------------- diff --git a/doc/docs/index.rst b/doc/docs/index.rst index a6a1b49..8a74976 100644 --- a/doc/docs/index.rst +++ b/doc/docs/index.rst @@ -2,15 +2,8 @@ English ======= .. toctree:: - overview - installation - quick-start - programming-guide - distributed-training - data - checkpoint - python - test - gpu - examples + installation + software_stack + device + examples http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs/jp/index.md ---------------------------------------------------------------------- diff --git a/doc/docs/jp/index.md b/doc/docs/jp/index.md deleted file mode 100644 index 6679198..0000000 --- a/doc/docs/jp/index.md +++ /dev/null @@ -1,23 +0,0 @@ -# ææ°ããã¥ã¡ã³ã - ---- - -* [ã¤ã³ãããã¯ã·ã§ã³](overview.html) -* [ã¤ã³ã¹ãã¼ã«](installation.html) -* [ã¯ã¤ãã¯ã¹ã¿ã¼ã](quick-start.html) -* [ããã°ã©ãã³ã° ã¬ã¤ã](programming-guide.html) - * [NeuralNet](neural-net.html) - * [Layer](layer.html) - * [Param](param.html) - * [TrainOneBatch](train-one-batch.html) - * [Updater](updater.html) -* [åæ£ ãã¬ã¼ãã³ã°](distributed-training.html) -* [ãã¼ã¿ã®æºå](data.html) -* [Checkpoint 㨠Resume](checkpoint.html) -* [ããã©ã¼ãã³ã¹ãã¹ã 㨠ç¹å¾´æ½åº](test.html) -* [ãµã³ãã«](examples.html) - * Feed-forward ã¢ãã« - * [CNN](cnn.html) - * [MLP](mlp.html) - * [RBM + Auto-encoder](rbm.html) - * [RNN](rnn.html) http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs/kr/index.md ---------------------------------------------------------------------- diff --git a/doc/docs/kr/index.md b/doc/docs/kr/index.md deleted file mode 100644 index 990d5d9..0000000 --- a/doc/docs/kr/index.md +++ /dev/null @@ -1,23 +0,0 @@ -# ìµì 문ì - ---- - -* [ê°ì](overview.html) -* [ì¸ì¤í¨](installation.html) -* [íµ ì¤íí¸](quick-start.html) -* [íë¡ê·¸ëë° ê°ì´ë](programming-guide.html) - * [NeuralNet](neural-net.html) - * [Layer](layer.html) - * [Param](param.html) - * [TrainOneBatch](train-one-batch.html) - * [Updater](updater.html) -* [ë¶ì° í¸ë ì´ë](distributed-training.html) -* [ë°ì´í° ì¤ë¹](data.html) -* [Checkpoint ì Resume](checkpoint.html) -* [ì±ë¥í ì¤í¸ ë° í¹ì§ì¶ì¶](test.html) -* [ìí](examples.html) - * Feed-forward ëª¨ë¸ - * [CNN](cnn.html) - * [MLP](mlp.html) - * [RBM + Auto-encoder](rbm.html) - * [RNN](rnn.html) http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs/software_stack.md ---------------------------------------------------------------------- diff --git a/doc/docs/software_stack.md b/doc/docs/software_stack.md new file mode 100644 index 0000000..c60b6a5 --- /dev/null +++ b/doc/docs/software_stack.md @@ -0,0 +1,99 @@ +# Software Stack + +SINGA's software stack includes three major components, namely, core, IO and +model. Figure 1 illustrates these components together with the hardware. +The core component provides memory management and tensor operations; +IO has classes for reading (and writing) data from (to) disk and network; The +model component provides data structures and algorithms for machine learning models, +e.g., layers for neural network models, optimizers/initializer/metric/loss for +general machine learning models. + + +<img src="../_static/images/singav1-sw.png" align="center" width="500px"/> +<br/> +<span><strong>Figure 1 - SINGA V1 software stack.</strong></span> + +## Core + +[Tensor](tensor.html) and [Device](device.html) are two core abstractions in SINGA. Tensor class represents a +multi-dimensional array, which stores model variables and provides linear algebra +operations for machine learning +algorithms, including matrix multiplication and random functions. Each tensor +instance (i.e. a tensor) is allocated on a Device instance. +Each Device instance (i.e. a device) is created against one hardware device, +e.g. a GPU card or a CPU core. Devices manage the memory of tensors and execute +tensor operations on its execution units, e.g. CPU threads or CUDA streams. + +Depending on the hardware and the programming language, SINGA have implemented +the following specific device classes: + +* **CudaGPU** represents an Nvidia GPU card. The execution units are the CUDA streams. +* **CppCPU** represents a normal CPU. The execution units are the CPU threads. +* **OpenclGPU** represents normal GPU card from both Nvidia and AMD. + The execution units are the CommandQueues. Given that OpenCL is compatible with + many hardware devices, e.g. FPGA and ARM, the OpenclGPU has the potential to be + extended for other devices. + +Different types of devices use different programming languages to write the kernel +functions for tensor operations, + +* CppMath (tensor_math_cpp.h) implements the tensor operations using Cpp for CppCPU +* CudaMath (tensor_math_cuda.h) implements the tensor operations using CUDA for CudaGPU +* OpenclMath (tensor_math_opencl.h) implements the tensor operations using OpenCL for OpenclGPU + +In addition, different types of data, such as float32 and float16, could be supported by adding +the corresponding tensor functions. + +Typically, users would create a device instance and pass it to create multiple +tensor instances. When users call the Tensor functions, these function would invoke +the corresponding implementation (CppMath/CudaMath/OpenclMath) automatically. In +other words, the implementation of Tensor operations is transparent to users. + +Most machine learning algorithms could be expressed using (dense or sparse) tensors. +Therefore, with the Tensor abstraction, SINGA would be able to run a wide range of models, +including deep learning models and other traditional machine learning models. + +The Tensor and Device abstractions are extensible to support a wide range of hardware device +using different programming languages. A new hardware device would be supported by +adding a new Device subclass and the corresponding implementation of the Tensor +operations (xxxMath). + +Optimizations in terms of speed and memory could be implemented by Device, which +manages both operation execution and memory malloc/free. More optimization details +would be described in the [Device page](device.html). + + +## Model + +On top of the Tensor and Device abstractions, SINGA provides some higher level +classes for machine learning modules. + +* [Layer](layer.html) and its subclasses are specific for neural networks. Every layer provides + functions for forward propagating features and backward propagating gradients w.r.t the training loss functions. + They wraps the complex layer operations so that users can easily create neural nets + by connecting a set of layers. + +* [Initializer](initializer.html) and its subclasses provide variant methods of initializing + model parameters (stored in Tensor instances), following Uniform, Gaussian, etc. + +* [Loss](loss.html) and its subclasses defines the training objective loss functions. + Both functions of computing the loss values and computing the gradient of the prediction w.r.t the + objective loss are implemented. Example loss functions include squared error and cross entropy. + +* [Metric](metric.html) and its subclasses provide the function to measure the + performance of the model, e.g., the accuracy. + +* [Optimizer](optimizer.html) and its subclasses implement the methods for updating + model parameter values using parameter gradients, including SGD, AdaGrad, RMSProp etc. + + +## IO + +The IO module consists of classes for data loading, data preprocessing and message passing. + +* Reader and its subclasses load string records from disk files +* Writer and its subclasses write string records to disk files +* Encoder and its subclasses encode Tensor instances into string records +* Decoder and its subclasses decodes string records into Tensor instances +* Endpoint represents a communication endpoint which provides functions for passing messages to each other. +* Message represents communication messages between Endpoint instances. It carries both meta data and payload. http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs/tensor.md ---------------------------------------------------------------------- diff --git a/doc/docs/tensor.md b/doc/docs/tensor.md new file mode 100644 index 0000000..eaf8362 --- /dev/null +++ b/doc/docs/tensor.md @@ -0,0 +1,7 @@ +# Tensor + + +## + + +## http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/3299b0c2/doc/docs/zh/index.md ---------------------------------------------------------------------- diff --git a/doc/docs/zh/index.md b/doc/docs/zh/index.md index c44a2cf..4b49d5f 100644 --- a/doc/docs/zh/index.md +++ b/doc/docs/zh/index.md @@ -1,7 +1,9 @@ SINGA ä¸æææ¡£ ---- +============== -* [ç®ä»](overview.html) -* [å®è£ ](installation_source.html) -* [使ç¨æå](programming-guide.html) +.. toctree:: + + overview + installation_source + programming-guide
