Author: jinyang
Date: Sat Sep 26 12:09:58 2015
New Revision: 1705434
URL: http://svn.apache.org/viewvc?rev=1705434&view=rev
Log:
unified title format
Modified:
incubator/singa/site/trunk/content/markdown/docs/code-structure.md
incubator/singa/site/trunk/content/markdown/docs/data.md
incubator/singa/site/trunk/content/markdown/docs/debug.md
incubator/singa/site/trunk/content/markdown/docs/distributed-training.md
incubator/singa/site/trunk/content/markdown/docs/examples.md
incubator/singa/site/trunk/content/markdown/docs/frameworks.md
incubator/singa/site/trunk/content/markdown/docs/model-config.md
incubator/singa/site/trunk/content/markdown/docs/neuralnet-partition.md
incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md
incubator/singa/site/trunk/content/markdown/docs/programming-guide.md
incubator/singa/site/trunk/content/markdown/docs/train-one-batch.md
Modified: incubator/singa/site/trunk/content/markdown/docs/code-structure.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/code-structure.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/code-structure.md
(original)
+++ incubator/singa/site/trunk/content/markdown/docs/code-structure.md Sat Sep
26 12:09:58 2015
@@ -1,6 +1,6 @@
# Code Structure
-___
+---
<!--
Modified: incubator/singa/site/trunk/content/markdown/docs/data.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/data.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/data.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/data.md Sat Sep 26
12:09:58 2015
@@ -1,5 +1,7 @@
# Data Preparation
+---
+
To submit a training job, users need to convert raw data (e.g., images, text
documents) into SINGA recognizable [Record](../api/classsinga_1_1Record.html)s.
SINGA uses [data layers](layer#data-layers)
Modified: incubator/singa/site/trunk/content/markdown/docs/debug.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/debug.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/debug.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/debug.md Sat Sep 26
12:09:58 2015
@@ -1,5 +1,6 @@
# How to Debug
-___
+
+---
Since SINGA is developed on Linux using C++, GDB is the preferred debugging
tool. To use GDB, the code must be compiled with `-g` flag. This is enabled by
Modified:
incubator/singa/site/trunk/content/markdown/docs/distributed-training.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/distributed-training.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/distributed-training.md
(original)
+++ incubator/singa/site/trunk/content/markdown/docs/distributed-training.md
Sat Sep 26 12:09:58 2015
@@ -1,5 +1,7 @@
# Distributed Training
+---
+
SINGA is designed for distributed training of large deep learning models with
huge amount of training data.
Modified: incubator/singa/site/trunk/content/markdown/docs/examples.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/examples.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/examples.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/examples.md Sat Sep 26
12:09:58 2015
@@ -1,5 +1,7 @@
# Example Models
+---
+
Different models are provided as examples to help users get familiar with
SINGA.
[Neural Network](neural-net.html) gives details on the models that are
supported by SINGA.
Modified: incubator/singa/site/trunk/content/markdown/docs/frameworks.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/frameworks.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/frameworks.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/frameworks.md Sat Sep 26
12:09:58 2015
@@ -1,7 +1,8 @@
# Distributed Training Framework
+---
-### Cluster Topology Configuration
+## Cluster Topology Configuration
Here we describe how to configure SINGA's cluster topology to support
different distributed training frameworks.
@@ -32,7 +33,7 @@ The mostly used fields are as follows:
* `server_worker_separate`:
separate servers and workers in different processes.
-### Different Training Frameworks
+## Different Training Frameworks
In SINGA, worker groups run asynchronously and
workers within one group run synchronously.
@@ -44,7 +45,7 @@ popular distributed training frameworks
<img src="../images/frameworks.png" style="width: 800px"/>
<p><strong> Fig.1 - Training frameworks in SINGA</strong></p>
-####Sandblaster
+###Sandblaster
This is a **synchronous** framework used by Google Brain.
Fig.2(a) shows the Sandblaster framework implemented in SINGA.
@@ -63,7 +64,7 @@ A worker computes on its partition of th
and only communicates with servers handling related parameters.
-####AllReduce
+###AllReduce
This is a **synchronous** framework used by Baidu's DeepImage.
Fig.2(b) shows the AllReduce framework implemented in SINGA.
@@ -81,7 +82,7 @@ We bind each worker with a server on the
node is responsible for maintaining a partition of parameters and
collecting updates from all other nodes.
-####Downpour
+###Downpour
This is a **asynchronous** framework used by Google Brain.
Fig.2(c) shows the Downpour framework implemented in SINGA.
@@ -100,7 +101,7 @@ requests to a global server group. We di
worker groups, each running independently and working on parameters
from the last *update* response.
-####Distributed Hogwild
+###Distributed Hogwild
This is a **asynchronous** framework used by Caffe.
Fig.2(d) shows the Distributed Hogwild framework implemented in SINGA.
Modified: incubator/singa/site/trunk/content/markdown/docs/model-config.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/model-config.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/model-config.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/model-config.md Sat Sep 26
12:09:58 2015
@@ -1,5 +1,7 @@
# Model Configuration
+---
+
SINGA uses the stochastic gradient descent (SGD) algorithm to train parameters
of deep learning models. For each SGD iteration, there is a
[Worker](architecture.html) computing
@@ -12,9 +14,9 @@ workspace folder. E.g., the [cifar10 exa
has a model.conf file.
-### NeuralNet
+## NeuralNet
-#### Uniform model (neuralnet) representation
+### Uniform model (neuralnet) representation
<img src = "../images/model-categorization.png" style = "width: 400px"> Fig. 1:
Deep learning model categorization</img>
@@ -66,7 +68,7 @@ instances in the layer class, e.g.,
Param weight;
-#### Configure the structure of a NeuralNet instance
+### Configure the structure of a NeuralNet instance
To train a deep learning model, the first step is to write the configurations
for the model structure, i.e., the layers and connections for the NeuralNet.
@@ -150,7 +152,7 @@ model](https://github.com/apache/incubat
<img src="../images/dcnn-cifar10.png" style = "width: 200px"> Fig. 4: Neural
network structure for the example DCNN model (cifar10)</img>
-### Worker
+## Worker
At the beginning, the Work will initialize the values of Param instances of
each layer either randomly (according to user configured distribution) or
@@ -222,7 +224,7 @@ All algorithms will finally call two fun
All [Layer implementations]() must implement the above two functions.
-### Updater
+## Updater
Once the gradients of parameters are computed, the Updater will update
parameter values. There are many SGD variants for updating parameters, like
@@ -285,7 +287,7 @@ Users can configure them according to th
}
-### Other model configuration fields
+## Other model configuration fields
Some other important configuration fields for training a deep learning model is
listed:
Modified:
incubator/singa/site/trunk/content/markdown/docs/neuralnet-partition.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/neuralnet-partition.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/neuralnet-partition.md
(original)
+++ incubator/singa/site/trunk/content/markdown/docs/neuralnet-partition.md Sat
Sep 26 12:09:58 2015
@@ -1,6 +1,6 @@
# Neural Net Partition
-___
+---
The purposes of partitioning neural network is to distribute the partitions
onto
different working units (e.g., threads or nodes, called workers in this
article)
Modified: incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md
(original)
+++ incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md Sat
Sep 26 12:09:58 2015
@@ -1,6 +1,6 @@
# Programmer Guide
-___
+---
To submit a training job, users must provide the configuration of the
four components shown in Figure 1:
Modified: incubator/singa/site/trunk/content/markdown/docs/programming-guide.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/programming-guide.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/programming-guide.md
(original)
+++ incubator/singa/site/trunk/content/markdown/docs/programming-guide.md Sat
Sep 26 12:09:58 2015
@@ -1,5 +1,6 @@
# Programming Guide
+---
To submit a training job, users must provide the configuration of the
four components shown in Figure 1:
Modified: incubator/singa/site/trunk/content/markdown/docs/train-one-batch.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/train-one-batch.md?rev=1705434&r1=1705433&r2=1705434&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/train-one-batch.md
(original)
+++ incubator/singa/site/trunk/content/markdown/docs/train-one-batch.md Sat Sep
26 12:09:58 2015
@@ -1,5 +1,7 @@
# Train-One-Batch
+---
+
For each SGD iteration, every worker calls the `TrainOneBatch` function to
compute gradients of parameters associated with local layers (i.e., layers
dispatched to it). SINGA has implemented two algorithms for the
