Author: wangwei
Date: Thu Jul 23 06:03:42 2015
New Revision: 1692350
URL: http://svn.apache.org/r1692350
Log:
rename user-guide to programmer-guide
Added:
incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md
Removed:
incubator/singa/site/trunk/content/markdown/docs/user-guide.md
Modified:
incubator/singa/site/trunk/content/markdown/docs.md
incubator/singa/site/trunk/content/markdown/introduction.md
incubator/singa/site/trunk/content/markdown/quick-start.md
incubator/singa/site/trunk/content/site.xml
Modified: incubator/singa/site/trunk/content/markdown/docs.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs.md?rev=1692350&r1=1692349&r2=1692350&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs.md Thu Jul 23 06:03:42 2015
@@ -3,7 +3,8 @@
___
* [Installation](docs/installation.html)
+* [Programmer Guide](docs/programmer-guide.html)
* [System Architecture](docs/architecture.html)
* [Communication](docs/communication.html)
-* [Neural Network Partition](docs/neuralnet-partition.html)
-* [Programming Model](docs/programming-model.html)
+* [Training Examples](docs/examples.html)
+
Added: incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md?rev=1692350&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md (added)
+++ incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md Thu
Jul 23 06:03:42 2015
@@ -0,0 +1,101 @@
+## Programming Model
+
+We describe the programming model of SINGA to provide users instructions of
+implementing a new model and submitting the training job. The programming model
+is made almost transparent to the underlying distributed environment. Hence
+users do not need to worry much about the communication and synchronization of
+nodes, which is discussed in [architecture](architecture.html) in details.
+
+### Deep learning training
+
+Deep learning is labeled as a feature learning technique, which usually
+consists of multiple layers. Each layer is associated a feature transformation
+function. After going through all layers, the raw input feature (e.g., pixels
+of images) would be converted into a high-level feature that is easier for
+tasks like classification.
+
+Training a deep learning model is to find the optimal parameters involved in
+the transformation functions that generates good features for specific tasks.
+The goodness of a set of parameters is measured by a loss function, e.g.,
+[Cross-Entropy Loss](https://en.wikipedia.org/wiki/Cross_entropy). Since the
+loss functions are usually non-linear and non-convex, it is difficult to get a
+closed form solution. Normally, people uses the SGD algorithm which randomly
+initializes the parameters and then iteratively update them to reduce the loss.
+
+
+### Steps to submit a training job
+
+SINGA uses the stochastic gradient descent (SGD) algorithm to train parameters
+of deep learning models. For each SGD iteration, there is a
+[Worker](architecture.html) computing gradients of parameters from the
+NeuralNet and a [Updater]() updating parameter values based on gradients. SINGA
+has implemented three algorithms for gradient calculation, namely Back
+propagation algorithm for feed-forward models, back-propagation through time
+for recurrent neural networks and contrastive divergence for energy models like
+RBM and DBM. Variant SGD updaters are also provided, including
+[AdaDelta](http://arxiv.org/pdf/1212.5701v1.pdf),
+[AdaGrad](http://www.magicbroom.info/Papers/DuchiHaSi10.pdf),
+[RMSProp](http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf),
+[Nesterov](http://scholar.google.com/citations?view_op=view_citation&hl=en&user=DJ8Ep8YAAAAJ&citation_for_view=DJ8Ep8YAAAAJ:hkOj_22Ku90C).
+
+Consequently, what a user needs to do to submit a training job is
+
+ 1. [Prepare the data](data.html) for training, validation and test.
+
+ 2. [Implement the new Layers](layer.html) to support specific feature
transformations
+ required in the new model.
+
+ 3. Configure the training job including the [cluster
setting](architecture.html)
+ and [model configuration](model-config.html)
+
+### Driver program
+
+Each training job has a driver program that
+
+ * registers the layers implemented by the user and,
+
+ * starts the
[Trainer](https://github.com/apache/incubator-singa/blob/master/include/trainer/trainer.h)
+ by providing the job configuration.
+
+An example driver program is like
+
+ #include "singa.h"
+ #include "user-layer.h" // header for user defined layers
+
+ DEFINE_int32(job, -1, "Job ID"); // job ID generated by the SINGA script
+ DEFINE_string(workspace, "examples/mnist/", "workspace of the training
job");
+ DEFINE_bool(resume, false, "resume from checkpoint");
+
+ int main(int argc, char** argv) {
+ google::InitGoogleLogging(argv[0]);
+ gflags::ParseCommandLineFlags(&argc, &argv, true);
+
+ // register all user defined layers in user-layer.h
+ Register(kFooLayer, FooLayer);
+ ...
+
+ JobProto jobConf;
+ // read job configuration from text conf file
+ ReadProtoFromTextFile(&jobConf, FLAGS_workspace + "/job.conf");
+ Trainer trainer;
+ trainer.Start(FLAGS_job, jobConf, FLAGS_resume);
+ }
+
+Users can also configure the job in the driver program instead of writing the
+configuration file
+
+
+ JobProto jobConf;
+ jobConf.set_job_name("my singa job");
+ ... // configure cluster and model
+ Trainer trainer;
+ trainer.Start(FLAGS_job, jobConf, FLAGS_resume);
+
+We will provide helper functions to make the configuration easier in the
+future, like [keras](https://github.com/fchollet/keras).
+
+Compile and link the driver program with singa library to generate an
+executable, e.g., with name `mysinga`. To submit the job, just pass the path of
+the executable and the workspace to the singa job submission script
+
+ ./bin/singa-run.sh <path to mysinga> -workspace=<my job workspace>
Modified: incubator/singa/site/trunk/content/markdown/introduction.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/introduction.md?rev=1692350&r1=1692349&r2=1692350&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/introduction.md (original)
+++ incubator/singa/site/trunk/content/markdown/introduction.md Thu Jul 23
06:03:42 2015
@@ -59,10 +59,10 @@ general enough for programmers to overri
### Where to go from here
- * SINGA [User guide](user-guide.html) describes how to submit a
+ * SINGA [Programmer guide](user-guide.html) describes how to submit a
training job for your own deep learning model.
- * SINGA [architecture](architecture.html) illustrates how different training
frameworks are
+ * SINGA [Architecture](architecture.html) illustrates how different training
frameworks are
supported using a general system architecture.
* [Training examples](examples.html) are provided to help users get started
with SINGA.
Modified: incubator/singa/site/trunk/content/markdown/quick-start.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/quick-start.md?rev=1692350&r1=1692349&r2=1692350&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/quick-start.md (original)
+++ incubator/singa/site/trunk/content/markdown/quick-start.md Thu Jul 23
06:03:42 2015
@@ -129,7 +129,7 @@ there is no model partitioning. More det
described in the [System Architecture](docs/architecture.html) page.
-->
-#### Distributed Training
+### Distributed Training
To train the model in a distributed environment, we first change the job
configuration, e.g., using 2 worker groups (one worker per group) and 2 servers
Modified: incubator/singa/site/trunk/content/site.xml
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/site.xml?rev=1692350&r1=1692349&r2=1692350&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/site.xml (original)
+++ incubator/singa/site/trunk/content/site.xml Thu Jul 23 06:03:42 2015
@@ -55,7 +55,7 @@
<menu name="Documentaion">
<item name="Installation" href="docs/installation.html"/>
- <item name="User Guide" href="docs/user-guide.html">
+ <item name="Programmer Guide" href="docs/programmer-guide.html">
<item name ="Model Configuration" href="docs/model-config.html"/>
<item name="Neural Network" href="docs/neuralnet.html"/>
<item name="Layer" href="docs/layer.html"/>
