Modified: incubator/singa/site/trunk/target/site/docs/neuralnet-partition.html URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/target/site/docs/neuralnet-partition.html?rev=1700817&r1=1700816&r2=1700817&view=diff ============================================================================== --- incubator/singa/site/trunk/target/site/docs/neuralnet-partition.html (original) +++ incubator/singa/site/trunk/target/site/docs/neuralnet-partition.html Wed Sep 2 13:53:06 2015 @@ -9,7 +9,7 @@ <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="Date-Revision-yyyymmdd" content="20150902" /> <meta http-equiv="Content-Language" content="en" /> - <title>Apache SINGA – Neural Network Partition</title> + <title>Apache SINGA – Neural Net Partition</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" /> <link rel="stylesheet" href="../css/site.css" /> <link rel="stylesheet" href="../css/print.css" media="print" /> @@ -55,32 +55,36 @@ <li> <a href="../index.html" title="Welcome">Welcome</a> </li> - - <li> <a href="../introduction.html" title="Introduction">Introduction</a> -</li> - - <li> <a href="../quick-start.html" title="Quick Start">Quick Start</a> -</li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a> <ul class="dropdown-menu"> + <li> <a href="../docs/overview.html" title="Introduction">Introduction</a> +</li> + <li> <a href="../docs/installation.html" title="Installation">Installation</a> </li> + <li> <a href="../docs/quick-start.html" title="Quick Start">Quick Start</a> +</li> + <li class="dropdown-submenu"> <a href="../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a> <ul class="dropdown-menu"> <li> <a href="../docs/model-config.html" title="Model Configuration">Model Configuration</a> </li> - <li> <a href="../docs/neuralnet.html" title="Neural Network">Neural Network</a> + <li> <a href="../docs/neural-net.html" title="Neural Network">Neural Network</a> </li> <li> <a href="../docs/layer.html" title="Layer">Layer</a> </li> <li> <a href="../docs/param.html" title="Param">Param</a> </li> + <li> <a href="../docs/train-one-batch.html" title="TrainOneBatch">TrainOneBatch</a> +</li> + <li> <a href="../docs/updater.html" title="Updater">Updater</a> +</li> </ul> </li> @@ -102,8 +106,19 @@ <li> <a href="../docs/checkpoint.html" title="Checkpoint">Checkpoint</a> </li> - <li> <a href="../docs/examples.html" title="Examples">Examples</a> + <li class="dropdown-submenu"> + <a href="../docs/examples.html" title="Examples">Examples</a> + <ul class="dropdown-menu"> + <li> <a href="../docs/cnn.html" title="CNN">CNN</a> +</li> + <li> <a href="../docs/mlp.html" title="MLP">MLP</a> </li> + <li> <a href="../docs/rbm.html" title="RBM + Auto-encoder">RBM + Auto-encoder</a> +</li> + <li> <a href="../docs/rnn.html" title="RNN">RNN</a> +</li> + </ul> + </li> <li> <a href="../docs/debug.html" title="Debug">Debug</a> </li> @@ -189,7 +204,7 @@ Apache SINGA</a> <span class="divider">/</span> </li> - <li class="active ">Neural Network Partition</li> + <li class="active ">Neural Net Partition</li> @@ -212,29 +227,29 @@ <span class="none"></span> Welcome</a> </li> - + <li class="nav-header">Documentaion</li> + <li> - <a href="../introduction.html" title="Introduction"> + <a href="../docs/overview.html" title="Introduction"> <span class="none"></span> Introduction</a> </li> <li> - <a href="../quick-start.html" title="Quick Start"> + <a href="../docs/installation.html" title="Installation"> <span class="none"></span> - Quick Start</a> + Installation</a> </li> - <li class="nav-header">Documentaion</li> - + <li> - <a href="../docs/installation.html" title="Installation"> + <a href="../docs/quick-start.html" title="Quick Start"> <span class="none"></span> - Installation</a> + Quick Start</a> </li> - + <li> <a href="../docs/programmer-guide.html" title="Programmer Guide"> @@ -251,7 +266,7 @@ <li> - <a href="../docs/neuralnet.html" title="Neural Network"> + <a href="../docs/neural-net.html" title="Neural Network"> <span class="none"></span> Neural Network</a> </li> @@ -269,6 +284,20 @@ <span class="none"></span> Param</a> </li> + + <li> + + <a href="../docs/train-one-batch.html" title="TrainOneBatch"> + <span class="none"></span> + TrainOneBatch</a> + </li> + + <li> + + <a href="../docs/updater.html" title="Updater"> + <span class="none"></span> + Updater</a> + </li> </ul> </li> @@ -315,13 +344,43 @@ <span class="none"></span> Checkpoint</a> </li> - + <li> <a href="../docs/examples.html" title="Examples"> - <span class="none"></span> + <span class="icon-chevron-down"></span> Examples</a> + <ul class="nav nav-list"> + + <li> + + <a href="../docs/cnn.html" title="CNN"> + <span class="none"></span> + CNN</a> </li> + + <li> + + <a href="../docs/mlp.html" title="MLP"> + <span class="none"></span> + MLP</a> + </li> + + <li> + + <a href="../docs/rbm.html" title="RBM + Auto-encoder"> + <span class="none"></span> + RBM + Auto-encoder</a> + </li> + + <li> + + <a href="../docs/rnn.html" title="RNN"> + <span class="none"></span> + RNN</a> + </li> + </ul> + </li> <li> @@ -425,8 +484,7 @@ <div id="bodyColumn" class="span10" > - <div class="section"> -<h2><a name="Neural_Network_Partition"></a>Neural Network Partition</h2> + <h1>Neural Net Partition</h1> <hr /> <p>The purposes of partitioning neural network is to distribute the partitions onto different working units (e.g., threads or nodes, called workers in this article) and parallelize the processing. Another reason for partition is to handle large neural network which cannot be hold in a single node. For instance, to train models against images with high resolution we need large neural networks (in terms of training parameters).</p> <p>Since <i>Layer</i> is the first class citizen in SIGNA, we do the partition against layers. Specifically, we support partitions at two levels. First, users can configure the location (i.e., worker ID) of each layer. In this way, users assign one worker for each layer. Secondly, for one layer, we can partition its neurons or partition the instances (e.g, images). They are called layer partition and data partition respectively. We illustrate the two types of partitions using an simple convolutional neural network.</p> @@ -436,7 +494,7 @@ <p>The above figure shows the convolutional neural network after partitioning all layers except the DataLayer and ParserLayers, into 3 partitions using data partition. The read layers process 4 images of the batch, the black and blue layers process 2 images respectively. Some helper layers, i.e., SliceLayer, ConcateLayer, BridgeSrcLayer, BridgeDstLayer and SplitLayer, are added automatically by our partition algorithm. Layers of the same color resident in the same worker. There would be data transferring across different workers at the boundary layers (i.e., BridgeSrcLayer and BridgeDstLayer), e.g., between s-slice-mnist-conv1 and d-slice-mnist-conv1.</p> <p><img src="../images/conv-mnist-layerp.png" style="width: 1000px" alt="" /></p> <p>The above figure shows the convolutional neural network after partitioning all layers except the DataLayer and ParserLayers, into 2 partitions using layer partition. We can see that each layer processes all 8 images from the batch. But different partitions process different part of one image. For instance, the layer conv1-00 process only 4 channels. The other 4 channels are processed by conv1-01 which residents in another worker.</p> -<p>Since the partition is done at the layer level, we can apply different partitions for different layers to get a hybrid partition for the whole neural network. Moreover, we can also specify the layer locations to locate different layers to different workers.</p></div> +<p>Since the partition is done at the layer level, we can apply different partitions for different layers to get a hybrid partition for the whole neural network. Moreover, we can also specify the layer locations to locate different layers to different workers.</p> </div> </div> </div>
Modified: incubator/singa/site/trunk/target/site/docs/overview.html URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/target/site/docs/overview.html?rev=1700817&r1=1700816&r2=1700817&view=diff ============================================================================== --- incubator/singa/site/trunk/target/site/docs/overview.html (original) +++ incubator/singa/site/trunk/target/site/docs/overview.html Wed Sep 2 13:53:06 2015 @@ -9,7 +9,7 @@ <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="Date-Revision-yyyymmdd" content="20150902" /> <meta http-equiv="Content-Language" content="en" /> - <title>Apache SINGA – </title> + <title>Apache SINGA – Introduction</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" /> <link rel="stylesheet" href="../css/site.css" /> <link rel="stylesheet" href="../css/print.css" media="print" /> @@ -55,32 +55,36 @@ <li> <a href="../index.html" title="Welcome">Welcome</a> </li> - - <li> <a href="../introduction.html" title="Introduction">Introduction</a> -</li> - - <li> <a href="../quick-start.html" title="Quick Start">Quick Start</a> -</li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a> <ul class="dropdown-menu"> + <li> <a href="../docs/overview.html" title="Introduction">Introduction</a> +</li> + <li> <a href="../docs/installation.html" title="Installation">Installation</a> </li> + <li> <a href="../docs/quick-start.html" title="Quick Start">Quick Start</a> +</li> + <li class="dropdown-submenu"> <a href="../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a> <ul class="dropdown-menu"> <li> <a href="../docs/model-config.html" title="Model Configuration">Model Configuration</a> </li> - <li> <a href="../docs/neuralnet.html" title="Neural Network">Neural Network</a> + <li> <a href="../docs/neural-net.html" title="Neural Network">Neural Network</a> </li> <li> <a href="../docs/layer.html" title="Layer">Layer</a> </li> <li> <a href="../docs/param.html" title="Param">Param</a> </li> + <li> <a href="../docs/train-one-batch.html" title="TrainOneBatch">TrainOneBatch</a> +</li> + <li> <a href="../docs/updater.html" title="Updater">Updater</a> +</li> </ul> </li> @@ -102,8 +106,19 @@ <li> <a href="../docs/checkpoint.html" title="Checkpoint">Checkpoint</a> </li> - <li> <a href="../docs/examples.html" title="Examples">Examples</a> + <li class="dropdown-submenu"> + <a href="../docs/examples.html" title="Examples">Examples</a> + <ul class="dropdown-menu"> + <li> <a href="../docs/cnn.html" title="CNN">CNN</a> +</li> + <li> <a href="../docs/mlp.html" title="MLP">MLP</a> </li> + <li> <a href="../docs/rbm.html" title="RBM + Auto-encoder">RBM + Auto-encoder</a> +</li> + <li> <a href="../docs/rnn.html" title="RNN">RNN</a> +</li> + </ul> + </li> <li> <a href="../docs/debug.html" title="Debug">Debug</a> </li> @@ -189,7 +204,7 @@ Apache SINGA</a> <span class="divider">/</span> </li> - <li class="active "></li> + <li class="active ">Introduction</li> @@ -212,29 +227,27 @@ <span class="none"></span> Welcome</a> </li> + <li class="nav-header">Documentaion</li> + + <li class="active"> + + <a href="#"><span class="none"></span>Introduction</a> + </li> <li> - <a href="../introduction.html" title="Introduction"> + <a href="../docs/installation.html" title="Installation"> <span class="none"></span> - Introduction</a> + Installation</a> </li> <li> - <a href="../quick-start.html" title="Quick Start"> + <a href="../docs/quick-start.html" title="Quick Start"> <span class="none"></span> Quick Start</a> </li> - <li class="nav-header">Documentaion</li> - - <li> - - <a href="../docs/installation.html" title="Installation"> - <span class="none"></span> - Installation</a> - </li> - + <li> <a href="../docs/programmer-guide.html" title="Programmer Guide"> @@ -251,7 +264,7 @@ <li> - <a href="../docs/neuralnet.html" title="Neural Network"> + <a href="../docs/neural-net.html" title="Neural Network"> <span class="none"></span> Neural Network</a> </li> @@ -269,6 +282,20 @@ <span class="none"></span> Param</a> </li> + + <li> + + <a href="../docs/train-one-batch.html" title="TrainOneBatch"> + <span class="none"></span> + TrainOneBatch</a> + </li> + + <li> + + <a href="../docs/updater.html" title="Updater"> + <span class="none"></span> + Updater</a> + </li> </ul> </li> @@ -315,13 +342,43 @@ <span class="none"></span> Checkpoint</a> </li> - + <li> <a href="../docs/examples.html" title="Examples"> - <span class="none"></span> + <span class="icon-chevron-down"></span> Examples</a> + <ul class="nav nav-list"> + + <li> + + <a href="../docs/cnn.html" title="CNN"> + <span class="none"></span> + CNN</a> + </li> + + <li> + + <a href="../docs/mlp.html" title="MLP"> + <span class="none"></span> + MLP</a> + </li> + + <li> + + <a href="../docs/rbm.html" title="RBM + Auto-encoder"> + <span class="none"></span> + RBM + Auto-encoder</a> </li> + + <li> + + <a href="../docs/rnn.html" title="RNN"> + <span class="none"></span> + RNN</a> + </li> + </ul> + </li> <li> @@ -425,11 +482,8 @@ <div id="bodyColumn" class="span10" > - <p>— layout: post title: Introduction</p> -<div class="section"> -<h2><a name="category:_docs"></a>category: docs</h2> -<p>{% include JB/setup %}</p> -<p>SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is designed with an intuitive programming model based on the layer abstraction. A variety of popular deep learning models are supported, namely feed-forward models including convolutional neural networks (CNN), energy models like restricted Boltzmann machine (RBM), and recurrent neural networks (RNN). Many built-in layers are provided for users. SINGA architecture is sufficiently flexible to run synchronous, asynchronous and hybrid training frameworks. SINGA also supports different neural net partitioning schemes to parallelize the training of large models, namely partitioning on batch dimension, feature dimension or hybrid partitioning.</p></div> + <h1>Introduction</h1> +<p>SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is designed with an intuitive programming model based on the layer abstraction. A variety of popular deep learning models are supported, namely feed-forward models including convolutional neural networks (CNN), energy models like restricted Boltzmann machine (RBM), and recurrent neural networks (RNN). Many built-in layers are provided for users. SINGA architecture is sufficiently flexible to run synchronous, asynchronous and hybrid training frameworks. SINGA also supports different neural net partitioning schemes to parallelize the training of large models, namely partitioning on batch dimension, feature dimension or hybrid partitioning.</p> <div class="section"> <h2><a name="Goals"></a>Goals</h2> <p>As a distributed system, the first goal of SINGA is to have good scalability. In other words, SINGA is expected to reduce the total training time to achieve certain accuracy with more computing resources (i.e., machines).</p> Modified: incubator/singa/site/trunk/target/site/docs/param.html URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/target/site/docs/param.html?rev=1700817&r1=1700816&r2=1700817&view=diff ============================================================================== --- incubator/singa/site/trunk/target/site/docs/param.html (original) +++ incubator/singa/site/trunk/target/site/docs/param.html Wed Sep 2 13:53:06 2015 @@ -9,7 +9,7 @@ <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="Date-Revision-yyyymmdd" content="20150902" /> <meta http-equiv="Content-Language" content="en" /> - <title>Apache SINGA – </title> + <title>Apache SINGA – Parameters</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" /> <link rel="stylesheet" href="../css/site.css" /> <link rel="stylesheet" href="../css/print.css" media="print" /> @@ -55,32 +55,36 @@ <li> <a href="../index.html" title="Welcome">Welcome</a> </li> - - <li> <a href="../introduction.html" title="Introduction">Introduction</a> -</li> - - <li> <a href="../quick-start.html" title="Quick Start">Quick Start</a> -</li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a> <ul class="dropdown-menu"> + <li> <a href="../docs/overview.html" title="Introduction">Introduction</a> +</li> + <li> <a href="../docs/installation.html" title="Installation">Installation</a> </li> + <li> <a href="../docs/quick-start.html" title="Quick Start">Quick Start</a> +</li> + <li class="dropdown-submenu"> <a href="../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a> <ul class="dropdown-menu"> <li> <a href="../docs/model-config.html" title="Model Configuration">Model Configuration</a> </li> - <li> <a href="../docs/neuralnet.html" title="Neural Network">Neural Network</a> + <li> <a href="../docs/neural-net.html" title="Neural Network">Neural Network</a> </li> <li> <a href="../docs/layer.html" title="Layer">Layer</a> </li> <li> <a href="../docs/param.html" title="Param">Param</a> </li> + <li> <a href="../docs/train-one-batch.html" title="TrainOneBatch">TrainOneBatch</a> +</li> + <li> <a href="../docs/updater.html" title="Updater">Updater</a> +</li> </ul> </li> @@ -102,8 +106,19 @@ <li> <a href="../docs/checkpoint.html" title="Checkpoint">Checkpoint</a> </li> - <li> <a href="../docs/examples.html" title="Examples">Examples</a> + <li class="dropdown-submenu"> + <a href="../docs/examples.html" title="Examples">Examples</a> + <ul class="dropdown-menu"> + <li> <a href="../docs/cnn.html" title="CNN">CNN</a> +</li> + <li> <a href="../docs/mlp.html" title="MLP">MLP</a> </li> + <li> <a href="../docs/rbm.html" title="RBM + Auto-encoder">RBM + Auto-encoder</a> +</li> + <li> <a href="../docs/rnn.html" title="RNN">RNN</a> +</li> + </ul> + </li> <li> <a href="../docs/debug.html" title="Debug">Debug</a> </li> @@ -189,7 +204,7 @@ Apache SINGA</a> <span class="divider">/</span> </li> - <li class="active "></li> + <li class="active ">Parameters</li> @@ -212,29 +227,29 @@ <span class="none"></span> Welcome</a> </li> - + <li class="nav-header">Documentaion</li> + <li> - <a href="../introduction.html" title="Introduction"> + <a href="../docs/overview.html" title="Introduction"> <span class="none"></span> Introduction</a> </li> <li> - <a href="../quick-start.html" title="Quick Start"> + <a href="../docs/installation.html" title="Installation"> <span class="none"></span> - Quick Start</a> + Installation</a> </li> - <li class="nav-header">Documentaion</li> - + <li> - <a href="../docs/installation.html" title="Installation"> + <a href="../docs/quick-start.html" title="Quick Start"> <span class="none"></span> - Installation</a> + Quick Start</a> </li> - + <li> <a href="../docs/programmer-guide.html" title="Programmer Guide"> @@ -251,7 +266,7 @@ <li> - <a href="../docs/neuralnet.html" title="Neural Network"> + <a href="../docs/neural-net.html" title="Neural Network"> <span class="none"></span> Neural Network</a> </li> @@ -267,6 +282,20 @@ <a href="#"><span class="none"></span>Param</a> </li> + + <li> + + <a href="../docs/train-one-batch.html" title="TrainOneBatch"> + <span class="none"></span> + TrainOneBatch</a> + </li> + + <li> + + <a href="../docs/updater.html" title="Updater"> + <span class="none"></span> + Updater</a> + </li> </ul> </li> @@ -313,13 +342,43 @@ <span class="none"></span> Checkpoint</a> </li> - + <li> <a href="../docs/examples.html" title="Examples"> - <span class="none"></span> + <span class="icon-chevron-down"></span> Examples</a> + <ul class="nav nav-list"> + + <li> + + <a href="../docs/cnn.html" title="CNN"> + <span class="none"></span> + CNN</a> + </li> + + <li> + + <a href="../docs/mlp.html" title="MLP"> + <span class="none"></span> + MLP</a> + </li> + + <li> + + <a href="../docs/rbm.html" title="RBM + Auto-encoder"> + <span class="none"></span> + RBM + Auto-encoder</a> </li> + + <li> + + <a href="../docs/rnn.html" title="RNN"> + <span class="none"></span> + RNN</a> + </li> + </ul> + </li> <li> @@ -423,11 +482,8 @@ <div id="bodyColumn" class="span10" > - <p>— layout: post title: Param category : docs</p> -<div class="section"> -<h2><a name="tags_:_parameter_"></a>tags : [parameter ]</h2> -<p>{% include JB/setup %}</p> -<p>A <tt>Param</tt> object in SINGA represents a set of parameters, e.g., a weight matrix or a bias vector. <i>Basic user guide</i> describes how to configure for a <tt>Param</tt> object, and <i>Advanced user guide</i> provides details on implementing users’ parameter initialization methods.</p></div> + <h1>Parameters</h1> +<p>A <tt>Param</tt> object in SINGA represents a set of parameters, e.g., a weight matrix or a bias vector. <i>Basic user guide</i> describes how to configure for a <tt>Param</tt> object, and <i>Advanced user guide</i> provides details on implementing users’ parameter initialization methods.</p> <div class="section"> <h2><a name="Basic_user_guide"></a>Basic user guide</h2> <p>The configuration of a Param object is inside a layer configuration, as the <tt>Param</tt> are associated with layers. An example configuration is like</p> Modified: incubator/singa/site/trunk/target/site/docs/programmer-guide.html URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/target/site/docs/programmer-guide.html?rev=1700817&r1=1700816&r2=1700817&view=diff ============================================================================== --- incubator/singa/site/trunk/target/site/docs/programmer-guide.html (original) +++ incubator/singa/site/trunk/target/site/docs/programmer-guide.html Wed Sep 2 13:53:06 2015 @@ -55,32 +55,36 @@ <li> <a href="../index.html" title="Welcome">Welcome</a> </li> - - <li> <a href="../introduction.html" title="Introduction">Introduction</a> -</li> - - <li> <a href="../quick-start.html" title="Quick Start">Quick Start</a> -</li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a> <ul class="dropdown-menu"> + <li> <a href="../docs/overview.html" title="Introduction">Introduction</a> +</li> + <li> <a href="../docs/installation.html" title="Installation">Installation</a> </li> + <li> <a href="../docs/quick-start.html" title="Quick Start">Quick Start</a> +</li> + <li class="dropdown-submenu"> <a href="../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a> <ul class="dropdown-menu"> <li> <a href="../docs/model-config.html" title="Model Configuration">Model Configuration</a> </li> - <li> <a href="../docs/neuralnet.html" title="Neural Network">Neural Network</a> + <li> <a href="../docs/neural-net.html" title="Neural Network">Neural Network</a> </li> <li> <a href="../docs/layer.html" title="Layer">Layer</a> </li> <li> <a href="../docs/param.html" title="Param">Param</a> </li> + <li> <a href="../docs/train-one-batch.html" title="TrainOneBatch">TrainOneBatch</a> +</li> + <li> <a href="../docs/updater.html" title="Updater">Updater</a> +</li> </ul> </li> @@ -102,8 +106,19 @@ <li> <a href="../docs/checkpoint.html" title="Checkpoint">Checkpoint</a> </li> - <li> <a href="../docs/examples.html" title="Examples">Examples</a> + <li class="dropdown-submenu"> + <a href="../docs/examples.html" title="Examples">Examples</a> + <ul class="dropdown-menu"> + <li> <a href="../docs/cnn.html" title="CNN">CNN</a> +</li> + <li> <a href="../docs/mlp.html" title="MLP">MLP</a> </li> + <li> <a href="../docs/rbm.html" title="RBM + Auto-encoder">RBM + Auto-encoder</a> +</li> + <li> <a href="../docs/rnn.html" title="RNN">RNN</a> +</li> + </ul> + </li> <li> <a href="../docs/debug.html" title="Debug">Debug</a> </li> @@ -212,29 +227,29 @@ <span class="none"></span> Welcome</a> </li> - + <li class="nav-header">Documentaion</li> + <li> - <a href="../introduction.html" title="Introduction"> + <a href="../docs/overview.html" title="Introduction"> <span class="none"></span> Introduction</a> </li> <li> - <a href="../quick-start.html" title="Quick Start"> + <a href="../docs/installation.html" title="Installation"> <span class="none"></span> - Quick Start</a> + Installation</a> </li> - <li class="nav-header">Documentaion</li> - + <li> - <a href="../docs/installation.html" title="Installation"> + <a href="../docs/quick-start.html" title="Quick Start"> <span class="none"></span> - Installation</a> + Quick Start</a> </li> - + <li class="active"> <a href="#"><span class="icon-chevron-down"></span>Programmer Guide</a> @@ -249,7 +264,7 @@ <li> - <a href="../docs/neuralnet.html" title="Neural Network"> + <a href="../docs/neural-net.html" title="Neural Network"> <span class="none"></span> Neural Network</a> </li> @@ -267,6 +282,20 @@ <span class="none"></span> Param</a> </li> + + <li> + + <a href="../docs/train-one-batch.html" title="TrainOneBatch"> + <span class="none"></span> + TrainOneBatch</a> + </li> + + <li> + + <a href="../docs/updater.html" title="Updater"> + <span class="none"></span> + Updater</a> + </li> </ul> </li> @@ -313,13 +342,43 @@ <span class="none"></span> Checkpoint</a> </li> - + <li> <a href="../docs/examples.html" title="Examples"> - <span class="none"></span> + <span class="icon-chevron-down"></span> Examples</a> + <ul class="nav nav-list"> + + <li> + + <a href="../docs/cnn.html" title="CNN"> + <span class="none"></span> + CNN</a> + </li> + + <li> + + <a href="../docs/mlp.html" title="MLP"> + <span class="none"></span> + MLP</a> + </li> + + <li> + + <a href="../docs/rbm.html" title="RBM + Auto-encoder"> + <span class="none"></span> + RBM + Auto-encoder</a> </li> + + <li> + + <a href="../docs/rnn.html" title="RNN"> + <span class="none"></span> + RNN</a> + </li> + </ul> + </li> <li> @@ -423,11 +482,11 @@ <div id="bodyColumn" class="span10" > - <div class="section"> -<h2><a name="Programmer_Guide"></a>Programmer Guide</h2> + <h1>Programmer Guide</h1> <hr /> <p>This guide provides instructions of implementing a new model and submitting the training job. The programming model is made almost transparent to the underlying distributed environment. Hence users do not need to worry much about the communication and synchronization of nodes, which is discussed in <a href="architecture.html">architecture</a> in details.</p> <div class="section"> +<div class="section"> <h3><a name="Deep_learning_training"></a>Deep learning training</h3> <p>Deep learning is labeled as a feature learning technique, which usually consists of multiple layers. Each layer is associated with a feature transformation function. After going through all layers, the raw input feature (e.g., pixels of images) would be converted into a high-level feature that is easier for tasks like classification.</p> <p>Training a deep learning model is to find the optimal parameters involved in the transformation functions that generates good features for specific tasks. The goodness of a set of parameters is measured by a loss function, e.g., <a class="externalLink" href="https://en.wikipedia.org/wiki/Cross_entropy";>Cross-Entropy Loss</a>. Since the loss functions are usually non-linear and non-convex, it is difficult to get a closed form solution. Normally, people uses the SGD algorithm which randomly initializes the parameters and then iteratively update them to reduce the loss.</p></div> Modified: incubator/singa/site/trunk/target/site/docs/programming-guide.html URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/target/site/docs/programming-guide.html?rev=1700817&r1=1700816&r2=1700817&view=diff ============================================================================== --- incubator/singa/site/trunk/target/site/docs/programming-guide.html (original) +++ incubator/singa/site/trunk/target/site/docs/programming-guide.html Wed Sep 2 13:53:06 2015 @@ -9,7 +9,7 @@ <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="Date-Revision-yyyymmdd" content="20150902" /> <meta http-equiv="Content-Language" content="en" /> - <title>Apache SINGA – </title> + <title>Apache SINGA – Programming Guide</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" /> <link rel="stylesheet" href="../css/site.css" /> <link rel="stylesheet" href="../css/print.css" media="print" /> @@ -55,32 +55,36 @@ <li> <a href="../index.html" title="Welcome">Welcome</a> </li> - - <li> <a href="../introduction.html" title="Introduction">Introduction</a> -</li> - - <li> <a href="../quick-start.html" title="Quick Start">Quick Start</a> -</li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a> <ul class="dropdown-menu"> + <li> <a href="../docs/overview.html" title="Introduction">Introduction</a> +</li> + <li> <a href="../docs/installation.html" title="Installation">Installation</a> </li> + <li> <a href="../docs/quick-start.html" title="Quick Start">Quick Start</a> +</li> + <li class="dropdown-submenu"> <a href="../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a> <ul class="dropdown-menu"> <li> <a href="../docs/model-config.html" title="Model Configuration">Model Configuration</a> </li> - <li> <a href="../docs/neuralnet.html" title="Neural Network">Neural Network</a> + <li> <a href="../docs/neural-net.html" title="Neural Network">Neural Network</a> </li> <li> <a href="../docs/layer.html" title="Layer">Layer</a> </li> <li> <a href="../docs/param.html" title="Param">Param</a> </li> + <li> <a href="../docs/train-one-batch.html" title="TrainOneBatch">TrainOneBatch</a> +</li> + <li> <a href="../docs/updater.html" title="Updater">Updater</a> +</li> </ul> </li> @@ -102,8 +106,19 @@ <li> <a href="../docs/checkpoint.html" title="Checkpoint">Checkpoint</a> </li> - <li> <a href="../docs/examples.html" title="Examples">Examples</a> + <li class="dropdown-submenu"> + <a href="../docs/examples.html" title="Examples">Examples</a> + <ul class="dropdown-menu"> + <li> <a href="../docs/cnn.html" title="CNN">CNN</a> +</li> + <li> <a href="../docs/mlp.html" title="MLP">MLP</a> </li> + <li> <a href="../docs/rbm.html" title="RBM + Auto-encoder">RBM + Auto-encoder</a> +</li> + <li> <a href="../docs/rnn.html" title="RNN">RNN</a> +</li> + </ul> + </li> <li> <a href="../docs/debug.html" title="Debug">Debug</a> </li> @@ -189,7 +204,7 @@ Apache SINGA</a> <span class="divider">/</span> </li> - <li class="active "></li> + <li class="active ">Programming Guide</li> @@ -212,29 +227,29 @@ <span class="none"></span> Welcome</a> </li> - + <li class="nav-header">Documentaion</li> + <li> - <a href="../introduction.html" title="Introduction"> + <a href="../docs/overview.html" title="Introduction"> <span class="none"></span> Introduction</a> </li> <li> - <a href="../quick-start.html" title="Quick Start"> + <a href="../docs/installation.html" title="Installation"> <span class="none"></span> - Quick Start</a> + Installation</a> </li> - <li class="nav-header">Documentaion</li> - + <li> - <a href="../docs/installation.html" title="Installation"> + <a href="../docs/quick-start.html" title="Quick Start"> <span class="none"></span> - Installation</a> + Quick Start</a> </li> - + <li> <a href="../docs/programmer-guide.html" title="Programmer Guide"> @@ -251,7 +266,7 @@ <li> - <a href="../docs/neuralnet.html" title="Neural Network"> + <a href="../docs/neural-net.html" title="Neural Network"> <span class="none"></span> Neural Network</a> </li> @@ -269,6 +284,20 @@ <span class="none"></span> Param</a> </li> + + <li> + + <a href="../docs/train-one-batch.html" title="TrainOneBatch"> + <span class="none"></span> + TrainOneBatch</a> + </li> + + <li> + + <a href="../docs/updater.html" title="Updater"> + <span class="none"></span> + Updater</a> + </li> </ul> </li> @@ -315,13 +344,43 @@ <span class="none"></span> Checkpoint</a> </li> - + <li> <a href="../docs/examples.html" title="Examples"> - <span class="none"></span> + <span class="icon-chevron-down"></span> Examples</a> + <ul class="nav nav-list"> + + <li> + + <a href="../docs/cnn.html" title="CNN"> + <span class="none"></span> + CNN</a> </li> + + <li> + + <a href="../docs/mlp.html" title="MLP"> + <span class="none"></span> + MLP</a> + </li> + + <li> + + <a href="../docs/rbm.html" title="RBM + Auto-encoder"> + <span class="none"></span> + RBM + Auto-encoder</a> + </li> + + <li> + + <a href="../docs/rnn.html" title="RNN"> + <span class="none"></span> + RNN</a> + </li> + </ul> + </li> <li> @@ -425,10 +484,7 @@ <div id="bodyColumn" class="span10" > - <p>— layout: post title: Programming Guide category : docs</p> -<div class="section"> -<h2><a name="tags_:_programming"></a>tags : [programming]</h2> -<p>{% include JB/setup %}</p> + <h1>Programming Guide</h1> <p>To submit a training job, users must provide the configuration of the four components shown in Figure 1:</p> <ul> @@ -442,7 +498,7 @@ <li>a <a class="externalLink" href="http://singa.incubator.apache.org/docs/distributed-training";>Cluster Topology</a> specifying the distributed architecture of workers and servers.</li> </ul> <p>The <i>Basic user guide</i> section describes how to submit a training job using built-in components; while the <i>Advanced user guide</i> section presents details on writing user’s own main function to register components implemented by themselves. In addition, the training data must be prepared, which has the same <a class="externalLink" href="http://singa.incubator.apache.org/docs/data";>process</a> for both advanced users and basic users.</p> -<p><img src="http://singa.incubator.apache.org/assets/image/overview.png"; align="center" width="400px" alt="" /> <span><b>Figure 1 - SINGA overview.</b></span></p></div> +<p><img src="http://singa.incubator.apache.org/assets/image/overview.png"; align="center" width="400px" alt="" /> <span><b>Figure 1 - SINGA overview.</b></span></p> <div class="section"> <h2><a name="Basic_user_guide"></a>Basic user guide</h2> <p>Users can use the default main function provided SINGA to submit the training job. For this case, a job configuration file written as a google protocol buffer message for the <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1JobProto.html";>JobProto</a> must be provided in the command line,</p> Modified: incubator/singa/site/trunk/target/site/docs/quick-start.html URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/target/site/docs/quick-start.html?rev=1700817&r1=1700816&r2=1700817&view=diff ============================================================================== --- incubator/singa/site/trunk/target/site/docs/quick-start.html (original) +++ incubator/singa/site/trunk/target/site/docs/quick-start.html Wed Sep 2 13:53:06 2015 @@ -9,7 +9,7 @@ <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="Date-Revision-yyyymmdd" content="20150902" /> <meta http-equiv="Content-Language" content="en" /> - <title>Apache SINGA – </title> + <title>Apache SINGA – Quick Start</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" /> <link rel="stylesheet" href="../css/site.css" /> <link rel="stylesheet" href="../css/print.css" media="print" /> @@ -55,32 +55,36 @@ <li> <a href="../index.html" title="Welcome">Welcome</a> </li> - - <li> <a href="../introduction.html" title="Introduction">Introduction</a> -</li> - - <li> <a href="../quick-start.html" title="Quick Start">Quick Start</a> -</li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a> <ul class="dropdown-menu"> + <li> <a href="../docs/overview.html" title="Introduction">Introduction</a> +</li> + <li> <a href="../docs/installation.html" title="Installation">Installation</a> </li> + <li> <a href="../docs/quick-start.html" title="Quick Start">Quick Start</a> +</li> + <li class="dropdown-submenu"> <a href="../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a> <ul class="dropdown-menu"> <li> <a href="../docs/model-config.html" title="Model Configuration">Model Configuration</a> </li> - <li> <a href="../docs/neuralnet.html" title="Neural Network">Neural Network</a> + <li> <a href="../docs/neural-net.html" title="Neural Network">Neural Network</a> </li> <li> <a href="../docs/layer.html" title="Layer">Layer</a> </li> <li> <a href="../docs/param.html" title="Param">Param</a> </li> + <li> <a href="../docs/train-one-batch.html" title="TrainOneBatch">TrainOneBatch</a> +</li> + <li> <a href="../docs/updater.html" title="Updater">Updater</a> +</li> </ul> </li> @@ -102,8 +106,19 @@ <li> <a href="../docs/checkpoint.html" title="Checkpoint">Checkpoint</a> </li> - <li> <a href="../docs/examples.html" title="Examples">Examples</a> + <li class="dropdown-submenu"> + <a href="../docs/examples.html" title="Examples">Examples</a> + <ul class="dropdown-menu"> + <li> <a href="../docs/cnn.html" title="CNN">CNN</a> +</li> + <li> <a href="../docs/mlp.html" title="MLP">MLP</a> </li> + <li> <a href="../docs/rbm.html" title="RBM + Auto-encoder">RBM + Auto-encoder</a> +</li> + <li> <a href="../docs/rnn.html" title="RNN">RNN</a> +</li> + </ul> + </li> <li> <a href="../docs/debug.html" title="Debug">Debug</a> </li> @@ -189,7 +204,7 @@ Apache SINGA</a> <span class="divider">/</span> </li> - <li class="active "></li> + <li class="active ">Quick Start</li> @@ -212,29 +227,27 @@ <span class="none"></span> Welcome</a> </li> - + <li class="nav-header">Documentaion</li> + <li> - <a href="../introduction.html" title="Introduction"> + <a href="../docs/overview.html" title="Introduction"> <span class="none"></span> Introduction</a> </li> <li> - <a href="../quick-start.html" title="Quick Start"> - <span class="none"></span> - Quick Start</a> - </li> - <li class="nav-header">Documentaion</li> - - <li> - <a href="../docs/installation.html" title="Installation"> <span class="none"></span> Installation</a> </li> - + + <li class="active"> + + <a href="#"><span class="none"></span>Quick Start</a> + </li> + <li> <a href="../docs/programmer-guide.html" title="Programmer Guide"> @@ -251,7 +264,7 @@ <li> - <a href="../docs/neuralnet.html" title="Neural Network"> + <a href="../docs/neural-net.html" title="Neural Network"> <span class="none"></span> Neural Network</a> </li> @@ -269,6 +282,20 @@ <span class="none"></span> Param</a> </li> + + <li> + + <a href="../docs/train-one-batch.html" title="TrainOneBatch"> + <span class="none"></span> + TrainOneBatch</a> + </li> + + <li> + + <a href="../docs/updater.html" title="Updater"> + <span class="none"></span> + Updater</a> + </li> </ul> </li> @@ -315,13 +342,43 @@ <span class="none"></span> Checkpoint</a> </li> - + <li> <a href="../docs/examples.html" title="Examples"> - <span class="none"></span> + <span class="icon-chevron-down"></span> Examples</a> + <ul class="nav nav-list"> + + <li> + + <a href="../docs/cnn.html" title="CNN"> + <span class="none"></span> + CNN</a> </li> + + <li> + + <a href="../docs/mlp.html" title="MLP"> + <span class="none"></span> + MLP</a> + </li> + + <li> + + <a href="../docs/rbm.html" title="RBM + Auto-encoder"> + <span class="none"></span> + RBM + Auto-encoder</a> + </li> + + <li> + + <a href="../docs/rnn.html" title="RNN"> + <span class="none"></span> + RNN</a> + </li> + </ul> + </li> <li> @@ -425,10 +482,7 @@ <div id="bodyColumn" class="span10" > - <p>— layout: post title: Quick Start category : docs</p> -<div class="section"> -<h2><a name="tags_:_installation_examples"></a>tags : [installation, examples]</h2> -<p>{% include JB/setup %}</p></div> + <h1>Quick Start</h1> <div class="section"> <h2><a name="SINGA_setup"></a>SINGA setup</h2> <p>Please refer to the <a class="externalLink" href="http://singa.incubator.apache.org/docs/installation}";>installation</a> page for guidance on installing SINGA.</p> Modified: incubator/singa/site/trunk/target/site/docs/rbm.html URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/target/site/docs/rbm.html?rev=1700817&r1=1700816&r2=1700817&view=diff ============================================================================== --- incubator/singa/site/trunk/target/site/docs/rbm.html (original) +++ incubator/singa/site/trunk/target/site/docs/rbm.html Wed Sep 2 13:53:06 2015 @@ -9,7 +9,7 @@ <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="Date-Revision-yyyymmdd" content="20150902" /> <meta http-equiv="Content-Language" content="en" /> - <title>Apache SINGA – </title> + <title>Apache SINGA – RBM Example</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" /> <link rel="stylesheet" href="../css/site.css" /> <link rel="stylesheet" href="../css/print.css" media="print" /> @@ -55,32 +55,36 @@ <li> <a href="../index.html" title="Welcome">Welcome</a> </li> - - <li> <a href="../introduction.html" title="Introduction">Introduction</a> -</li> - - <li> <a href="../quick-start.html" title="Quick Start">Quick Start</a> -</li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a> <ul class="dropdown-menu"> + <li> <a href="../docs/overview.html" title="Introduction">Introduction</a> +</li> + <li> <a href="../docs/installation.html" title="Installation">Installation</a> </li> + <li> <a href="../docs/quick-start.html" title="Quick Start">Quick Start</a> +</li> + <li class="dropdown-submenu"> <a href="../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a> <ul class="dropdown-menu"> <li> <a href="../docs/model-config.html" title="Model Configuration">Model Configuration</a> </li> - <li> <a href="../docs/neuralnet.html" title="Neural Network">Neural Network</a> + <li> <a href="../docs/neural-net.html" title="Neural Network">Neural Network</a> </li> <li> <a href="../docs/layer.html" title="Layer">Layer</a> </li> <li> <a href="../docs/param.html" title="Param">Param</a> </li> + <li> <a href="../docs/train-one-batch.html" title="TrainOneBatch">TrainOneBatch</a> +</li> + <li> <a href="../docs/updater.html" title="Updater">Updater</a> +</li> </ul> </li> @@ -102,8 +106,19 @@ <li> <a href="../docs/checkpoint.html" title="Checkpoint">Checkpoint</a> </li> - <li> <a href="../docs/examples.html" title="Examples">Examples</a> + <li class="dropdown-submenu"> + <a href="../docs/examples.html" title="Examples">Examples</a> + <ul class="dropdown-menu"> + <li> <a href="../docs/cnn.html" title="CNN">CNN</a> +</li> + <li> <a href="../docs/mlp.html" title="MLP">MLP</a> </li> + <li> <a href="../docs/rbm.html" title="RBM + Auto-encoder">RBM + Auto-encoder</a> +</li> + <li> <a href="../docs/rnn.html" title="RNN">RNN</a> +</li> + </ul> + </li> <li> <a href="../docs/debug.html" title="Debug">Debug</a> </li> @@ -189,7 +204,7 @@ Apache SINGA</a> <span class="divider">/</span> </li> - <li class="active "></li> + <li class="active ">RBM Example</li> @@ -212,29 +227,29 @@ <span class="none"></span> Welcome</a> </li> - + <li class="nav-header">Documentaion</li> + <li> - <a href="../introduction.html" title="Introduction"> + <a href="../docs/overview.html" title="Introduction"> <span class="none"></span> Introduction</a> </li> <li> - <a href="../quick-start.html" title="Quick Start"> + <a href="../docs/installation.html" title="Installation"> <span class="none"></span> - Quick Start</a> + Installation</a> </li> - <li class="nav-header">Documentaion</li> - + <li> - <a href="../docs/installation.html" title="Installation"> + <a href="../docs/quick-start.html" title="Quick Start"> <span class="none"></span> - Installation</a> + Quick Start</a> </li> - + <li> <a href="../docs/programmer-guide.html" title="Programmer Guide"> @@ -251,7 +266,7 @@ <li> - <a href="../docs/neuralnet.html" title="Neural Network"> + <a href="../docs/neural-net.html" title="Neural Network"> <span class="none"></span> Neural Network</a> </li> @@ -269,6 +284,20 @@ <span class="none"></span> Param</a> </li> + + <li> + + <a href="../docs/train-one-batch.html" title="TrainOneBatch"> + <span class="none"></span> + TrainOneBatch</a> + </li> + + <li> + + <a href="../docs/updater.html" title="Updater"> + <span class="none"></span> + Updater</a> + </li> </ul> </li> @@ -315,13 +344,41 @@ <span class="none"></span> Checkpoint</a> </li> - + <li> <a href="../docs/examples.html" title="Examples"> - <span class="none"></span> + <span class="icon-chevron-down"></span> Examples</a> + <ul class="nav nav-list"> + + <li> + + <a href="../docs/cnn.html" title="CNN"> + <span class="none"></span> + CNN</a> + </li> + + <li> + + <a href="../docs/mlp.html" title="MLP"> + <span class="none"></span> + MLP</a> + </li> + + <li class="active"> + + <a href="#"><span class="none"></span>RBM + Auto-encoder</a> + </li> + + <li> + + <a href="../docs/rnn.html" title="RNN"> + <span class="none"></span> + RNN</a> </li> + </ul> + </li> <li> @@ -425,14 +482,8 @@ <div id="bodyColumn" class="span10" > - <p>— layout: post title: Example-Restricted Boltzmann Machine category : docs</p> -<div class="section"> -<h2><a name="tags_:_rbm_example"></a>tags : [rbm, example]</h2> - -<div class="source"> -<div class="source"><pre class="prettyprint">{% include JB/setup %} -</pre></div></div> -<p>This example uses SINGA to train 4 RBM models and one auto-encoder model over the <a class="externalLink" href="http://yann.lecun.com/exdb/mnist/";>MNIST dataset</a>. The auto-encoder model is trained to reduce the dimensionality of the MNIST image feature. The RBM models are trained to initialize parameters of the auto-encoder model. This example application is from <a class="externalLink" href="http://www.cs.toronto.edu/~hinton/science.pdf";>Hinton’s science paper</a>.</p></div> + <h1>RBM Example</h1> +<p>This example uses SINGA to train 4 RBM models and one auto-encoder model over the <a class="externalLink" href="http://yann.lecun.com/exdb/mnist/";>MNIST dataset</a>. The auto-encoder model is trained to reduce the dimensionality of the MNIST image feature. The RBM models are trained to initialize parameters of the auto-encoder model. This example application is from <a class="externalLink" href="http://www.cs.toronto.edu/~hinton/science.pdf";>Hinton’s science paper</a>.</p> <div class="section"> <h2><a name="Running_instructions"></a>Running instructions</h2> <p>Running scripts are provided in <i>SINGA_ROOT/examples/rbm</i> folder.</p> @@ -462,7 +513,7 @@ $ ./bin/singa-run.sh -conf examples/rbm/ <h2><a name="Training_details"></a>Training details</h2> <div class="section"> <h3><a name="RBM0"></a>RBM0</h3> -<p><img src="http://singa.incubator.apache.org/assets/image/RBM0_new.PNG"; align="center" width="200px" alt="" /> <span><b>Figure 1 - RBM0.</b></span></p> +<p><img src="http://singa.incubator.apache.org/images/RBM0_new.PNG"; align="center" width="200px" alt="" /> <span><b>Figure 1 - RBM0.</b></span></p> <p>The neural net structure for training RBM0 is shown in Figure 1. The data layer and parser layer provides features for training RBM0. The visible layer (connected with parser layer) of RBM0 accepts the image feature (784 dimension). The hidden layer is set to have 1000 neurons (units). These two layers are configured as,</p> <div class="source"> @@ -536,7 +587,7 @@ updater{ <p>Then SINGA will <a class="externalLink" href="http://singa.incubator.apache.org/docs/checkpoint";>checkpoint the parameters</a> into <i>SINGA_ROOT/rbm0/</i>.</p></div> <div class="section"> <h3><a name="RBM1"></a>RBM1</h3> -<p><img src="http://singa.incubator.apache.org/assets/image/RBM1_new.PNG"; align="center" width="200px" alt="" /> <span><b>Figure 2 - RBM1.</b></span></p> +<p><img src="http://singa.incubator.apache.org/images/RBM1_new.PNG"; align="center" width="200px" alt="" /> <span><b>Figure 2 - RBM1.</b></span></p> <p>Figure 2 shows the net structure of training RBM1. The visible units of RBM1 accept the output from the Sigmoid1 layer. The Inner1 layer is a <tt>InnerProductLayer</tt> whose parameters are set to the <tt>w0</tt> and <tt>b1</tt> learned from RBM0. The neural net configuration is (with layers for data layer and parser layer omitted).</p> <div class="source"> @@ -617,11 +668,11 @@ cluster{ <p>The workspace is changed for checkpointing w1, b2 and b3 into <i>SINGA_ROOT/rbm1/</i>.</p></div> <div class="section"> <h3><a name="RBM2"></a>RBM2</h3> -<p><img src="http://singa.incubator.apache.org/assets/image/RBM2_new.PNG"; align="center" width="200px" alt="" /> <span><b>Figure 3 - RBM2.</b></span></p> +<p><img src="http://singa.incubator.apache.org/images/RBM2_new.PNG"; align="center" width="200px" alt="" /> <span><b>Figure 3 - RBM2.</b></span></p> <p>Figure 3 shows the net structure of training RBM2. In this model, a layer with 250 units is added as the hidden layer of RBM2. The visible units of RBM2 accepts output from Sigmoid2 layer. Parameters of Inner1 and inner2 are set to <tt>w0,b1,w1,b2</tt> which can be load from the checkpoint file of RBM1, i.e., “SINGA_ROOT/rbm1/”.</p></div> <div class="section"> <h3><a name="RBM3"></a>RBM3</h3> -<p><img src="http://singa.incubator.apache.org/assets/image/RBM3_new.PNG"; align="center" width="200px" alt="" /> <span><b>Figure 4 - RBM3.</b></span></p> +<p><img src="http://singa.incubator.apache.org/images/RBM3_new.PNG"; align="center" width="200px" alt="" /> <span><b>Figure 4 - RBM3.</b></span></p> <p>Figure 4 shows the net structure of training RBM3. It is similar to Figure 3, but according to <a class="externalLink" href="http://www.cs.toronto.edu/~hinton/science.pdf";>Hinton’s science paper</a>, the hidden units of the top RBM (RBM3) have stochastic real-valued states drawn from a unit variance Gaussian whose mean is determined by the input from the RBM’s logistic visible units. So we add a <tt>gaussian</tt> field in the RBMHid layer to control the sampling distribution (Gaussian or Bernoulli). In addition, this RBM has a much smaller learning rate (0.001). The neural net configuration for the RBM3 and the updating protocol is (with layers for data layer and parser layer omitted),</p> <div class="source"> Modified: incubator/singa/site/trunk/target/site/docs/rnn.html URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/target/site/docs/rnn.html?rev=1700817&r1=1700816&r2=1700817&view=diff ============================================================================== --- incubator/singa/site/trunk/target/site/docs/rnn.html (original) +++ incubator/singa/site/trunk/target/site/docs/rnn.html Wed Sep 2 13:53:06 2015 @@ -9,7 +9,7 @@ <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="Date-Revision-yyyymmdd" content="20150902" /> <meta http-equiv="Content-Language" content="en" /> - <title>Apache SINGA – </title> + <title>Apache SINGA – RNN Example</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" /> <link rel="stylesheet" href="../css/site.css" /> <link rel="stylesheet" href="../css/print.css" media="print" /> @@ -55,32 +55,36 @@ <li> <a href="../index.html" title="Welcome">Welcome</a> </li> - - <li> <a href="../introduction.html" title="Introduction">Introduction</a> -</li> - - <li> <a href="../quick-start.html" title="Quick Start">Quick Start</a> -</li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a> <ul class="dropdown-menu"> + <li> <a href="../docs/overview.html" title="Introduction">Introduction</a> +</li> + <li> <a href="../docs/installation.html" title="Installation">Installation</a> </li> + <li> <a href="../docs/quick-start.html" title="Quick Start">Quick Start</a> +</li> + <li class="dropdown-submenu"> <a href="../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a> <ul class="dropdown-menu"> <li> <a href="../docs/model-config.html" title="Model Configuration">Model Configuration</a> </li> - <li> <a href="../docs/neuralnet.html" title="Neural Network">Neural Network</a> + <li> <a href="../docs/neural-net.html" title="Neural Network">Neural Network</a> </li> <li> <a href="../docs/layer.html" title="Layer">Layer</a> </li> <li> <a href="../docs/param.html" title="Param">Param</a> </li> + <li> <a href="../docs/train-one-batch.html" title="TrainOneBatch">TrainOneBatch</a> +</li> + <li> <a href="../docs/updater.html" title="Updater">Updater</a> +</li> </ul> </li> @@ -102,8 +106,19 @@ <li> <a href="../docs/checkpoint.html" title="Checkpoint">Checkpoint</a> </li> - <li> <a href="../docs/examples.html" title="Examples">Examples</a> + <li class="dropdown-submenu"> + <a href="../docs/examples.html" title="Examples">Examples</a> + <ul class="dropdown-menu"> + <li> <a href="../docs/cnn.html" title="CNN">CNN</a> </li> + <li> <a href="../docs/mlp.html" title="MLP">MLP</a> +</li> + <li> <a href="../docs/rbm.html" title="RBM + Auto-encoder">RBM + Auto-encoder</a> +</li> + <li> <a href="../docs/rnn.html" title="RNN">RNN</a> +</li> + </ul> + </li> <li> <a href="../docs/debug.html" title="Debug">Debug</a> </li> @@ -189,7 +204,7 @@ Apache SINGA</a> <span class="divider">/</span> </li> - <li class="active "></li> + <li class="active ">RNN Example</li> @@ -212,29 +227,29 @@ <span class="none"></span> Welcome</a> </li> - + <li class="nav-header">Documentaion</li> + <li> - <a href="../introduction.html" title="Introduction"> + <a href="../docs/overview.html" title="Introduction"> <span class="none"></span> Introduction</a> </li> <li> - <a href="../quick-start.html" title="Quick Start"> + <a href="../docs/installation.html" title="Installation"> <span class="none"></span> - Quick Start</a> + Installation</a> </li> - <li class="nav-header">Documentaion</li> - + <li> - <a href="../docs/installation.html" title="Installation"> + <a href="../docs/quick-start.html" title="Quick Start"> <span class="none"></span> - Installation</a> + Quick Start</a> </li> - + <li> <a href="../docs/programmer-guide.html" title="Programmer Guide"> @@ -251,7 +266,7 @@ <li> - <a href="../docs/neuralnet.html" title="Neural Network"> + <a href="../docs/neural-net.html" title="Neural Network"> <span class="none"></span> Neural Network</a> </li> @@ -269,6 +284,20 @@ <span class="none"></span> Param</a> </li> + + <li> + + <a href="../docs/train-one-batch.html" title="TrainOneBatch"> + <span class="none"></span> + TrainOneBatch</a> + </li> + + <li> + + <a href="../docs/updater.html" title="Updater"> + <span class="none"></span> + Updater</a> + </li> </ul> </li> @@ -315,13 +344,41 @@ <span class="none"></span> Checkpoint</a> </li> - + <li> <a href="../docs/examples.html" title="Examples"> - <span class="none"></span> + <span class="icon-chevron-down"></span> Examples</a> + <ul class="nav nav-list"> + + <li> + + <a href="../docs/cnn.html" title="CNN"> + <span class="none"></span> + CNN</a> + </li> + + <li> + + <a href="../docs/mlp.html" title="MLP"> + <span class="none"></span> + MLP</a> </li> + + <li> + + <a href="../docs/rbm.html" title="RBM + Auto-encoder"> + <span class="none"></span> + RBM + Auto-encoder</a> + </li> + + <li class="active"> + + <a href="#"><span class="none"></span>RNN</a> + </li> + </ul> + </li> <li> @@ -425,7 +482,7 @@ <div id="bodyColumn" class="span10" > - <p>— layout: post title: Example — Recurrent Neural Network category : docs tags : [rnn, example]<a class="externalLink" href="http://singa.incubator.apache.org";>http://singa.incubator.apache.org</a> {% include JB/setup %}</p> + <h1>RNN Example</h1> <p>Recurrent Neural Networks (RNN) are widely used for modeling sequential data, such as music, videos and sentences. In this example, we use SINGA to train a <a class="externalLink" href="http://www.fit.vutbr.cz/research/groups/speech/publi/2010/mikolov_interspeech2010_IS100722.pdf";>RNN model</a> proposed by Tomas Mikolov for <a class="externalLink" href="https://en.wikipedia.org/wiki/Language_model";>language modeling</a>. The training objective (loss) is minimize the <a class="externalLink" href="https://en.wikipedia.org/wiki/Perplexity";>perplexity per word</a>, which is equivalent to maximize the probability of predicting the next word given the current word in a sentence.</p> <p>Different to the <a class="externalLink" href="http://singa.incubator.apache.org/docs/cnn";>CNN</a>, <a class="externalLink" href="http://singa.incubator.apache.org/docs/mlp";>MLP</a> and <a class="externalLink" href="http://singa.incubator.apache.org/docs/rbm";>RBM</a> examples which use built-in <a class="externalLink" href="http://singa.incubator.apache.org/docs/layer";>Layer</a>s and <a class="externalLink" href="http://singa.incubator.apache.org/docs/data";>Record</a>s, none of the layers in this model is built-in. Hence users can get examples of implementing their own Layers and data Records in this page.</p> <div class="section"> @@ -445,7 +502,7 @@ $ ./bin/singa-run.sh -conf SINGA_ROOT/ex </pre></div></div></div> <div class="section"> <h2><a name="Implementations"></a>Implementations</h2> -<p><img src="http://singa.incubator.apache.org/assets/image/rnn-refine.png"; align="center" width="300px" alt="" /> <span><b>Figure 1 - Net structure of the RNN model.</b></span></p> +<p><img src="http://singa.incubator.apache.org/images/rnn-refine.png"; align="center" width="300px" alt="" /> <span><b>Figure 1 - Net structure of the RNN model.</b></span></p> <p>The neural net structure is shown Figure 1. Word records are loaded by <tt>RnnlmDataLayer</tt> from <tt>WordShard</tt>. <tt>RnnlmWordparserLayer</tt> parses word records to get word indexes (in the vocabulary). For every iteration, <tt>window_size</tt> words are processed. <tt>RnnlmWordinputLayer</tt> looks up a word embedding matrix to extract feature vectors for words in the window. These features are transformed by <tt>RnnlmInnerproductLayer</tt> layer and <tt>RnnlmSigmoidLayer</tt>. <tt>RnnlmSigmoidLayer</tt> is a recurrent layer that forwards features from previous words to next words. Finally, <tt>RnnlmComputationLayer</tt> computes the perplexity loss with word class information from <tt>RnnlmClassparserLayer</tt>. The word class is a cluster ID. Words are clustered based on their frequency in the dataset, e.g., frequent words are clustered together and less frequent words are clustered together. Clustering is to improve the efficiency of the final prediction process.</p> <div class="section"> <h3><a name="Data_preparation"></a>Data preparation</h3>
![http://singa.incubator.apache.org/assets/image/overview.png"](http://singa.incubator.apache.org/assets/image/overview.png"){kind=link}
![http://singa.incubator.apache.org/assets/image/RBM0_new.PNG"](http://singa.incubator.apache.org/assets/image/RBM0_new.PNG"){kind=link}
![http://singa.incubator.apache.org/images/RBM0_new.PNG"](http://singa.incubator.apache.org/images/RBM0_new.PNG"){kind=link}
![http://singa.incubator.apache.org/assets/image/RBM1_new.PNG"](http://singa.incubator.apache.org/assets/image/RBM1_new.PNG"){kind=link}
![http://singa.incubator.apache.org/images/RBM1_new.PNG"](http://singa.incubator.apache.org/images/RBM1_new.PNG"){kind=link}
![http://singa.incubator.apache.org/assets/image/RBM2_new.PNG"](http://singa.incubator.apache.org/assets/image/RBM2_new.PNG"){kind=link}
![http://singa.incubator.apache.org/images/RBM2_new.PNG"](http://singa.incubator.apache.org/images/RBM2_new.PNG"){kind=link}
![http://singa.incubator.apache.org/assets/image/RBM3_new.PNG"](http://singa.incubator.apache.org/assets/image/RBM3_new.PNG"){kind=link}
![http://singa.incubator.apache.org/images/RBM3_new.PNG"](http://singa.incubator.apache.org/images/RBM3_new.PNG"){kind=link}
![http://singa.incubator.apache.org/assets/image/rnn-refine.png"](http://singa.incubator.apache.org/assets/image/rnn-refine.png"){kind=link}
![http://singa.incubator.apache.org/images/rnn-refine.png"](http://singa.incubator.apache.org/images/rnn-refine.png"){kind=link}