Author: buildbot
Date: Tue May 12 13:05:25 2015
New Revision: 951037
Log:
Staging update by buildbot for singa
Added:
websites/staging/singa/trunk/content/docs/code-structure.html
websites/staging/singa/trunk/content/docs/neuralnet-partition.html
websites/staging/singa/trunk/content/docs/programming-model.html
websites/staging/singa/trunk/content/introduction.html
Modified:
websites/staging/singa/trunk/content/ (props changed)
websites/staging/singa/trunk/content/community.html
websites/staging/singa/trunk/content/docs.html
websites/staging/singa/trunk/content/docs/architecture.html
websites/staging/singa/trunk/content/docs/communication.html
websites/staging/singa/trunk/content/docs/installation.html
websites/staging/singa/trunk/content/index.html
websites/staging/singa/trunk/content/quick-start.html
Propchange: websites/staging/singa/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Tue May 12 13:05:25 2015
@@ -1 +1 @@
-1678870
+1678931
Modified: websites/staging/singa/trunk/content/community.html
==============================================================================
--- websites/staging/singa/trunk/content/community.html (original)
+++ websites/staging/singa/trunk/content/community.html Tue May 12 13:05:25 2015
@@ -53,7 +53,10 @@
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
<ul class="dropdown-menu">
- <li> <a href="index.html"
title="Introduction">Introduction</a>
+ <li> <a href="index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="introduction.html"
title="Introduction">Introduction</a>
</li>
<li> <a href="quick-start.html" title="Quick
Start">Quick Start</a>
@@ -72,6 +75,15 @@
<li> <a href="docs/communication.html"
title="Communication">Communication</a>
</li>
+
+ <li> <a href="docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
</ul>
</li>
<li class="dropdown">
@@ -135,7 +147,14 @@
<li>
- <a href="index.html" title="Introduction">
+ <a href="index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="introduction.html" title="Introduction">
<span class="none"></span>
Introduction</a>
</li>
@@ -168,6 +187,27 @@
<span class="none"></span>
Communication</a>
</li>
+
+ <li>
+
+ <a href="docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
<li class="nav-header">External Links</li>
<li>
@@ -188,7 +228,7 @@
<div class="clear"></div>
<div class="clear"></div>
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
- <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; width="150" />
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
</a>
</div>
</div>
@@ -197,8 +237,9 @@
<div id="bodyColumn" class="span10" >
- <h1>Community</h1>
-<hr />
+ <div class="section">
+<h2><a name="Community"></a>Community</h2>
+<hr /></div>
</div>
</div>
</div>
Modified: websites/staging/singa/trunk/content/docs.html
==============================================================================
--- websites/staging/singa/trunk/content/docs.html (original)
+++ websites/staging/singa/trunk/content/docs.html Tue May 12 13:05:25 2015
@@ -53,7 +53,10 @@
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
<ul class="dropdown-menu">
- <li> <a href="index.html"
title="Introduction">Introduction</a>
+ <li> <a href="index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="introduction.html"
title="Introduction">Introduction</a>
</li>
<li> <a href="quick-start.html" title="Quick
Start">Quick Start</a>
@@ -72,6 +75,15 @@
<li> <a href="docs/communication.html"
title="Communication">Communication</a>
</li>
+
+ <li> <a href="docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
</ul>
</li>
<li class="dropdown">
@@ -135,7 +147,14 @@
<li>
- <a href="index.html" title="Introduction">
+ <a href="index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="introduction.html" title="Introduction">
<span class="none"></span>
Introduction</a>
</li>
@@ -168,6 +187,27 @@
<span class="none"></span>
Communication</a>
</li>
+
+ <li>
+
+ <a href="docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
<li class="nav-header">External Links</li>
<li>
@@ -188,7 +228,7 @@
<div class="clear"></div>
<div class="clear"></div>
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
- <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; width="150" />
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
</a>
</div>
</div>
@@ -208,6 +248,12 @@
<li><a href="docs/architecture.html">System Architecture</a></li>
<li><a href="docs/communication.html">Communication</a></li>
+
+<li><a href="docs/code-structure.html">Code Structure</a></li>
+
+<li><a href="docs/neuralnet-partition.html">Neural Network Partition</a></li>
+
+<li><a href="docs/programming-model.html">Programming Model</a></li>
</ul></div>
</div>
</div>
Modified: websites/staging/singa/trunk/content/docs/architecture.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/architecture.html (original)
+++ websites/staging/singa/trunk/content/docs/architecture.html Tue May 12
13:05:25 2015
@@ -53,7 +53,10 @@
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
<ul class="dropdown-menu">
- <li> <a href="../index.html"
title="Introduction">Introduction</a>
+ <li> <a href="../index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="../introduction.html"
title="Introduction">Introduction</a>
</li>
<li> <a href="../quick-start.html" title="Quick
Start">Quick Start</a>
@@ -72,6 +75,15 @@
<li> <a href="../docs/communication.html"
title="Communication">Communication</a>
</li>
+
+ <li> <a href="../docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="../docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
</ul>
</li>
<li class="dropdown">
@@ -135,7 +147,14 @@
<li>
- <a href="../index.html" title="Introduction">
+ <a href="../index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="../introduction.html" title="Introduction">
<span class="none"></span>
Introduction</a>
</li>
@@ -166,6 +185,27 @@
<span class="none"></span>
Communication</a>
</li>
+
+ <li>
+
+ <a href="../docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
<li class="nav-header">External Links</li>
<li>
@@ -186,7 +226,7 @@
<div class="clear"></div>
<div class="clear"></div>
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
- <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; width="150" />
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
</a>
</div>
</div>
Added: websites/staging/singa/trunk/content/docs/code-structure.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/code-structure.html (added)
+++ websites/staging/singa/trunk/content/docs/code-structure.html Tue May 12
13:05:25 2015
@@ -0,0 +1,330 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2015-05-12
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20150512" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – Code Structure</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript"
src="../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa";>
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index:
10000;"
+
src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png";
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn
btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="../introduction.html"
title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../quick-start.html" title="Quick
Start">Quick Start</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle"
data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../docs/installation.html"
title="Installation">Installation</a>
+</li>
+
+ <li> <a href="../docs/architecture.html"
title="System Architecture">System Architecture</a>
+</li>
+
+ <li> <a href="../docs/communication.html"
title="Communication">Communication</a>
+</li>
+
+ <li> <a href="../docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="../docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External
Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/";
title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../index.html"
id="bannerLeft" title="Apache SINGA">
+
<img src="../images/singa-logo.png" alt="Apache SINGA"
width="150"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+
<img src="../images/singa-title.png" alt="Apache SINGA"
width="600"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Code Structure</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="../introduction.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../docs/installation.html"
title="Installation">
+ <span class="none"></span>
+ Installation</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/architecture.html" title="System
Architecture">
+ <span class="none"></span>
+ System Architecture</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/communication.html"
title="Communication">
+ <span class="none"></span>
+ Communication</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><span class="none"></span>Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/";
class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <div class="section">
+<h2><a name="Code_Structure"></a>Code Structure</h2>
+<hr />
+<div class="section">
+<h3><a name="Worker_Side"></a>Worker Side</h3>
+<div class="section">
+<h4><a name="Main_Classes"></a>Main Classes</h4>
+<p><img src="../images/code-structure/main.jpg" style="width:70%;"
align="center" alt="" /></p>
+
+<ul>
+
+<li><b>Worker</b>: start the solver to conduct training or resume from
previous training snapshots.</li>
+
+<li><b>Solver</b>: construct the neural network and run training algorithms
over it. Validation and testing is also done by the solver along the
training.</li>
+
+<li><b>TableDelegate</b>: delegate for the parameter table physically stored
in parameter servers. it runs a thread to communicate with table servers for
parameter transferring.</li>
+
+<li><b>Net</b>: the neural network consists of multiple layers constructed
from input configuration file.</li>
+
+<li><b>Layer</b>: the core abstraction, read data (neurons) from connecting
layers, and compute the data of itself according to layer specific
ComputeFeature functions. Data from the bottom layer is forwarded layer by
layer to the top.</li>
+</ul></div>
+<div class="section">
+<h4><a name="Data_types"></a>Data types</h4>
+<p><img src="../images/code-structure/layer.jpg" style="width:90%;"
align="center" alt="" /></p>
+
+<ul>
+
+<li><b>ComputeFeature</b>: read data (neurons) from in-coming layers, and
compute the data of itself according to layer type. This function can be
overrided to implement different types layers.</li>
+
+<li><b>ComputeGradient</b>: read gradients (and data) from in-coming layers
and compute gradients of parameters and data w.r.t the learning objective
(loss).</li>
+</ul>
+<p>We adpat the implementation for <b>PoolingLayer</b>, <b>Im2colLayer</b> and
<b>LRNLayer</b> from <a class="externalLink"
href="http://caffe.berkeleyvision.org/";>Caffe</a>.</p>
+<p><img src="../images/code-structure/darray.jpg" style="width:55%;"
align="center" alt="" /></p>
+
+<ul>
+
+<li><b>DArray</b>: provide the abstraction of distributed array on multiple
nodes, supporting array/matrix operations and element-wise operations. Users
can use it as a local structure.</li>
+
+<li><b>LArray</b>: the local part for the DArray. Each LArray is treated as an
independent array, and support all array-related operations.</li>
+
+<li><b>MemSpace</b>: manage the memory used by DArray. Distributed memory are
allocated and managed by armci. Multiple DArray can share a same MemSpace, the
memory will be released when no DArray uses it anymore.</li>
+
+<li><b>Partition</b>: maintain both global shape and local partition
information. used when two DArray are going to interact.</li>
+
+<li><b>Shape</b>: basic class for representing the scope of a
DArray/LArray</li>
+
+<li><b>Range</b>: basic class for representing the scope of a Partition</li>
+</ul></div></div>
+<div class="section">
+<h3><a name="Parameter_Server"></a>Parameter Server</h3>
+<div class="section">
+<h4><a name="Main_classes"></a>Main classes</h4>
+<p><img src="../images/code-structure/uml.jpg" style="width:90%;"
align="center" alt="" /></p>
+
+<ul>
+
+<li><b>NetworkService</b>: provide access to the network (sending and
receiving messages). It maintains a queue for received messages, implemented by
NetworkQueue.</li>
+
+<li><b>RequestDispatcher</b>: pick up next message (request) from the queue,
and invoked a method (callback) to process them.</li>
+
+<li><b>TableServer</b>: provide access to the data table (parameters).
Register callbacks for different types of requests to RequestDispatcher.</li>
+
+<li><b>GlobalTable</b>: implement the table. Data is partitioned into multiple
Shard objects per table. User-defined consistency model supported by extending
TableServerHandler for each table.</li>
+</ul></div>
+<div class="section">
+<h4><a name="Data_types"></a>Data types</h4>
+<p><img src="../images/code-structure/type.jpg" style="width:400px;"
align="middle" alt="" /></p>
+<p>Table related messages are either of type <b>RequestBase</b> which contains
different types of request, or of type <b>TableData</b> containing a key-value
tuple.</p></div>
+<div class="section">
+<h4><a name="Control_flow_and_thread_model"></a>Control flow and thread
model</h4>
+<p><img src="../images/code-structure/threads.jpg" alt="uml" /></p>
+<p>The figure above shows how a GET request sent from a worker is processed by
the table server. The control flow for other types of requests is similar. At
the server side, there are at least 3 threads running at any time: two by
NetworkService for sending and receiving message, and at least one by the
RequestDispatcher for dispatching requests.</p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright ©
2015
+ <a href="http://www.apache.org";>The Apache Software
Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Modified: websites/staging/singa/trunk/content/docs/communication.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/communication.html (original)
+++ websites/staging/singa/trunk/content/docs/communication.html Tue May 12
13:05:25 2015
@@ -53,7 +53,10 @@
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
<ul class="dropdown-menu">
- <li> <a href="../index.html"
title="Introduction">Introduction</a>
+ <li> <a href="../index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="../introduction.html"
title="Introduction">Introduction</a>
</li>
<li> <a href="../quick-start.html" title="Quick
Start">Quick Start</a>
@@ -72,6 +75,15 @@
<li> <a href="../docs/communication.html"
title="Communication">Communication</a>
</li>
+
+ <li> <a href="../docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="../docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
</ul>
</li>
<li class="dropdown">
@@ -135,7 +147,14 @@
<li>
- <a href="../index.html" title="Introduction">
+ <a href="../index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="../introduction.html" title="Introduction">
<span class="none"></span>
Introduction</a>
</li>
@@ -166,6 +185,27 @@
<a href="#"><span class="none"></span>Communication</a>
</li>
+
+ <li>
+
+ <a href="../docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
<li class="nav-header">External Links</li>
<li>
@@ -186,7 +226,7 @@
<div class="clear"></div>
<div class="clear"></div>
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
- <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; width="150" />
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
</a>
</div>
</div>
Modified: websites/staging/singa/trunk/content/docs/installation.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/installation.html (original)
+++ websites/staging/singa/trunk/content/docs/installation.html Tue May 12
13:05:25 2015
@@ -53,7 +53,10 @@
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
<ul class="dropdown-menu">
- <li> <a href="../index.html"
title="Introduction">Introduction</a>
+ <li> <a href="../index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="../introduction.html"
title="Introduction">Introduction</a>
</li>
<li> <a href="../quick-start.html" title="Quick
Start">Quick Start</a>
@@ -72,6 +75,15 @@
<li> <a href="../docs/communication.html"
title="Communication">Communication</a>
</li>
+
+ <li> <a href="../docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="../docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
</ul>
</li>
<li class="dropdown">
@@ -135,7 +147,14 @@
<li>
- <a href="../index.html" title="Introduction">
+ <a href="../index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="../introduction.html" title="Introduction">
<span class="none"></span>
Introduction</a>
</li>
@@ -166,6 +185,27 @@
<span class="none"></span>
Communication</a>
</li>
+
+ <li>
+
+ <a href="../docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
<li class="nav-header">External Links</li>
<li>
@@ -186,7 +226,7 @@
<div class="clear"></div>
<div class="clear"></div>
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
- <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; width="150" />
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
</a>
</div>
</div>
@@ -294,16 +334,20 @@ zeromq zeromq lib
<p>After the execution, czmq will be installed in /usr/local while zeromq is
installed in /usr/local/zeromq.</p></div>
<div class="section">
<h3><a name="FAQ"></a>FAQ</h3>
-<p>Q1:While compiling Singa and installing glog on max OS X, I get fatal error
“‘ext/slist’ file not found” A1:You may install
glog individually and try command :</p>
+<div class="section">
+<h4><a
name="While_compiling_Singa_and_installing_glog_on_max_OS_X_I_get_fatal_error_extslist_file_not_found."></a>While
compiling Singa and installing glog on max OS X, I get fatal error
“‘ext/slist’ file not found”.</h4>
+<p>You may install glog individually and try command:</p>
<div class="source">
<div class="source"><pre class="prettyprint">$ make CFLAGS='-stdlib=libstdc++'
CXXFLAGS='stdlib=libstdc++'
-</pre></div></div>
-<p>Q2:While compiling Singa, I get error “SSE2 instruction set not
enabled” A2:You can try following command:</p>
+</pre></div></div></div>
+<div class="section">
+<h4><a
name="While_compiling_Singa_I_get_error_SSE2_instruction_set_not_enabled."></a>While
compiling Singa, I get error “SSE2 instruction set not
enabled”.</h4>
+<p>You can try following command:</p>
<div class="source">
<div class="source"><pre class="prettyprint">$ make CFLAGS='-msse2'
CXXFLAGS='-msse2'
-</pre></div></div></div></div>
+</pre></div></div></div></div></div>
</div>
</div>
</div>
Added: websites/staging/singa/trunk/content/docs/neuralnet-partition.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/neuralnet-partition.html (added)
+++ websites/staging/singa/trunk/content/docs/neuralnet-partition.html Tue May
12 13:05:25 2015
@@ -0,0 +1,270 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2015-05-12
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20150512" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – Neural Network Partition</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript"
src="../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa";>
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index:
10000;"
+
src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png";
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn
btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="../introduction.html"
title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../quick-start.html" title="Quick
Start">Quick Start</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle"
data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../docs/installation.html"
title="Installation">Installation</a>
+</li>
+
+ <li> <a href="../docs/architecture.html"
title="System Architecture">System Architecture</a>
+</li>
+
+ <li> <a href="../docs/communication.html"
title="Communication">Communication</a>
+</li>
+
+ <li> <a href="../docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="../docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External
Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/";
title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../index.html"
id="bannerLeft" title="Apache SINGA">
+
<img src="../images/singa-logo.png" alt="Apache SINGA"
width="150"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+
<img src="../images/singa-title.png" alt="Apache SINGA"
width="600"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Neural Network Partition</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="../introduction.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../docs/installation.html"
title="Installation">
+ <span class="none"></span>
+ Installation</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/architecture.html" title="System
Architecture">
+ <span class="none"></span>
+ System Architecture</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/communication.html"
title="Communication">
+ <span class="none"></span>
+ Communication</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><span class="none"></span>Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/";
class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <div class="section">
+<h2><a name="Neural_Network_Partition"></a>Neural Network Partition</h2>
+<hr />
+<p>The purposes of partitioning neural network is to distribute the partitions
onto different working units (e.g., threads or nodes, called workers in this
article) and parallelize the processing. Another reason for partition is to
handle large neural network which cannot be hold in a single node. For
instance, to train models against images with high resolution we need large
neural networks (in terms of training parameters).</p>
+<p>Since <i>Layer</i> is the first class citizen in SIGNA, we do the partition
against layers. Specifically, we support partitions at two levels. First, users
can configure the location (i.e., worker ID) of each layer. In this way, users
assign one worker for each layer. Secondly, for one layer, we can partition its
neurons or partition the instances (e.g, images). They are called layer
partition and data partition respectively. We illustrate the two types of
partitions using an simple convolutional neural network.</p>
+<p><img src="../images/conv-mnist.png" align="center" width="200px" alt=""
/></p>
+<p>The above figure shows a convolutional neural network without any
partition. It has 8 layers in total (one rectangular represents one layer). The
first layer is DataLayer (data) which reads data from local disk
files/databases (or HDFS). The second layer is a MnistLayer which parses the
records from MNIST data to get the pixels of a batch of 28 images (each image
is of size 28x28). The LabelLayer (label) parses the records to get the label
of each image in the batch. The ConvolutionalLayer (conv1) transforms the input
image to the shape of 8x27x27. The ReLULayer (relu1) conducts elementwise
transformations. The PoolingLayer (pool1) sub-samples the images. The fc1 layer
is fully connected with pool1 layer. It mulitplies each image with a weight
matrix to generate a 10 dimension hidden feature which is then normalized by a
SoftmaxLossLayer to get the prediction.</p>
+<p><img src="../images/conv-mnist-datap.png" align="center" width="400px"
alt="" /></p>
+<p>The above figure shows the convolutional neural network after partitioning
all layers except the DataLayer and ParserLayers, into 3 partitions using data
partition. The read layers process 4 images of the batch, the black and blue
layers process 2 images respectively. Some helper layers, i.e., SliceLayer,
ConcateLayer, BridgeSrcLayer, BridgeDstLayer and SplitLayer, are added
automatically by our partition algorithm. Layers of the same color resident in
the same worker. There would be data transferring across different workers at
the boundary layers (i.e., BridgeSrcLayer and BridgeDstLayer), e.g., between
s-slice-mnist-conv1 and d-slice-mnist-conv1.</p>
+<p><img src="../images/conv-mnist-layerp.png" align="center" width="400px"
alt="" /></p>
+<p>The above figure shows the convolutional neural network after partitioning
all layers except the DataLayer and ParserLayers, into 2 partitions using layer
partition. We can see that each layer processes all 8 images from the batch.
But different partitions process different part of one image. For instance, the
layer conv1-00 process only 4 channels. The other 4 channels are processed by
conv1-01 which residents in another worker.</p>
+<p>Since the partition is done at the layer level, we can apply different
partitions for different layers to get a hybrid partition for the whole neural
network. Moreover, we can also specify the layer locations to locate different
layers to different workers.</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright ©
2015
+ <a href="http://www.apache.org";>The Apache Software
Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Added: websites/staging/singa/trunk/content/docs/programming-model.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/programming-model.html (added)
+++ websites/staging/singa/trunk/content/docs/programming-model.html Tue May 12
13:05:25 2015
@@ -0,0 +1,359 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2015-05-12
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20150512" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – Programming Model</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript"
src="../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa";>
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index:
10000;"
+
src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png";
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn
btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="../introduction.html"
title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../quick-start.html" title="Quick
Start">Quick Start</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle"
data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../docs/installation.html"
title="Installation">Installation</a>
+</li>
+
+ <li> <a href="../docs/architecture.html"
title="System Architecture">System Architecture</a>
+</li>
+
+ <li> <a href="../docs/communication.html"
title="Communication">Communication</a>
+</li>
+
+ <li> <a href="../docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="../docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External
Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/";
title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../index.html"
id="bannerLeft" title="Apache SINGA">
+
<img src="../images/singa-logo.png" alt="Apache SINGA"
width="150"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+
<img src="../images/singa-title.png" alt="Apache SINGA"
width="600"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Programming Model</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="../introduction.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../docs/installation.html"
title="Installation">
+ <span class="none"></span>
+ Installation</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/architecture.html" title="System
Architecture">
+ <span class="none"></span>
+ System Architecture</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/communication.html"
title="Communication">
+ <span class="none"></span>
+ Communication</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><span class="none"></span>Programming Model</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/";
class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <div class="section">
+<h2><a name="Programming_Model"></a>Programming Model</h2>
+<p>We describe the programming model of SINGA in this article. Base data
structures are introduced firstly, and then we show examples for users with
different levels of deep learning background.</p>
+<div class="section">
+<h3><a name="Base_Data_Structures"></a>Base Data Structures</h3>
+<div class="section">
+<h4><a name="Layer"></a>Layer</h4>
+<p>Layer is the first class citizen in SINGA. Users construct their deep
learning models by creating layer objects and combining them. SINGA takes care
of running BackPropagation (or Contrastive Divergence) algorithms to calculate
the gradients for parameters and calling <a href="#updater">Updaters</a> to
update them.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">class Layer{
+ /**
+ * Setup layer properties.
+ * Setup the shapes for data and parameters, also setup some properties
+ * based on the layer configuration and connected src layers.
+ * @param conf user defined layer configuration of type
[LayerProto](#netproto)
+ * @param srclayers layers connecting to this layer
+ */
+ Setup(conf, srclayers);
+ /**
+ * Setup the layer properties.
+ * This function is called if the model is partitioned due to distributed
+ * training. Shape of the layer is already set by the partition algorithm,
+ * and is passed in to set other properties.
+ * @param conf user defined layer configuration of type
[LayerProto](#netproto)
+ * @param shape shape set by partition algorithm (for distributed training).
+ * @param srclayers layers connecting to this layer
+ */
+ SetupAfterPartition(conf, shape, srclayers);
+ /**
+ * Compute features of this layer based on connected layers.
+ * BP and CD will call this to calculate gradients
+ * @param training boolean phase indicator for training or test
+ * @param srclayers layers connecting to this layer
+ */
+ ComputeFeature(training, srclayers);
+ /**
+ * Compute gradients for parameters and connected layers.
+ * BP and CD will call this to calculate gradients
+ * @param srclayers layers connecting to this layer.
+ */
+ ComputeGradient(srclayers)=0;
+}
+</pre></div></div>
+<p>The above pseudo code shows the base Layer class. Users override these
methods to implement their own layer classes. For example, we have implemented
popular layers like ConvolutionLayer, InnerProductLayer. We also provide a
DataLayer which is a base layer for loading (and prefetching) data from disk or
HDFS. A base ParserLayer is created for parsing the raw data and convert it
into records that are recognizable by SINGA.</p></div>
+<div class="section">
+<h4><a name="NetProto"></a>NetProto</h4>
+<p>Since deep learning models consist of multiple layers. The model structure
includes the properties of each layer and the connections between layers. SINGA
uses google protocol buffer for users to configure the model structure. The
protocol buffer message for the model structure is defined as:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">NetProto{
+ repeated LayerProto layer;
+}
+
+LayerProto{
+ string name; // user defined layer name for displaying
+ string type; // One layer class has a unique type.
+ repeated string srclayer_name; // connected layer names;
+ repeated ParamProto param; // parameter configurations
+ ...
+}
+</pre></div></div>
+<p>Users can create a plain text file and fill it with the configurations.
SINGA parses it according to user provided path.</p></div>
+<div class="section">
+<h4><a name="Param"></a>Param</h4>
+<p>The Param class is shown below. Users do not need to extend the Param class
for most cases. We make it a base class just for future extension. For example,
if a new initialization trick is proposed in the future, we can override the
<tt>Init</tt> method to implement it.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">Param{
+ /**
+ * Set properties of the parameter.
+ * @param conf user defined parameter configuration of type ParamProto
+ * @param shape shape of the parameter
+ Setup(conf, shape);
+ /**
+ * Initialize the data of the parameter.
+ /
+ Init();
+ ...// methods to handle synchronizations with parameter servers and other
workers
+}
+</pre></div></div></div>
+<div class="section">
+<h4><a name="Updater"></a>Updater</h4>
+<p>There are many SGD extensions for updating parameters, like <a
class="externalLink" href="http://arxiv.org/pdf/1212.5701v1.pdf";>AdaDelta</a>,
<a class="externalLink"
href="http://www.magicbroom.info/Papers/DuchiHaSi10.pdf";>AdaGrad</a>, <a
class="externalLink"
href="http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf";>RMSProp</a>,
<a class="externalLink"
href="http://scholar.google.com/citations?view_op=view_citation&hl=en&user=DJ8Ep8YAAAAJ&citation_for_view=DJ8Ep8YAAAAJ:hkOj_22Ku90C";>Nesterov</a>
and SGD with momentum. We provide a base Updater to deal with these
algorithms. New parameter updating algorithms can be added by extending the
base Updater.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">Updater{
+ /**
+ * @param proto user configuration for the updater.
+ Init(conf);
+ /**
+ * Update parameter based on its gradient
+ * @param step training step
+ * @param param the Param object
+ */
+ Update(step, param);
+}
+</pre></div></div></div></div>
+<div class="section">
+<h3><a name="Examples"></a>Examples</h3>
+<p>The <a href="..">MLP example</a> shows how to configure the model through
google protocol buffer.</p></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright ©
2015
+ <a href="http://www.apache.org";>The Apache Software
Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Modified: websites/staging/singa/trunk/content/index.html
==============================================================================
--- websites/staging/singa/trunk/content/index.html (original)
+++ websites/staging/singa/trunk/content/index.html Tue May 12 13:05:25 2015
@@ -53,7 +53,10 @@
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
<ul class="dropdown-menu">
- <li> <a href="index.html"
title="Introduction">Introduction</a>
+ <li> <a href="index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="introduction.html"
title="Introduction">Introduction</a>
</li>
<li> <a href="quick-start.html" title="Quick
Start">Quick Start</a>
@@ -72,6 +75,15 @@
<li> <a href="docs/communication.html"
title="Communication">Communication</a>
</li>
+
+ <li> <a href="docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
</ul>
</li>
<li class="dropdown">
@@ -135,11 +147,18 @@
<li class="active">
- <a href="#"><span class="none"></span>Introduction</a>
+ <a href="#"><span class="none"></span>Welcome</a>
</li>
<li>
+ <a href="introduction.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
<a href="quick-start.html" title="Quick Start">
<span class="none"></span>
Quick Start</a>
@@ -166,6 +185,27 @@
<span class="none"></span>
Communication</a>
</li>
+
+ <li>
+
+ <a href="docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
<li class="nav-header">External Links</li>
<li>
@@ -186,7 +226,7 @@
<div class="clear"></div>
<div class="clear"></div>
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
- <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; width="150" />
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
</a>
</div>
</div>
@@ -217,7 +257,7 @@
<li>Please subscribe to our development mailing list <a class="externalLink"
href="mailto:[email protected]";>[email protected]</a>.</li>
-<li>If you find any issues using SIGNA, please report it to the <a
class="externalLink" href="https://issues.apache.org/jira/browse/singa";>Issue
Tracker</a>.</li>
+<li>If you find any issues using SINGA, please report it to the <a
class="externalLink" href="https://issues.apache.org/jira/browse/singa";>Issue
Tracker</a>.</li>
</ul>
<p>More details on contributing to SINGA is describe <a
href="community.html">here</a>.</p></div>
<div class="section">
Added: websites/staging/singa/trunk/content/introduction.html
==============================================================================
--- websites/staging/singa/trunk/content/introduction.html (added)
+++ websites/staging/singa/trunk/content/introduction.html Tue May 12 13:05:25
2015
@@ -0,0 +1,312 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2015-05-12
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20150512" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – Introduction</title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript"
src="./js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa";>
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index:
10000;"
+
src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png";
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn
btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="introduction.html"
title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="quick-start.html" title="Quick
Start">Quick Start</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle"
data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="docs/installation.html"
title="Installation">Installation</a>
+</li>
+
+ <li> <a href="docs/architecture.html"
title="System Architecture">System Architecture</a>
+</li>
+
+ <li> <a href="docs/communication.html"
title="Communication">Communication</a>
+</li>
+
+ <li> <a href="docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External
Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/";
title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="index.html"
id="bannerLeft" title="Apache SINGA">
+
<img src="images/singa-logo.png" alt="Apache SINGA"
width="150"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+
<img src="images/singa-title.png" alt="Apache SINGA"
width="600"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Introduction</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><span class="none"></span>Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="docs/installation.html"
title="Installation">
+ <span class="none"></span>
+ Installation</a>
+ </li>
+
+ <li>
+
+ <a href="docs/architecture.html" title="System
Architecture">
+ <span class="none"></span>
+ System Architecture</a>
+ </li>
+
+ <li>
+
+ <a href="docs/communication.html"
title="Communication">
+ <span class="none"></span>
+ Communication</a>
+ </li>
+
+ <li>
+
+ <a href="docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/";
class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <div class="section">
+<h2><a name="Introduction"></a>Introduction</h2>
+<hr />
+<p>SINGA is a distributed deep learning platform, for training large-scale
deep learning models. Our design is driven by two key observations. First, the
structures and training algorithms of deep learning models can be expressed
using simple abstractions, e.g., the layer. SINGA allows users to write their
own training algorithms by exposing intuitive programming abstractions and
hiding complex details pertaining distributed execution of the training.
Specifically, our programming model consists of data objects (layer and
network) that define the model, and of computation functions over the data
objects. Our second observation is that there are multiple approaches to
partitioning the model and the training data onto multiple machines to achieve
model parallelism, data parallelism or both. Each approach incurs different
communication and synchronization overhead which directly affects the
system’s scalability. We analyze the fundamental trade-offs of existing
parallelism appro
aches, and propose an optimization algorithm that generates the parallelism
scheme with minimal overhead.</p>
+<div class="section">
+<h3><a name="Goals_and_Principles"></a>Goals and Principles</h3>
+<div class="section">
+<h4><a name="Goals"></a>Goals</h4>
+
+<ul>
+
+<li>Scalability: A distributed platform that can scale to a large model and
training dataset, e.g., 1 Billion parameters and 10M images.</li>
+
+<li>Usability: To provide abstraction and easy to use interface so that users
can implement their deep learning model/algorithm without much awareness of the
underlying distributed platform.</li>
+
+<li>Extensibility: We try to make SINGA extensible for implementing different
consistency models, training algorithms and deep learning models.</li>
+</ul></div>
+<div class="section">
+<h4><a name="Principles"></a>Principles</h4>
+<p>To achieve the scalability goal, we parallelize the computation across a
cluster of nodes by the following partitioning approaches:</p>
+
+<ul>
+
+<li>Model Partition—one model replica spreads across multiple machines
to handle large models, which have too many parameters to be kept in the memory
of a single machine. Overhead: synchronize layer data across machines within
one model replica Partition.</li>
+
+<li>Data Partition—one model replica trains against a partition of the
whole training dataset. This approach can handle large training dataset.
Overhead: synchronize parameters among model replicas.</li>
+
+<li>Hybrid Partition—exploit a cost model to find optimal model and
data partitions which would reduce both overheads.</li>
+</ul>
+<p>To achieve the usability goal, we propose our programming model with the
following two major considerations:</p>
+
+<ul>
+
+<li>Extract common data structures and operations for deep learning training
algorithms, i.e., Back Propagation and Contrastive Divergence. Users implement
their models by inheriting these data structures and overriding the
operations.</li>
+
+<li>Manage model partition and data partition automatically through
distributed array. Users write code against the distributed array, without much
awareness of the array partition (which part is stored on which machine).</li>
+</ul>
+<p>Considering extensibility, we make our core data structures (e.g., Layer)
and operations general enough for programmers to override.</p></div></div>
+<div class="section">
+<h3><a name="System_Overview"></a>System Overview</h3>
+<p><img src="images/software_stack.jpg" alt="SINGA software stack" /></p>
+<p>Three goals are considered in designing SINGA, namely ease of use,
scalability and extensibility. We will introduce them together with the
software stack as shown in the above figure. Algorithms for deep learning
models are complex to code and hard to train. To make it ease of use, we
provide a simple concept ‘Layer’ to construct deep complex
models. Built-in Layer implementations include common layers, e.g., convolution
layer and fully connected layer. Users can configure their models by combining
these built-in layers through web interface or configuration files. Once the
model and training data is configured, we start SINGA to conduct the training
using the standard training algorithm (Back-Propagation,BP or Contrastive
Divergence, CD) on a cluster of nodes and visualize the training performance to
users (e.g., through web interface). Advanced users can also implement their
own layers by overloading the base Layer class through Python, Matlab, etc
wrappers. Distr
ibutedArray is proposed for easy array operations that are heavily used for
realizing layer logics. SINGA manages the distributed arrays (stored across
multiple nodes) automatically and efficiently based on MPI. Training
scalability is achieved by partitioning the training data and model onto
multiple computing nodes and parallelizing the computation. A logically
centralized parameter server maintains the model parameters in a
ParameterTable. Computing nodes work according to the consistency policy and
send information to the parameter server which updates the parameters based on
SGD (stochastic gradient descent) algorithms. Besides the Layer class, other
components like SGD algorithms and consistency module are also extensible. <!--
-
+The above figure shows the basic components of SINGA. It starts training a deep
+learning model by parsing a model configuration, which specifies the layer and
+network structure at the every worker node. After that, it initializes the
table servers and starts
+workers to run their tasks. Each table server maintains a partition (i.e., a
set
+of rows) of a distributed parameter table where model parameters are stored.
+Worker groups consisting one or more worker nodes run in parallel to compute
the
+gradients of parameters. In one iteration, every group fetches fresh parameters
+from the table servers, runs BP or CD algorithm to compute gradients against a
+mini-batch from the local data shard (a partition of the training dataset), and
+then sends gradients to the table servers. The data shard is created by loading
+training data from HDFS off-line. The master monitors the training progress and
+stops the workers and table servers once the model has converged to a given
loss. --></p></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright ©
2015
+ <a href="http://www.apache.org";>The Apache Software
Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Modified: websites/staging/singa/trunk/content/quick-start.html
==============================================================================
--- websites/staging/singa/trunk/content/quick-start.html (original)
+++ websites/staging/singa/trunk/content/quick-start.html Tue May 12 13:05:25
2015
@@ -53,7 +53,10 @@
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
<ul class="dropdown-menu">
- <li> <a href="index.html"
title="Introduction">Introduction</a>
+ <li> <a href="index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="introduction.html"
title="Introduction">Introduction</a>
</li>
<li> <a href="quick-start.html" title="Quick
Start">Quick Start</a>
@@ -72,6 +75,15 @@
<li> <a href="docs/communication.html"
title="Communication">Communication</a>
</li>
+
+ <li> <a href="docs/code-structure.html"
title="Code Structure">Code Structure</a>
+</li>
+
+ <li> <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">Neural Network Partition</a>
+</li>
+
+ <li> <a href="docs/programming-model.html"
title="Programming Model">Programming Model</a>
+</li>
</ul>
</li>
<li class="dropdown">
@@ -135,7 +147,14 @@
<li>
- <a href="index.html" title="Introduction">
+ <a href="index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="introduction.html" title="Introduction">
<span class="none"></span>
Introduction</a>
</li>
@@ -166,6 +185,27 @@
<span class="none"></span>
Communication</a>
</li>
+
+ <li>
+
+ <a href="docs/code-structure.html" title="Code
Structure">
+ <span class="none"></span>
+ Code Structure</a>
+ </li>
+
+ <li>
+
+ <a href="docs/neuralnet-partition.html"
title="Neural Network Partition">
+ <span class="none"></span>
+ Neural Network Partition</a>
+ </li>
+
+ <li>
+
+ <a href="docs/programming-model.html"
title="Programming Model">
+ <span class="none"></span>
+ Programming Model</a>
+ </li>
<li class="nav-header">External Links</li>
<li>
@@ -186,7 +226,7 @@
<div class="clear"></div>
<div class="clear"></div>
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
- <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; width="150" />
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
</a>
</div>
</div>
@@ -213,7 +253,7 @@ git clone https://github.com/apache/incu
<div class="source"><pre class="prettyprint">./configure
make
</pre></div></div>
-<p>If there are dependent libraries missing, please refer to [installation]({{
BASE_PATH }}{% post_url /docs/2015-01-20-installation %}) page for guidance on
installing them. After successful compilation, the libsinga.so and singa
executable will be built into the build folder.</p></div>
+<p>If there are dependent libraries missing, please refer to <a
href="docs/installation.html">installation</a> page for guidance on installing
them. After successful compilation, the libsinga.so and singa executable will
be built into the build folder.</p></div>
<div class="section">
<h3><a name="Run_in_standalone_mode"></a>Run in standalone mode</h3>
<p>Running SINGA in standalone mode is on the contrary of running it on Mesos
or YARN. For standalone mode, users have to manage the resources manually. For
instance, they have to prepare a host file containing all running nodes. There
is no management on CPU and memory resources, hence SINGA consumes as much CPU
and memory resources as it needs.</p>
@@ -243,7 +283,7 @@ nworkers_per_group: 1
nserver_groups: 1
nservers_per_group: 1
</pre></div></div>
-<p>One worker group trains against one partition of the training dataset. If
<i>nworker_groups</i> is set to 1, then there is no data partitioning. One
worker runs over a partition of the model. If <i>nworkers_per_group</i> is set
to 1, then there is no model partitioning. More details on the cluster
configuration are described in the <a href="">System Architecture</a> page.</p>
+<p>One worker group trains against one partition of the training dataset. If
<i>nworker_groups</i> is set to 1, then there is no data partitioning. One
worker runs over a partition of the model. If <i>nworkers_per_group</i> is set
to 1, then there is no model partitioning. More details on the cluster
configuration are described in the <a href="docs/architecture.html">System
Architecture</a> page.</p>
<p>Start the training by running:</p>
<div class="source">
![http://incubator.apache.org/images/egg-logo.png"](http://incubator.apache.org/images/egg-logo.png"){kind=link}
![https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"](https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"){kind=link}