Author: buildbot
Date: Wed Sep 2 10:31:57 2015
New Revision: 964016
Log:
Staging update by buildbot for singa
Added:
websites/staging/singa/trunk/content/docs/neural-net.html
websites/staging/singa/trunk/content/docs/overview.html
websites/staging/singa/trunk/content/docs/param.html
websites/staging/singa/trunk/content/docs/programming-guide.html
websites/staging/singa/trunk/content/docs/quick-start.html
websites/staging/singa/trunk/content/docs/train-one-batch.html
websites/staging/singa/trunk/content/docs/updater.html
Removed:
websites/staging/singa/trunk/content/docs/datashard.html
websites/staging/singa/trunk/content/docs/hdfs.html
Modified:
websites/staging/singa/trunk/content/ (props changed)
websites/staging/singa/trunk/content/docs/architecture.html
websites/staging/singa/trunk/content/docs/checkpoint.html
websites/staging/singa/trunk/content/docs/cnn.html
websites/staging/singa/trunk/content/docs/code-structure.html
websites/staging/singa/trunk/content/docs/communication.html
websites/staging/singa/trunk/content/docs/data.html
websites/staging/singa/trunk/content/docs/debug.html
websites/staging/singa/trunk/content/docs/distributed-training.html
websites/staging/singa/trunk/content/docs/examples.html
websites/staging/singa/trunk/content/docs/frameworks.html
websites/staging/singa/trunk/content/docs/installation.html
websites/staging/singa/trunk/content/docs/layer.html
websites/staging/singa/trunk/content/docs/lmdb.html
websites/staging/singa/trunk/content/docs/mlp.html
websites/staging/singa/trunk/content/docs/model-config.html
websites/staging/singa/trunk/content/docs/neuralnet-partition.html
websites/staging/singa/trunk/content/docs/programmer-guide.html
websites/staging/singa/trunk/content/docs/rbm.html
websites/staging/singa/trunk/content/docs/rnn.html
websites/staging/singa/trunk/content/index.html
websites/staging/singa/trunk/content/introduction.html
websites/staging/singa/trunk/content/quick-start.html
Propchange: websites/staging/singa/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Wed Sep 2 10:31:57 2015
@@ -1 +1 @@
-1700726
+1700761
Modified: websites/staging/singa/trunk/content/docs/architecture.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/architecture.html (original)
+++ websites/staging/singa/trunk/content/docs/architecture.html Wed Sep 2
10:31:57 2015
@@ -9,7 +9,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
- <title>Apache SINGA – </title>
+ <title>Apache SINGA – SINGA Architecture</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
<link rel="stylesheet" href="../css/site.css" />
<link rel="stylesheet" href="../css/print.css" media="print" />
@@ -189,7 +189,7 @@
Apache SINGA</a>
<span class="divider">/</span>
</li>
- <li class="active "></li>
+ <li class="active ">SINGA Architecture</li>
@@ -423,10 +423,8 @@
<div id="bodyColumn" class="span10" >
- <p>— layout: post title: Architecture category : docs</p>
+ <h1>SINGA Architecture</h1>
<div class="section">
-<h2><a name="tags_:_architecture"></a>tags : [architecture]</h2>
-<p>{% include JB/setup %}</p>
<div class="section">
<h3><a name="Logical_Architecture"></a>Logical Architecture</h3>
<p><img src="http://singa.incubator.apache.org/assets/image/logical.png";
style="width: 550px" alt="" />
Modified: websites/staging/singa/trunk/content/docs/checkpoint.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/checkpoint.html (original)
+++ websites/staging/singa/trunk/content/docs/checkpoint.html Wed Sep 2
10:31:57 2015
@@ -9,7 +9,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
- <title>Apache SINGA – </title>
+ <title>Apache SINGA – CheckPoint</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
<link rel="stylesheet" href="../css/site.css" />
<link rel="stylesheet" href="../css/print.css" media="print" />
@@ -189,7 +189,7 @@
Apache SINGA</a>
<span class="divider">/</span>
</li>
- <li class="active "></li>
+ <li class="active ">CheckPoint</li>
@@ -423,10 +423,7 @@
<div id="bodyColumn" class="span10" >
- <p>— layout: post title: Checkpoint and Resume category :
docs</p>
-<div class="section">
-<h2><a name="tags_:_checkpoint_restore"></a>tags : [checkpoint, restore]</h2>
-<p>{% include JB/setup %}</p>
+ <h1>CheckPoint</h1>
<p>SINGA checkpoints model parameters onto disk periodically according to user
configured frequency. By checkpointing model parameters, we can</p>
<ol style="list-style-type: decimal">
@@ -436,7 +433,7 @@
<li>
<p>use them to initialize a similar model. For example, the parameters from
training a RBM model can be used to initialize a <a class="externalLink"
href="http://singa.incubator.apache.org/docs/rbm";>deep auto-encoder</a>
model.</p></li>
-</ol></div>
+</ol>
<div class="section">
<h2><a name="Configuration"></a>Configuration</h2>
<p>Checkpointing is controlled by two configuration fields:</p>
Modified: websites/staging/singa/trunk/content/docs/cnn.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/cnn.html (original)
+++ websites/staging/singa/trunk/content/docs/cnn.html Wed Sep 2 10:31:57 2015
@@ -9,7 +9,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
- <title>Apache SINGA – </title>
+ <title>Apache SINGA – CNN Example</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
<link rel="stylesheet" href="../css/site.css" />
<link rel="stylesheet" href="../css/print.css" media="print" />
@@ -189,7 +189,7 @@
Apache SINGA</a>
<span class="divider">/</span>
</li>
- <li class="active "></li>
+ <li class="active ">CNN Example</li>
@@ -425,11 +425,8 @@
<div id="bodyColumn" class="span10" >
- <p>— layout: post title: Example — Convolution
Neural Network category : docs</p>
-<div class="section">
-<h2><a name="tags_:_cnn_example"></a>tags : [cnn, example]</h2>
-<p>{% include JB/setup %}</p>
-<p>Convolutional neural network (CNN) is a type of feed-forward artificial
neural network widely used for image and video classification. In this example,
we will use a deep CNN model to do image classification for the <a
class="externalLink" href="http://www.cs.toronto.edu/~kriz/cifar.html";>CIFAR10
dataset</a>.</p></div>
+ <h1>CNN Example</h1>
+<p>Convolutional neural network (CNN) is a type of feed-forward artificial
neural network widely used for image and video classification. In this example,
we will use a deep CNN model to do image classification for the <a
class="externalLink" href="http://www.cs.toronto.edu/~kriz/cifar.html";>CIFAR10
dataset</a>.</p>
<div class="section">
<h2><a name="Running_instructions"></a>Running instructions</h2>
<p>Please refer to the <a class="externalLink"
href="http://singa.incubator.apache.org/docs/installation";>installation</a>
page for instructions on building SINGA, and the <a class="externalLink"
href="http://singa.incubator.apache.org/docs/quick-start";>quick start</a> for
instructions on starting zookeeper.</p>
Modified: websites/staging/singa/trunk/content/docs/code-structure.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/code-structure.html (original)
+++ websites/staging/singa/trunk/content/docs/code-structure.html Wed Sep 2
10:31:57 2015
@@ -425,8 +425,7 @@
<div id="bodyColumn" class="span10" >
- <div class="section">
-<h2><a name="Code_Structure"></a>Code Structure</h2>
+ <h1>Code Structure</h1>
<hr />
<!-- ### Worker Side
@@ -495,7 +494,7 @@ The figure above shows how a GET request
table server. The control flow for other types of requests is similar. At
the server side, there are at least 3 threads running at any time: two by
NetworkService for sending and receiving message, and at least one by the
-RequestDispatcher for dispatching requests. --></div>
+RequestDispatcher for dispatching requests. -->
</div>
</div>
</div>
Modified: websites/staging/singa/trunk/content/docs/communication.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/communication.html (original)
+++ websites/staging/singa/trunk/content/docs/communication.html Wed Sep 2
10:31:57 2015
@@ -9,7 +9,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
- <title>Apache SINGA – </title>
+ <title>Apache SINGA – Communication</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
<link rel="stylesheet" href="../css/site.css" />
<link rel="stylesheet" href="../css/print.css" media="print" />
@@ -189,7 +189,7 @@
Apache SINGA</a>
<span class="divider">/</span>
</li>
- <li class="active "></li>
+ <li class="active ">Communication</li>
@@ -423,16 +423,14 @@
<div id="bodyColumn" class="span10" >
- <p>— layout: post title: Communication category : docs</p>
-<div class="section">
-<h2><a name="tags_:_rnn_example"></a>tags : [rnn, example]</h2>
-<p>{% include JB/setup %}</p>
+ <h1>Communication</h1>
<p>Different messaging libraries has different benefits and drawbacks. For
instance, MPI provides fast message passing between GPUs (using GPUDirect), but
does not support fault-tolerance well. On the contrary, systems using ZeroMQ
can be fault-tolerant, but does not support GPUDirect. The AllReduce function
of MPI is also missing in ZeroMQ which is efficient for data aggregation for
distributed training. In Singa, we provide general messaging APIs for
communication between threads within a process and across processes, and let
users choose the underlying implementation (MPI or ZeroMQ) that meets their
requirements.</p>
<p>Singa’s messaging library consists of two components, namely the
message, and the socket to send and receive messages. <b>Socket</b> refers to a
Singa defined data structure instead of the Linux Socket. We will introduce the
two components in detail with the following figure as an example
architecture.</p>
<p><img src="../images/arch/arch2.png" style="width: 550px" alt="" /> <img
src="../images/arch/comm.png" style="width: 550px" alt="" />
<p><b> Fig.1 - Example physical architecture and network connection</b></p>
<p>Fig.1 shows an example physical architecture and its network connection. <a
class="externalLink"
href="http://singa.incubator.apache.org/docs/architecture.html}";>Section-partition
server side ParamShard</a> has a detailed description of the architecture.
Each process consists of one main thread running the stub and multiple
background threads running the worker and server tasks. The stub of the main
thread forwards messages among threads . The worker and server tasks are
performed by the background threads.</p>
<div class="section">
+<div class="section">
<h3><a name="Message"></a>Message</h3>
<p><object type="image/svg+xml" style="width: 100px" data="../images/msg.svg">
Not supported </object>
<p><b> Fig.2 - Logical message format</b></p>
Modified: websites/staging/singa/trunk/content/docs/data.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/data.html (original)
+++ websites/staging/singa/trunk/content/docs/data.html Wed Sep 2 10:31:57 2015
@@ -0,0 +1,643 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2015-09-02
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20150902" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – Data Preparation</title>
+ <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../css/site.css" />
+ <link rel="stylesheet" href="../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript"
src="../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa";>
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index:
10000;"
+
src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png";
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn
btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache
SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../index.html"
title="Welcome">Welcome</a>
+</li>
+
+ <li> <a href="../introduction.html"
title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../quick-start.html" title="Quick
Start">Quick Start</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle"
data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../docs/installation.html"
title="Installation">Installation</a>
+</li>
+
+ <li class="dropdown-submenu">
+ <a href="../docs/programmer-guide.html"
title="Programmer Guide">Programmer Guide</a>
+ <ul class="dropdown-menu">
+ <li> <a
href="../docs/model-config.html" title="Model Configuration">Model
Configuration</a>
+</li>
+ <li> <a href="../docs/neuralnet.html"
title="Neural Network">Neural Network</a>
+</li>
+ <li> <a href="../docs/layer.html"
title="Layer">Layer</a>
+</li>
+ <li> <a href="../docs/param.html"
title="Param">Param</a>
+</li>
+ </ul>
+ </li>
+
+ <li class="dropdown-submenu">
+ <a
href="../docs/distributed-training.html" title="Distributed
Training">Distributed Training</a>
+ <ul class="dropdown-menu">
+ <li> <a
href="../docs/architecture.html" title="System Architecture">System
Architecture</a>
+</li>
+ <li> <a href="../docs/frameworks.html"
title="Frameworks">Frameworks</a>
+</li>
+ <li> <a
href="../docs/communication.html" title="Communication">Communication</a>
+</li>
+ </ul>
+ </li>
+
+ <li> <a href="../docs/data.html" title="Data
Preparation">Data Preparation</a>
+</li>
+
+ <li> <a href="../docs/checkpoint.html"
title="Checkpoint">Checkpoint</a>
+</li>
+
+ <li> <a href="../docs/examples.html"
title="Examples">Examples</a>
+</li>
+
+ <li> <a href="../docs/debug.html"
title="Debug">Debug</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development
<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../develop/schedule.html"
title="Schedule">Schedule</a>
+</li>
+
+ <li class="dropdown-submenu">
+ <a href="../develop/how-contribute.html"
title="How to Contribute">How to Contribute</a>
+ <ul class="dropdown-menu">
+ <li> <a
href="../develop/contribute-code.html" title="Code">Code</a>
+</li>
+ <li> <a
href="../develop/contribute-docs.html" title="Documentation">Documentation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community
<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../community/source-repository.html"
title="Source Repository">Source Repository</a>
+</li>
+
+ <li> <a href="../community/mail-lists.html"
title="Mailing Lists">Mailing Lists</a>
+</li>
+
+ <li> <a href="../community/issue-tracking.html"
title="Issue Tracking">Issue Tracking</a>
+</li>
+
+ <li> <a href="../community/team-list.html"
title="SINGA Team">SINGA Team</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External
Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/";
title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+
+ <li> <a
href="http://www.comp.nus.edu.sg/~dbsystem/singa/"; title="NUS School of
Computing">NUS School of Computing</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../index.html"
id="bannerLeft" title="Apache SINGA">
+
<img src="../images/singa-logo.png" alt="Apache SINGA"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+
<img src="../images/singa-title.png" alt="Apache SINGA"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Data Preparation</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../index.html" title="Welcome">
+ <span class="none"></span>
+ Welcome</a>
+ </li>
+
+ <li>
+
+ <a href="../introduction.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../docs/installation.html"
title="Installation">
+ <span class="none"></span>
+ Installation</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/programmer-guide.html"
title="Programmer Guide">
+ <span class="icon-chevron-down"></span>
+ Programmer Guide</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../docs/model-config.html" title="Model
Configuration">
+ <span class="none"></span>
+ Model Configuration</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/neuralnet.html" title="Neural
Network">
+ <span class="none"></span>
+ Neural Network</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/layer.html" title="Layer">
+ <span class="none"></span>
+ Layer</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/param.html" title="Param">
+ <span class="none"></span>
+ Param</a>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+
+ <a href="../docs/distributed-training.html"
title="Distributed Training">
+ <span class="icon-chevron-down"></span>
+ Distributed Training</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../docs/architecture.html" title="System
Architecture">
+ <span class="none"></span>
+ System Architecture</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/frameworks.html" title="Frameworks">
+ <span class="none"></span>
+ Frameworks</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/communication.html"
title="Communication">
+ <span class="none"></span>
+ Communication</a>
+ </li>
+ </ul>
+ </li>
+
+ <li class="active">
+
+ <a href="#"><span class="none"></span>Data Preparation</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/checkpoint.html" title="Checkpoint">
+ <span class="none"></span>
+ Checkpoint</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/examples.html" title="Examples">
+ <span class="none"></span>
+ Examples</a>
+ </li>
+
+ <li>
+
+ <a href="../docs/debug.html" title="Debug">
+ <span class="none"></span>
+ Debug</a>
+ </li>
+ <li class="nav-header">Development</li>
+
+ <li>
+
+ <a href="../develop/schedule.html" title="Schedule">
+ <span class="none"></span>
+ Schedule</a>
+ </li>
+
+ <li>
+
+ <a href="../develop/how-contribute.html" title="How
to Contribute">
+ <span class="icon-chevron-down"></span>
+ How to Contribute</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="../develop/contribute-code.html"
title="Code">
+ <span class="none"></span>
+ Code</a>
+ </li>
+
+ <li>
+
+ <a href="../develop/contribute-docs.html"
title="Documentation">
+ <span class="none"></span>
+ Documentation</a>
+ </li>
+ </ul>
+ </li>
+ <li class="nav-header">Community</li>
+
+ <li>
+
+ <a href="../community/source-repository.html"
title="Source Repository">
+ <span class="none"></span>
+ Source Repository</a>
+ </li>
+
+ <li>
+
+ <a href="../community/mail-lists.html"
title="Mailing Lists">
+ <span class="none"></span>
+ Mailing Lists</a>
+ </li>
+
+ <li>
+
+ <a href="../community/issue-tracking.html"
title="Issue Tracking">
+ <span class="none"></span>
+ Issue Tracking</a>
+ </li>
+
+ <li>
+
+ <a href="../community/team-list.html" title="SINGA
Team">
+ <span class="none"></span>
+ SINGA Team</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/";
class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+
+ <li>
+
+ <a
href="http://www.comp.nus.edu.sg/~dbsystem/singa/"; class="externalLink"
title="NUS School of Computing">
+ <span class="none"></span>
+ NUS School of Computing</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+
<a href="http://incubator.apache.org";
title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator"
src="http://incubator.apache.org/images/egg-logo.png"; />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <h1>Data Preparation</h1>
+<p>To submit a training job, users need to convert raw data (e.g., images,
text documents) into SINGA recognizable <a
href="api/classsinga_1_1Record.html">Record</a>s. SINGA uses <a
class="externalLink"
href="http://singa.incubator.apache.org/docs/layer#data-layers";>data layers</a>
to load these records into memory and uses <a class="externalLink"
href="http://singa.incubator.apache.org/docs/layer#parser-layers";>parser
layers</a> to parse features (e.g., image pixels and labels) from these
<tt>Record</tt>s. <tt>Record</tt>s could be stored in a file, a database, or
HDFS, as long as there is a corresponding <a class="externalLink"
href="http://singa.incubator.apache.org/api/classsinga_1_1DataLayer.html";>DataLayer</a>.</p>
+<div class="section">
+<h2><a name="DataShard"></a>DataShard</h2>
+<p>SINGA comes with a light-weight database named <a class="externalLink"
href="http://singa.incubator.apache.org/api/classsinga_1_1DataShard.html";>DataShard</a>.
It provides operations for inserting <tt>Record</tt>, and read <tt>Record</tt>
in sequential order. <tt>Record</tt>s are flushed once the maximum cache size
is reached. It loads <tt>Record</tt>s in batch and returns them to users one by
one through the <a class="externalLink"
href="http://singa.incubator.apache.org/api/classsinga_1_1DataShard.html";>Next</a>
function. The disk folder in which the <tt>Record</tt>s are stored, is called
a (data) shard. The <a class="externalLink"
href="http://singa.incubator.apache.org/api/classsinga_1_1ShardDataLayer.html";>ShardDataLayer</a>
is a built-in layer for loading <tt>Record</tt>s from <tt>DataShard</tt>.</p>
+<p>To create data shards for users’ own data, they can follow the
subsequent sections.</p>
+<div class="section">
+<h3><a name="User_record_definition"></a>User record definition</h3>
+<p>Users define their own record for storing their data. E.g., the built-in <a
class="externalLink"
href="http://singa.incubator.apache.org/api/classsinga_1_1SingleLabelImageRecord.html";>SingleLabelImageRecord</a>
has an int field for image label, and a pixel array for image RGB values. The
code below shows an example of defining a new record <tt>UserRecord</tt>, and
extending the base <tt>Record</tt> to include <tt>UserRecord</tt>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">package singa;
+
+import "common.proto"; // required to import common.proto
+
+message UserRecord {
+ repeated int userVAR1 = 1; // unique field id
+ optional string userVAR2 = 2; // unique field id
+ ...
+}
+
+extend Record {
+ optional UserRecord user_record = 101; // unique extension field id,
reserved for users (e.g., 101-200)
+}
+</pre></div></div>
+<p>Please refer to the <a class="externalLink"
href="https://developers.google.com/protocol-buffers/docs/reference/cpp-generated?hl=en#extension";>Tutorial</a>
for extension of protocol messages.</p>
+<p>The extended <tt>Record</tt> will be parsed by a parser layer to extract
features (e.g., label or pixel values). Users need to write their own <a
class="externalLink"
href="http://singa.incubator.apache.org/docs/layer#parser-layers";>parser
layers</a> to parse the extended <tt>Record</tt>.</p>
+<p>{% comment %} <i>Note</i></p>
+<p>There is an alternative way to define the proto extension. In this way, you
should be careful of the scope of fields and how to access the fields, which
are different from the above.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">message UserRecord {
+ extend Record {
+ optional UserRecord user_record = 101; // unique extension field id,
reserved for users (e.g., 101-200)
+ }
+ repeated int userVAR1 = 1; // unique field id
+ optional string userVAR2 = 2; // unique field id
+ ...
+}
+</pre></div></div>
+<p>{% endcomment %}</p></div>
+<div class="section">
+<h3><a name="DataShard_creation"></a>DataShard creation</h3>
+<p>Users write code to convert their data into <tt>Record</tt>s and insert
them into shards following the subsequent steps.</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Create a folder <i>USER_DATA</i> under <i>SINGA_ROOT</i>.</p></li>
+
+<li>
+<p>Prepare the source file, e.g., <tt>create_shard.cc</tt>, in
<tt>SINGA_ROOT/USER_DATA</tt></p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">singa::DataShard
myShard(outputpath, kCreate);
+// outputpath is the path of the folder for storing the shard
+</pre></div></div>
+<p>the above code opens a folder for storing the data shard.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">singa::Record record;
+singa::UserRecord* r = record.MutableExtension(singa::user_record);
+</pre></div></div>
+<p>an user-defined record is allocated by the above code.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">r->add_userVAR1( int_val );
// for repeated field
+r->set_userVAR2( string_val );
+</pre></div></div>
+<p>users load raw data and set/add them into user-defined record as shown
above.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">// key (string) is a unique
record ID (e.g., converted from a number starting from 0)
+myShard.Insert( key, record );
+</pre></div></div>
+<p>Once the <tt>record</tt> object is filled, it is inserted into the shard as
shown above. If there are multiple data records, they should be inserted
sequentially. After inserting all records, the shard is created into the
<tt>outputpath</tt> folder.</p></li>
+
+<li>
+<p>Compile and link. Both <i>user.proto</i> and <i>create.cc</i> should be
compiled and linked with libsinga.so. The following instruction generates
<i>user.pb.cc</i> and <i>user.pb.h</i> from <i>user.proto</i>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">protoc -I=SINGA_ROOT/USER_DATA
--cpp_out=SINGA_ROOT/USER_DATA user.proto
+</pre></div></div>
+<p>All code can be compiled and linked into an executable file</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">g++ create_shard.cc user.pb.cc
-std=c++11 -lsinga \
+ -ISINGA_ROOT/include -LSINGA_ROOT/.libs/
-Wl,-unresolved-symbols=ignore-in-shared-libs \
+
+ -Wl,-rpath=SINGA_ROOT/.libs/ -o create_shard.bin
+</pre></div></div></li>
+</ol>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Run the program. Once the executable file is generated, users can run it to
create data shards.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">./create_shard.bin <args>
+</pre></div></div></li>
+</ol></div>
+<div class="section">
+<h3><a name="Example_-_CIFAR_dataset"></a>Example - CIFAR dataset</h3>
+<p>This example uses the <a class="externalLink"
href="http://www.cs.toronto.edu/~kriz/cifar.html";>CIFAR-10 image dataset</a>
collected by Alex Krizhevsky. It consists of 60,000 32x32 color images in 10
classes, with 6,000 images per class. There are 50,000 training images and
10,000 test images. Each image has a single label. This dataset is stored in
binary files with specific format. SINGA has written the <a
class="externalLink"
href="https://github.com/apache/incubator-singa/blob/master/examples/cifar10/create_shard.cc";>create_shard.cc</a>
to convert images in the binary files into <tt>Record</tt>s and insert them
into training and test shards.</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Download raw data. The following command will download the dataset into
<i>cifar-10-batches-bin</i> folder.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># in SINGA_ROOT/examples/cifar10
+$ cp Makefile.example Makefile // an example makefile is provided
+$ make download
+</pre></div></div></li>
+
+<li>
+<p>Since <tt>Record</tt> already has one <tt>image</tt> field which is
designed for single-label images, e.g., images from CIFAR10, we can use it
directly. Particularly, the type of <tt>image</tt> is
<tt>SingleLabelImageRecord</tt>,</p></li>
+</ol>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # common.proto
+ package singa;
+
+ message Record {
+ enum Type {
+ kSingleLabelImage = 0;
+ }
+ optional Type type = 1 [default = kSingleLabelImage];
+ optional SingleLabelImageRecord image = 2; // for configuration
+ }
+
+ message SingleLabelImageRecord {
+ repeated int32 shape = 1; // it obtains 3 (rgb channels),
32 (row), 32 (col)
+ optional int32 label = 2; // label
+ optional bytes pixel = 3; // pixels
+ repeated float data = 4 [packed = true]; // it is used for normalization
+ }
+</pre></div></div>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Add/Set data into the record, and insert it to shard.
<tt>create_shard.cc</tt> reads images (and labels) from the downloaded binary
files. For each image, it puts the image feature and label into a
<tt>SingleLabelImageRecord</tt> of <tt>Record</tt>, and then inserts the
<tt>Record</tt> into <tt>DataShard</tt>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> ...// open binary files
+DataShard train_shard("cifar10_train_datashard", DataShard::kCreate);
+
+singa::Record record;
+singa::SingleLabelImageRecord* image = record.mutable_image();;
+for (int image_id = 0; image_id < 50000; image_id ++) {
+ read_image(&data_file, &label, str_buffer); // read feature and
label from binary file
+ image->set_label(label); // put label
+ image->set_pixel(str_buffer); // put image feature
+ train_shard.Insert(to_string(image_id), record); // insert a record
with unique ID
+}
+</pre></div></div>
+<p>The data shard for testing data is created similarly. In addition, it
computes average values (not shown here) of image pixels as another
<tt>Record</tt> which is directly serialized into
<i>SINGA_ROOT/USER_DATA/image_mean.bin</i>. The mean values will be used for
preprocessing image features.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">for (int itemid = 0; itemid <
kCIFARBatchSize; ++itemid) {
+ const string& pixels = image->pixel();
+ for(int i=0; i<kCIFARImageNBytes; i++)
+ mean.set_data(i, mean.data(i)+static_cast<uint8_t>(pixels[i]));
+ count += 1;
+}
+for(int i=0; i<kCIFARImageNBytes; i++)
+ mean.set_data(i, mean.data(i)/count);
+</pre></div></div></li>
+
+<li>
+<p>Compile and run the program. SINGA provides an example Makefile that
contains instructions for compiling the source code and linking it with
<i>libsinga.so</i>. Users just execute the following command.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ make create
+</pre></div></div>
+<p>The data shards for training and testing will be generated into
<i>cifar10_train_shard</i> and <i>cifar10_test_shard</i> folders
respectively.</p></li>
+</ol></div>
+<div class="section">
+<h3><a name="Example_-_MNIST_dataset"></a>Example - MNIST dataset</h3>
+<p>This example creates <tt>DataShard</tt>s for the <a class="externalLink"
href="http://yann.lecun.com/exdb/mnist/";>MNIST dataset</a>. It has a training
set of 60,000 handwritten digit images, and a test set of 10,000 images.
Similar to the images of CIFAR10, each MNIST image has a single label. Hence,
we still use the built-in <tt>Record</tt>. The process is almost the same as
that for the CIFAR10 dataset, except that the MNIST dataset is downloaded as
binary files with another format. SINGA has written the <i>create_shard.cc</i>
program to parse the binary files and convert MNIST images into
<tt>Record</tt>s.</p>
+<p>The following command will download the dataset</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ cp Makefile.example Makefile
// an example makefile is provided
+$ make download
+</pre></div></div>
+<p>Data shards will be generated into <i>mnist_train_shard</i> and
<i>mnist_test_shard</i> by</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ make create
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="LMDB"></a>LMDB</h2>
+<p>To be filled soon.</p></div>
+<div class="section">
+<h2><a name="HDFS"></a>HDFS</h2>
+<p>To be filled soon.</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved.
Apache Singa, Apache, the Apache feather logo, and the Apache Singa project
logos are trademarks of The Apache Software Foundation. All other marks
mentioned may be trademarks or registered trademarks of their respective
owners.</p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Modified: websites/staging/singa/trunk/content/docs/debug.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/debug.html (original)
+++ websites/staging/singa/trunk/content/docs/debug.html Wed Sep 2 10:31:57
2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-17
+ | Generated by Apache Maven Doxia at 2015-09-02
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150817" />
+ <meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – How to Debug</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -423,10 +423,10 @@
<div id="bodyColumn" class="span10" >
- <div class="section">
-<h2><a name="How_to_Debug"></a>How to Debug</h2>
+ <h1>How to Debug</h1>
<hr />
<div class="section">
+<div class="section">
<h3><a name="Compile_with_-g"></a>Compile with -g</h3>
<p>Since SINGA is developed on Linux using C++, GDB is the preferred debugging
tool. To use GDB, the code must be compiled with <tt>-g</tt> flag. The files
(e.g., Makefile.am) from GNU compilation tool are not easy to change. Hence
SINGA comes with a Makefile.example for developers. To compile the code for
debugging, just</p>
Modified: websites/staging/singa/trunk/content/docs/distributed-training.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/distributed-training.html
(original)
+++ websites/staging/singa/trunk/content/docs/distributed-training.html Wed Sep
2 10:31:57 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-17
+ | Generated by Apache Maven Doxia at 2015-09-02
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150817" />
+ <meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Distributed Training</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -423,23 +423,21 @@
<div id="bodyColumn" class="span10" >
- <div class="section">
-<h2><a name="Distributed_Training"></a>Distributed Training</h2>
-<hr />
+ <h1>Distributed Training</h1>
<p>SINGA is designed for distributed training of large deep learning models
with huge amount of training data.</p>
<p>Here we introduce distrbuted SINGA in following aspects:</p>
<ul>
<li>
-<p><a href="architecture.html">System Architecture</a></p></li>
+<p><a class="externalLink"
href="http://singa.incubator.apache.org/docs/architecture";>System
Architecture</a></p></li>
<li>
-<p><a href="frameworks.html">Training Frameworks</a></p></li>
+<p><a class="externalLink"
href="http://singa.incubator.apache.org/docs/frameworks";>Training
Frameworks</a></p></li>
<li>
-<p><a href="communication.html">System Communication</a></p></li>
-</ul></div>
+<p><a class="externalLink"
href="http://singa.incubator.apache.org/docs/communication";>System
Communication</a></p></li>
+</ul>
</div>
</div>
</div>
Modified: websites/staging/singa/trunk/content/docs/examples.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/examples.html (original)
+++ websites/staging/singa/trunk/content/docs/examples.html Wed Sep 2 10:31:57
2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-17
+ | Generated by Apache Maven Doxia at 2015-09-02
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150817" />
+ <meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Example Models</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -423,10 +423,10 @@
<div id="bodyColumn" class="span10" >
- <div class="section">
-<h2><a name="Example_Models"></a>Example Models</h2>
+ <h1>Example Models</h1>
<p>Different models are provided as examples to help users get familiar with
SINGA. <a href="neuralnet.html">Neural Network</a> gives details on the models
that are supported by SINGA.</p>
<div class="section">
+<div class="section">
<h3><a name="Feed-forward_neural_networks"></a>Feed-forward neural
networks</h3>
<ul>
Modified: websites/staging/singa/trunk/content/docs/frameworks.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/frameworks.html (original)
+++ websites/staging/singa/trunk/content/docs/frameworks.html Wed Sep 2
10:31:57 2015
@@ -1,15 +1,15 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-17
+ | Generated by Apache Maven Doxia at 2015-09-02
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150817" />
+ <meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
- <title>Apache SINGA – Distributed Training Frameworks</title>
+ <title>Apache SINGA – Distributed Training Framework</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
<link rel="stylesheet" href="../css/site.css" />
<link rel="stylesheet" href="../css/print.css" media="print" />
@@ -189,7 +189,7 @@
Apache SINGA</a>
<span class="divider">/</span>
</li>
- <li class="active ">Distributed Training Frameworks</li>
+ <li class="active ">Distributed Training Framework</li>
@@ -423,9 +423,8 @@
<div id="bodyColumn" class="span10" >
- <div class="section">
-<h2><a name="Distributed_Training_Frameworks"></a>Distributed Training
Frameworks</h2>
-<hr />
+ <h1>Distributed Training Framework</h1>
+<div class="section">
<div class="section">
<h3><a name="Cluster_Topology_Configuration"></a>Cluster Topology
Configuration</h3>
<p>Here we describe how to configure SINGA’s cluster topology to
support different distributed training frameworks. The cluster topology is
configured in the <tt>cluster</tt> field in <tt>JobProto</tt>. The
<tt>cluster</tt> is of type <tt>ClusterProto</tt>:</p>
@@ -458,7 +457,7 @@
<div class="section">
<h3><a name="Different_Training_Frameworks"></a>Different Training
Frameworks</h3>
<p>In SINGA, worker groups run asynchronously and workers within one group run
synchronously. Users can leverage this general design to run both
<b>synchronous</b> and <b>asynchronous</b> training frameworks. Here we
illustrate how to configure popular distributed training frameworks in
SINGA.</p>
-<p><img src="../images/distributed/frameworks.png" style="width: 800px" alt=""
/>
+<p><img src="http://singa.incubator.apache.org/assets/image/frameworks.png";
style="width: 800px" alt="" />
<p><b> Fig.1 - Training frameworks in SINGA</b></p>
<div class="section">
<h4><a name="Sandblaster"></a>Sandblaster</h4>
Modified: websites/staging/singa/trunk/content/docs/installation.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/installation.html (original)
+++ websites/staging/singa/trunk/content/docs/installation.html Wed Sep 2
10:31:57 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-17
+ | Generated by Apache Maven Doxia at 2015-09-02
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150817" />
+ <meta name="Date-Revision-yyyymmdd" content="20150902" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Installation</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -423,27 +423,23 @@
<div id="bodyColumn" class="span10" >
- <div class="section">
-<h2><a name="Installation"></a>Installation</h2>
-<hr />
+ <h1>Installation</h1>
+<div class="section">
<div class="section">
<h3><a name="Dependencies"></a>Dependencies</h3>
-<p>SINGA is developed and tested on Linux platforms with the following
external libraries.</p>
+<p>SINGA is developed and tested on Linux platforms.</p>
<p>The following dependenies are required:</p>
<ul>
<li>
-<p>gflags version 2.1.1, use the default setting for namespace (i.e.,
gflags).</p></li>
-
-<li>
-<p>glog version 0.3.3.</p></li>
+<p>glog version 0.3.3</p></li>
<li>
-<p>google-protobuf version 2.6.0.</p></li>
+<p>google-protobuf version 2.6.0</p></li>
<li>
-<p>openblas version >= 0.2.10.</p></li>
+<p>openblas version >= 0.2.10</p></li>
<li>
<p>zeromq version >= 3.2</p></li>
@@ -459,49 +455,79 @@
<ul>
<li>
-<p>gtest version 1.7.0.</p></li>
+<p>gtest version 1.7.0</p></li>
<li>
-<p>opencv version 2.4.9.</p></li>
+<p>opencv version 2.4.9</p></li>
<li>
<p>lmdb version 0.9.10</p></li>
</ul>
-<p>Tips: For libraries like openblas, opencv, older versions may also work,
because we do not use any newly added features.</p></div>
+<p>SINGA comes with a script for installing the external libraries (see
below).</p></div>
<div class="section">
-<h3><a name="Building_SINGA_From_Source"></a>Building SINGA From Source</h3>
-<p>The build system of SINGA is based on GNU autotools. To build singa, you
need gcc version >= 4.8. The common steps to build SINGA can be:</p>
+<h3><a name="Building_SINGA_from_source"></a>Building SINGA from source</h3>
+<p>SINGA is built using GNU autotools. GCC (version >= 4.8) is required.
There are three ways to build SINGA,</p>
+<ul>
+
+<li>
+<p>If you want to use the latest code, please clone it from <a
class="externalLink"
href="https://github.com/apache/incubator-singa.git";>Github</a> and execute
the following commands,</p>
+
<div class="source">
-<div class="source"><pre class="prettyprint">1.Extract source files;
-2.Run configure script to generate makefiles;
-3.Build and install SINGA.
-</pre></div></div>
-<p>On Unix-like systems with GNU Make as build tool, these build steps can be
summarized by the following sequence of commands executed in a shell.</p>
+<div class="source"><pre class="prettyprint">$ git clone
[email protected]:apache/incubator-singa.git
+$ cd incubator-singa
+$ ./autogen.sh
+$ ./configure
+$ make
+</pre></div></div></li>
+</ul>
+<ul>
+
+<li>
+<p>If you download a release package, please follow the instructions below,</p>
+
<div class="source">
-<div class="source"><pre class="prettyprint">$ cd SINGA/FOLDER
+<div class="source"><pre class="prettyprint">$ tar xvf singa-xxx
+$ cd singa-xxx
$ ./configure
$ make
-$ make install
</pre></div></div>
-<p>After executing above commands, SINGA library will be installed in the
system default directory. If you want to specify your own installation
directory, use the following command instead.</p>
+<p>Some features of SINGA depend on external libraries. These features can be
compiled with <tt>--enable-<feature></tt>. For example, to build SINGA
with lmdb support, you can run:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ ./configure --enable-lmdb
+</pre></div></div></li>
+</ul>
+<ul>
+
+<li>
+<p>In case you do not have the GNU auto tools to run <tt>autogen.sh</tt>,
SINGA provides a Makefile.example file, which is used as</p>
+
<div class="source">
-<div class="source"><pre class="prettyprint">$ ./configure
--prefix=/YOUR/OWN/FOLDER
+<div class="source"><pre class="prettyprint">$ cp Makefile.example Makefile
+$ make
</pre></div></div>
-<p>The result of configure script will indicate you whether there exist
dependency missings in your system. If you do not install the dependencies, you
can run the following commands. To download & install the thirdparty
dependencies:</p>
+<p>Code depending on lmdb can be added into the compilation by</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">make -DUSE_LMDB
+</pre></div></div></li>
+</ul>
+<p>After compiling SINGA successfully, the <tt>libsinga.so</tt> will be
generated into .lib/ folder and an executable file <tt>singa</tt> is generated
under bin/.</p>
+<p>If some dependent libraries are missing (or not detected), you can use the
following script to download and install them:</p>
<div class="source">
<div class="source"><pre class="prettyprint">$ cd thirdparty
$ ./install.sh MISSING_LIBRARY_NAME1 YOUR_INSTALL_PATH1 MISSING_LIBRARY_NAME2
YOUR_INSTALL_PATH2 ...
</pre></div></div>
-<p>If you do not specify the installation path, the library will be installed
in default folder. For example, if you want to build zeromq library in system
folder and gflags in /usr/local, just run:</p>
+<p>If you do not specify the installation path, the library will be installed
in the default folder specified by the software itself. For example, if you
want to build <tt>zeromq</tt> library in system folder and <tt>gflags</tt> in
<tt>/usr/local</tt>, just run:</p>
<div class="source">
<div class="source"><pre class="prettyprint">$ ./install.sh zeromq gflags
/usr/local
</pre></div></div>
-<p>Another example can be to install all dependencies in /usr/local
directory:</p>
+<p>You can also install all dependencies in <tt>/usr/local</tt> directory:</p>
<div class="source">
<div class="source"><pre class="prettyprint">$ ./install.sh all /usr/local
@@ -521,34 +547,34 @@ protobuf Google protobuf
zeromq zeromq lib
zookeeper Apache zookeeper
</pre></div></div>
-<p>*: Since czmq depends on zeromq, the script offers you one more argument to
indicate zeromq location. The installation commands of czmq can be:</p>
+<p>*: Since <tt>czmq</tt> depends on <tt>zeromq</tt>, the script offers you
one more argument to indicate <tt>zeromq</tt> location. The installation
commands of <tt>czmq</tt> is:</p>
<div class="source">
<div class="source"><pre class="prettyprint">$./install.sh czmq /usr/local
/usr/local/zeromq
</pre></div></div>
-<p>After the execution, czmq will be installed in /usr/local while zeromq is
installed in /usr/local/zeromq.</p></div>
+<p>After the execution, <tt>czmq</tt> will be installed in <tt>/usr/local</tt>
while <tt>zeromq</tt> is installed in <tt>/usr/local/zeromq</tt>.</p></div>
<div class="section">
<h3><a name="FAQ"></a>FAQ</h3>
-<p>Q1:While compiling Singa and installing glog on max OS X, I get fatal error
“‘ext/slist’ file not found”</p>
-<p>A1:You may install glog individually and try command :</p>
+<p>Q1:While compiling SINGA and installing <tt>glog</tt> on max OS X, I get
fatal error <tt>'ext/slist' file not found</tt></p>
+<p>A1:Please install <tt>glog</tt> individually and try :</p>
<div class="source">
<div class="source"><pre class="prettyprint">$ make CFLAGS='-stdlib=libstdc++'
CXXFLAGS='stdlib=libstdc++'
</pre></div></div>
-<p>Q2:While compiling Singa, I get error “SSE2 instruction set not
enabled”</p>
+<p>Q2:While compiling SINGA, I get error <tt>SSE2 instruction set not
enabled</tt></p>
<p>A2:You can try following command:</p>
<div class="source">
<div class="source"><pre class="prettyprint">$ make CFLAGS='-msse2'
CXXFLAGS='-msse2'
</pre></div></div>
-<p>Q3:I get error “./configure –> cannot find blas_segmm()
function” even I run “install.sh OpenBLAS”.</p>
-<p>A3:Since OpenBLAS library is installed in /opt folder by default or
/other/folder by your preference, you may edit your environment settings. You
need add its default installation directories before linking, just run:</p>
+<p>Q3:I get error <tt>./configure --> cannot find blas_segmm()
function</tt> even I run <tt>install.sh OpenBLAS</tt>.</p>
+<p>A3:Since <tt>OpenBLAS</tt> library is installed in <tt>/opt</tt> folder by
default or <tt>/other/folder</tt> by your preference, you may edit your
environment settings. You need add its default installation directories before
linking, just run:</p>
<div class="source">
<div class="source"><pre class="prettyprint">$ export LDFLAGS=-L/opt
</pre></div></div>
<p>Or as an alternative option, you can also edit LIBRARY_PATH to figure it
out.</p>
-<p>Q4:I get ImportError from google.protobuf.internal when I try to import .py
files. (ImportError: cannot import name enum_type_wrapper)</p>
+<p>Q4:I get <tt>ImportError: cannot import name enum_type_wrapper</tt> from
google.protobuf.internal when I try to import .py files.</p>
<p>A4:After install google protobuf by “make install”, we should
install python runtime libraries. Go to protobuf source directory, run:</p>
<div class="source">
@@ -557,7 +583,7 @@ $ cd python
$ python setup.py build
$ python setup.py install
</pre></div></div>
-<p>You may need “sudo” when you try to install python runtime
libraries in system folder.</p></div></div>
+<p>You may need <tt>sudo</tt> when you try to install python runtime libraries
in the system folder.</p></div></div>
</div>
</div>
</div>
![http://singa.incubator.apache.org/assets/image/logical.png"](http://singa.incubator.apache.org/assets/image/logical.png"){kind=link}
![https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"](https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"){kind=link}
![http://incubator.apache.org/images/egg-logo.png"](http://incubator.apache.org/images/egg-logo.png"){kind=link}
![http://singa.incubator.apache.org/assets/image/frameworks.png"](http://singa.incubator.apache.org/assets/image/frameworks.png"){kind=link}