Author: wangwei
Date: Mon Oct 12 04:07:02 2015
New Revision: 1708035
URL: http://svn.apache.org/viewvc?rev=1708035&view=rev
Log:
update docs for output layers
Modified:
incubator/singa/site/trunk/content/markdown/docs/layer.md
Modified: incubator/singa/site/trunk/content/markdown/docs/layer.md
URL:
http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/layer.md?rev=1708035&r1=1708034&r2=1708035&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/layer.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/layer.md Mon Oct 12
04:07:02 2015
@@ -52,7 +52,7 @@ These layers are categorized according t
* Input layers for loading records (e.g., images) from disk files, HDFS or
network into memory.
* Neuron layers for feature transformation, e.g.,
[convolution](../api/classsinga_1_1ConvolutionLayer.html),
[pooling](../api/classsinga_1_1PoolingLayer.html), dropout, etc.
* Loss layers for measuring the training objective loss, e.g., Cross Entropy
loss or Euclidean loss.
- * Output layers for outputting the prediction results (e.g., probabilities
of each category), e.g., onto disk.
+ * Output layers for outputting the prediction results (e.g., probabilities
of each category) or features into persistent storage, e.g., disk or HDFS.
* Connection layers for connecting layers when the neural net is partitioned.
#### Input layers
@@ -95,33 +95,33 @@ It may do some preprocessing like [stand
The data for preprocessing is loaded by and parsed in a virtual function,
which is implemented by
its subclasses.
-#### ProtoRecordInputLayer
+##### RecordInputLayer
It is a subclass of SingleLabelRecordLayer. It parses the value field from one
-tuple into a SingleLabelImageRecord, which is generated by Google Protobuf
according
+tuple into a RecordProto, which is generated by Google Protobuf according
to common.proto. It can be used to store features for images (e.g., using the
pixel field)
-or other objects (using the data field). The key field is not used.
+or other objects (using the data field). The key field is not parsed.
- type: kProtoRecordInput
+ type: kRecordInput
store_conf {
has_label: # default is true
...
}
-#### CSVRecordInputLayer
+##### CSVInputLayer
It is a subclass of SingleLabelRecordLayer. The value field from one tuple is
parsed
as a CSV line (separated by comma). The first number would be parsed as a
label if
`has_label` is configured in `store_conf`. Otherwise, all numbers would be
parsed
into one row of the `data_` Blob.
- type: kCSVRecordInput
+ type: kCSVInput
store_conf {
has_label: # default is true
...
}
-#### ImagePreprocessLayer
+##### ImagePreprocessLayer
This layer does image preprocessing, e.g., cropping, mirroring and scaling,
against
the data Blob from its source layer. It deprecates the RGBImageLayer which
@@ -215,7 +215,7 @@ the mean of each pixel over all training
meanfile: "Image_Mean_File_Path"
}
-#### PrefetchLayer
+##### PrefetchLayer
[PrefetchLayer](../api/classsinga_1_1PrefetchLayer.html) embeds other input
layers
to do data prefeching. It will launch a thread to call the embedded layers to
load and extract features.
@@ -248,6 +248,44 @@ The layers on top of the PrefetchLayer s
layers as their source layers. For example, the "rgb" and "label" should be
configured to the `srclayers` of other layers.
+
+#### Output Layers
+
+Output layers get data from their source layers and write them to persistent
storage,
+e.g., disk files or HDFS (to be supported).
+
+##### RecordOutputLayer
+
+This layer gets data (and label if it is available) from its source layer and
converts it into records of type
+RecordProto. Records are written as (key = instance No., value = serialized
record) tuples into Store, e.g., KVFile. The configuration of this layer
+should include the specifics of the Store backend via `store_conf`.
+
+ layer {
+ name: "output"
+ type: kRecordOutput
+ srclayers:
+ store_conf {
+ backend: "kvfile"
+ path:
+ }
+ }
+
+##### CSVOutputLayer
+This layer gets data (and label if it available) from its source layer and
converts it into
+a string per instance with fields separated by commas (i.e., CSV format). The
shape information
+is not kept in the string. All strings are written into
+Store, e.g., text file. The configuration of this layer should include the
specifics of the Store backend via `store_conf`.
+
+ layer {
+ name: "output"
+ type: kCSVOutput
+ srclayers:
+ store_conf {
+ backend: "textfile"
+ path:
+ }
+ }
+
#### Neuron Layers
Neuron layers conduct feature transformations.
