wangwei created SINGA-37:
----------------------------
Summary: Enable users to set parameter sharing in model
configuration
Key: SINGA-37
URL: https://issues.apache.org/jira/browse/SINGA-37
Project: Singa
Issue Type: New Feature
Reporter: wangwei
For some models (e.g., RBM, RNN), the parameters of some layers are shared
(tied). To enable this feature, SINGA needs to provide a configuration field
which indicates from whom to share the parameter.
In specific, we can add one field in the ParamProto:
{code}
// the name of a Param who owns the Param and shares its values with this Param.
optional string share_from;
{code}
The NeuralNet class with share the Params based on the share_from field. Params
only share values, i.e., data field. Everyone has its own gradient, history,
etc fields. SINGA will automatically average the gradients for the shared
Params before sending the update request.
This ticket will not consider the case in which the original (owner of a
shared) Param object is partitioned due to model partitioning, e.g., the weight
matrix is partitioned into two sub-matrix if the inner-product layer is
partitioned on dimension 1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
