[
https://issues.apache.org/jira/browse/SINGA-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698171#comment-14698171
]
ASF subversion and git services commented on SINGA-58:
------------------------------------------------------
Commit fcd377aed543f5a44deeb3145551b107e6cc2324 in incubator-singa's branch
refs/heads/master from Wei Wang
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=fcd377a ]
SINGA-58 Fix fan-in dimension of weight matrix
Use the num of cols of a weight matrix as its fan-in.
Layer that have weight matrix should follow this assumption.
Otherwise, if there would be errors when the weight matrix is initialized based
on fan-in.
> Fix fan-in dimension of weight matrix
> -------------------------------------
>
> Key: SINGA-58
> URL: https://issues.apache.org/jira/browse/SINGA-58
> Project: Singa
> Issue Type: Improvement
> Reporter: wangwei
>
> There are many layers using weight matrix, e.g., convolution layer and
> innerproduct layer. Some initialization methods of the weight matrix are
> based on the fan-in of the matrix, i.e., the dimension interacts with the
> other operator. E.g., for v*W, W's fan-in is the number of rows; For v*W^T,
> W's fan-in is the number of columns.
> However, the Param::InitValues() method does not know the position (and
> transposition) of the matrix in the multiplication operation. We have to fix
> the fan-in dimension. Particularly, we fix it to the second dimension (i.e.,
> number of columns) and assume users use the weight matrix correctly.
> The current implementation of convolution layer and innerproduct layer have
> different fan-in dimension, which should be unified to the second dimension.
> For the convolution layer, its operation is W*v. W's fan-in is already the
> number of columns. For the innerproduct layer, the current operation is v*W,
> which should be updated to v*W^T, then the fan-in is the number of columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
