[
https://issues.apache.org/jira/browse/SYSTEMML-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006946#comment-16006946
]
Mike Dusenberry commented on SYSTEMML-1599:
-------------------------------------------
Thanks for bringing this up, [~niketanpansare]. This has been on my todo list
for a while now. Specifically, I'd like to create an {{init}} function in
{{utils.dml}} that accepts {{(rows, cols, type)}} that all of the layer
{{init}} functions call once they decide the number of rows & cols appropriate
for the their respective layers. Additionally, I'd like to add a few more
functions to {{utils.dml}} for each of the initialization types that the
{{utils::init}} function calls, such as {{normal()}}, {{uniform()}}, etc.
Also, in addition to the Caffe init types, we should also include the
initialization schemes from He et al., such as the He normal variant that is
used in all of the current layers. All together, that will basically give us
the union of the init schemes used by Caffe and Keras
(https://keras.io/initializers).
> Extend nn layers to support different initialization type
> ---------------------------------------------------------
>
> Key: SYSTEMML-1599
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1599
> Project: SystemML
> Issue Type: Sub-task
> Reporter: Niketan Pansare
>
> Caffe currently supports the users to configure different initialization
> types:
> 1. constant
> 2. uniform
> 3. gaussian
> 4. positive_unitball
> 5. xavier
> 6. msra
> 7. bilinear
> The init() function of the layer should accept `type` variable which can be
> passed by Caffe2DML.
> [~mwdus...@us.ibm.com] [~prithvi_r_s]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
