[
https://issues.apache.org/jira/browse/SINGA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022788#comment-16022788
]
ASF subversion and git services commented on SINGA-315:
-------------------------------------------------------
Commit ea078dca9cfafdda9d5a15ae2f2823897d217292 in incubator-singa's branch
refs/heads/master from wangwei
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=ea078dc ]
SINGA-315 Reduce memory footprint by Python generator for parameter gradient
Update the API of net::backward() function.
1. add arguments dy and output. dy for the input gradient tensor(s), e.g. from
the loss functions.
output is a list of layer names, whose output gradient tensor(s) would be
returned in addition to the param gradient tensor(s).
2. returnes a generator iterator that generates (param_names, param_values,
param_grads, out_grads) after processing each layer.
The callee function can update the parameters and release the gradient tensors
in layerwise.
> Reduce memory footprint by Python generator for parameter gradient
> ------------------------------------------------------------------
>
> Key: SINGA-315
> URL: https://issues.apache.org/jira/browse/SINGA-315
> Project: Singa
> Issue Type: New Feature
> Reporter: wangwei
>
> The parameter gradient tensors are stored in memory until BP is finished.
> This is not necessary as we can update the parameter once its gradient is
> ready. Then we free the gradient tensor.
> In this way, we reduce the memory footprint by avoiding store all gradient
> tensors.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
