wangwei created SINGA-126:
-----------------------------
Summary: Improve Python Binding for interactive training
Key: SINGA-126
URL: https://issues.apache.org/jira/browse/SINGA-126
Project: Singa
Issue Type: Improvement
Reporter: wangwei
Currently, python APIs only configure the layer and model. All objects are
created after the the JobProto is passed to Driver. Hence, users cannot query
the layer object returned by
{code}
conv1 = Convolution2D()
{code}
to get its internal data (e.g, feature and param values). These internal data
is useful for debugging.
To support this feature, we need to create the SINGA::Layer object and store it
in conv1.
Users can write their own BP algorithm like this,
{code}
data = numpy.loadtxt("csv.txt")
x, y = data[:, 1:], data[:, 0]
input = Dummy() // dummy layer to get input data
label = Dummy() // dummy layer to get label
conv = Convolution2D(...)
pool = Pool2D()
inner = Dense()
loss = ...
for i in range(x.shape[0] / batchsize):
xb, yb = ...
input.SetData(x)
label.SetData(y)
conv.ComputeFeature(input)
pool.ComputeFeature(conv)
inner.ComputeFeature(pool)
loss.ComputeGradient(inner, label)
....
{code}
In this way, users know exactly how the training is conducted, and can access
the internal data of each layer directly, e.g., conv.data(), conv.GetParams().
We may also learn from chainer to call the ComputeGradient functions
automatically for the backward pass.
This feature requires the python APIs for singa::Layer.
It is easy for training with a single worker. For multiple workers, we need to
think more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
