Question about MNIST LeNet example

[Guobao Li]

 Hi Matthias,

I'm currently reading the dml script MNIST LeNet example and got some
questions. I hope that you could help me out of them.

1) Is it possible to define a matrix containing the variables? Because I'm
wondering how to represent the model as a parameter for the "paramserv"
function.
2) What is the role of bias? Why we need it?

Additionally, I'm updating the JIRA for SYSTEMML-2083 and hoping to get
some feedback. Thanks!

Regards,
Guobao

Re: Questions about MNIST LeNet example

[Matthias Boehm]

Hi Guobao,

that sounds very good. In general, the "model" refers to the
collection of all weights and bias matrices of a given architecture.
Similar to a classic regression model, we can view the weights as the
"slope", i.e., multiplicative terms, while the biases are the
"intercept", i.e., additive terms that shift the layer output. Both
are subject to training and thus part of the model.

This implies that the number of matrices in the model depends on the
architecture. Hence, we have two choices here: (a) allow for a
variable number of inputs and outputs, or (b) create a struct-like
data type that allows passing the collection of matrices via a single
handle. We've discussed the second option in other contexts as well
because this would also be useful for reducing the number of
parameters passed through function calls. I'm happy to help out
integrating these struct-like data types if needed.

Great to see that you're in the process of updating the related JIRAs.
Let us know whenever you think you're ready with an initial draft -
then I'd make a detailed pass over it.

Furthermore, I would recommend to experiment with running these
existing mnist lenet examples (which is one of our baselines moving
forward):
* Download the "infinite MNIST" data generator
(http://leon.bottou.org/projects/infimnist), and generate a moderately
sized dataset (e.g., 256K instances).
* Convert the input into SystemML's binary block format. The generator
produces the data in libsvm format and we provide a data converter
(see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our
internal binary representation.
* Run the basic mnist lenet example for a few epochs.
* Install the native BLAS libraries mkl or openblas and try using it
for the above example to ensure its setup and configured correctly.


Regards,
Matthias

On Sun, May 6, 2018 at 3:24 AM, Guobao Li  wrote:
> Hi Matthias,
>
> I'm currently reading the dml script MNIST LeNet example and got some
> questions. I hope that you could help me out of them.
>
> 1) Is it possible to define a matrix containing the variables? Because I'm
> wondering how to represent the model as a parameter for the "paramserv"
> function.
> 2) What is the role of bias? Why we need it?
>
> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope
> to get some feedback. Thanks!
>
> Regards,
> Guobao
>

Questions about MNIST LeNet example

[Guobao Li]

 Hi Matthias,

I'm currently reading the dml script MNIST LeNet example and got some
questions. I hope that you could help me out of them.

1) Is it possible to define a matrix containing the variables? Because I'm
wondering how to represent the model as a parameter for the "paramserv"
function.
2) What is the role of bias? Why we need it?

Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope
to get some feedback. Thanks!

Regards,
Guobao