Thanks Matthias! It will be great for passing the model to the paramserv function.
Regards, Guobao 2018-05-10 21:47 GMT+02:00 Matthias Boehm <mboe...@gmail.com>: > just FYI: we now have support for list and named-list data types in > SystemML, which allow passing the entire model as a single handle. For > example, you can define the following > > l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or > l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4) > > and access entries via l1[7] or l2['g'] accordingly. We're still > working on additional features to make the integration with IPA, > functions, and size/type propagation smoother, but the basic > functionality is already available. > > Regards, > Matthias > > On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm <mboe...@gmail.com> wrote: > > Hi Guobao, > > > > that sounds very good. In general, the "model" refers to the > > collection of all weights and bias matrices of a given architecture. > > Similar to a classic regression model, we can view the weights as the > > "slope", i.e., multiplicative terms, while the biases are the > > "intercept", i.e., additive terms that shift the layer output. Both > > are subject to training and thus part of the model. > > > > This implies that the number of matrices in the model depends on the > > architecture. Hence, we have two choices here: (a) allow for a > > variable number of inputs and outputs, or (b) create a struct-like > > data type that allows passing the collection of matrices via a single > > handle. We've discussed the second option in other contexts as well > > because this would also be useful for reducing the number of > > parameters passed through function calls. I'm happy to help out > > integrating these struct-like data types if needed. > > > > Great to see that you're in the process of updating the related JIRAs. > > Let us know whenever you think you're ready with an initial draft - > > then I'd make a detailed pass over it. > > > > Furthermore, I would recommend to experiment with running these > > existing mnist lenet examples (which is one of our baselines moving > > forward): > > * Download the "infinite MNIST" data generator > > (http://leon.bottou.org/projects/infimnist), and generate a moderately > > sized dataset (e.g., 256K instances). > > * Convert the input into SystemML's binary block format. The generator > > produces the data in libsvm format and we provide a data converter > > (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our > > internal binary representation. > > * Run the basic mnist lenet example for a few epochs. > > * Install the native BLAS libraries mkl or openblas and try using it > > for the above example to ensure its setup and configured correctly. > > > > > > Regards, > > Matthias > > > > On Sun, May 6, 2018 at 3:24 AM, Guobao Li <guobao1993...@gmail.com> > wrote: > >> Hi Matthias, > >> > >> I'm currently reading the dml script MNIST LeNet example and got some > >> questions. I hope that you could help me out of them. > >> > >> 1) Is it possible to define a matrix containing the variables? Because > I'm > >> wondering how to represent the model as a parameter for the "paramserv" > >> function. > >> 2) What is the role of bias? Why we need it? > >> > >> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and > hope > >> to get some feedback. Thanks! > >> > >> Regards, > >> Guobao > >> >
