Hi Christian, it's nice to hear back from you on the list :) The backprop algorithm is something that I'd be very happy to see implemented here, and also I've spent some time myself on it some months ago but didn't manage to finalize the implementation so far.
Your approach sounds reasonable, I don't have read the paper pointed by Edward (thanks!) yet but it may help us evaluate how to split things. Regarding deep learning that's surely something interesting we should keep an eye on, maybe we can start from Christian's proposal, implement that and maybe move to the DL if/when we have something ready. Thanks and have a nice day, Tommaso p.s.: Regarding the matrix library my opinion is that, for starting, we should try to use something that just works (I don't know Colt so I can't say) in order to go straight to the algorithm itself but for the mid / long term I'd also prefer to use an own matrix multiplication / inverse / etc. just because that would be useful also for other tasks. 2012/11/15 [email protected] <[email protected]> > Dear All, > what do you think about to scale out the learning of Multi Layer > Perceptrons > (MLP) with BSPs? > I heard the talk of Tommaso at the apacheConAt first glance the pramming > model > BSP seems to fit better the MapReduce for this purpose. > The basic idea is to distribute the backprop algorithm is the following: > > Distribution of learning can be done by (batch learning): > 1 Partioning of the data in x chunks > 2 On each working node: Learning the weight changes (as matrices) in each > chunk > 3 Combining the matrixes (weight changes) and simultaneous update of the > weights > in each node - back to 2 > > Maybe this procedure can be done with random parts of the chunks > (distributed > quasi online learning). > > I wrote the (basic) backprob algorithm of a multi layer preceptron (see > mahout > patch https://issues.apache.org/jira/browse/MAHOUT-976). It uses the > Mahout > Matrix Library, which is under the hood the Colt Matrix Library from Cern. > > Probably using the Cern Matrix Library would also suitable for Hama. Then > it > could be easy to port the MLP to Hama. > > What do you think about it? > > Thanks for you response. > > Cheers > Christian
