Hi Nakul, I really don't like the fact that eigen, lu, qr, cholesky only have local implementation and we have qualified them to the status of builtin functions. We should definitely consider the option of implementing a SPARK instructions for them (as you mentioned in the email) before we officially mark them to "local_only". In fact, instead of marking them as "local_only", I would much rather prefer to support them as external builtin functions.
Thanks, Niketan Pansare IBM Almaden Research Center E-mail: npansar At us.ibm.com http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar From: Deron Eriksson <deroneriks...@gmail.com> To: dev@systemml.incubator.apache.org Date: 10/21/2016 12:52 PM Subject: Re: Local versions of Linear Algebra Operators in DML Hi Nakul, +1 I think having some clear characteristic to distinguish operations that only operate locally is a great idea. Otherwise, how would a user know that these operations are only local and not distributed? Adding this naming convention for local operations sounds reasonable to me so that we don't anger users who expect an operation to be distributed when in actuality it only currently runs locally. Deron On Fri, Oct 21, 2016 at 12:34 PM, Nakul Jindal <naku...@gmail.com> wrote: > Hi, > > Imran was planning on implementing a distributed SVD as a DML bodied > function. > The algorithm is described in the paper titled "A Distributed and > Incremental SVD Algorithm for Agglomerative Data Analysis on Large > Networks" available at https://arxiv.org/abs/1601.07010. > > This algorithm requires the availability of a local SVD function, which we > currently do not have in SystemML. > Seeing as how there are other linear algebra functions (eigen, lu, qr, > cholesky) in DML that reroute to Apache Common Math and only operate in > standalone/CP mode, would it be ok to add "svd" to this set? > > Also, since these operations are local and not distributed and the > documentation doesn't make it clear that these operations wont operate in > distributed mode, would it make sense to rename them to "local_eigen", > "local_qr", "local_cholesky", etc? > Obviously, this change would go into the version after 0.11. > > I understand that the ideal solution to this problem is to have a > distributed version of the aforementioned linear algebra routines, but for > the time being, would it be ok to go ahead do the rename, while also > introducing a "local_svd" ? > > > Niketan, Berthold, Matthias, Sasha - Any thoughts? > > Thanks, > Nakul Jindal >
