Re: svd( ) implementation

2017-07-28 Thread Janardhan Pulivarthi
Thanks, Imran. As per the paper, at first we perform QR decomposition of input matrix (A), from which we obtain R. And then, we compute the svd(R) using the builtin local function (??). I'll try this. Tall-skinny matrix: so, do we have problem with square matrices?. or do we have to partition the

Re: Issue with Clustering : Predict

2017-07-28 Thread Krishna Kalyan
Thanks for the quick fix Matthias. Cheers, Krishna On Thu, Jul 27, 2017 at 12:50 AM, Matthias Boehm wrote: > thanks for catching this issue Krishna - I just pushed a hot fix for it to > master. > > Regards, > Matthias > > On Wed, Jul 26, 2017 at 7:58 PM, Krishna Kalyan > wrote: > > > Hello All

Re: Matrix non-range indexing should return a scalar

2017-07-28 Thread Matthias Boehm
Thanks for bringing this up Mike - this is a useful discussion. Let us first clarify the R semantics. In R, any scalar/vector matrix indexing gives a numeric vector because there are no scalars but vectors of length 1. So Nakul, R does not behave like this proposal. If I remember correctly, Matlab

Re: Update Spark Configuration to improve SystemML performance

2017-07-28 Thread Mike Dusenberry
Awesome! We certainly welcome new contributors! Just to highlight a few areas, in terms of ML/DL tasks, SYSTEMML-540 [1] covers all of our work related to deep learning, SYSTEMML-618 [2] covers our work within that to create a DML deep learning library ("nn"), SYSTEMML-1479 [3] covers work to be

Re: Matrix non-range indexing should return a scalar

2017-07-28 Thread Imran Younus
+1 Numpy also behaves the way Mike is suggesting here. imran On Fri, Jul 28, 2017 at 5:21 PM, Nakul Jindal wrote: > +1 to Mike & Deron. > > Two other languages/packages that behave like this: > R : http://www.r-tutor.com/r-introduction/matrix > Octave : > https://www.gnu.org/software/octave/do

Re: Matrix non-range indexing should return a scalar

2017-07-28 Thread Nakul Jindal
+1 to Mike & Deron. Two other languages/packages that behave like this: R : http://www.r-tutor.com/r-introduction/matrix Octave : https://www.gnu.org/software/octave/doc/interpreter/Index-Expressions.html -Nakul On Fri, Jul 28, 2017 at 4:03 PM, Deron Eriksson wrote: > Thank you Mike for br

Re: Matrix non-range indexing should return a scalar

2017-07-28 Thread Deron Eriksson
Thank you Mike for bringing this up. To me, this definitely makes sense at the user (DML) level. For a Java-style pseudocode example, currently we require the user to do the following: int[][] m = int[][]{1,2,3,4}; int[][] n = m[0][0]; int x = (int) n; I feel the following would be more 'na

Matrix non-range indexing should return a scalar

2017-07-28 Thread dusenberrymw
Currently, non-range matrix indexing, such as `X[1,2]`, returns a 1x1 matrix in SystemML rather than a single scalar value. This is inconsistent with mathematical semantics, and with array indexing semantics of any major language, thus leading to confusion for users. I would like to propose th

Re: svd( ) implementation

2017-07-28 Thread Imran Younus
Just to clarify one thing. For QR based, method, you can assume that R matrix is small enough to fit on driver memory and them perform SVD on the driver. That means your actual matrix has to tall-skinny matrix. imran On Fri, Jul 28, 2017 at 11:15 AM, Imran Younus wrote: > Janardhan, > > The pap

Re: svd( ) implementation

2017-07-28 Thread Imran Younus
Janardhan, The papers you're referring may not be relevant. The first paper, as far as I can tell, is about updating an existing svd decomposition as new data comes in. The 3rd paper in this list is the one I used, but that method is not good. There is also a method that uses QR decomposition and

Re: svd( ) implementation

2017-07-28 Thread Janardhan Pulivarthi
Hi Nakul & all the committers, Till now I am half way through the literature. But, for now a couple of things to mention, in SVD there are three stages 1. Bidiagonal reduction step 2. Computation of the singular values 3. Computation of the singular vectors of these three, The* Bidiagonal r