Re: Matrix inversion

. Kinda what i was > saying -- and in this case it doesn't seem that algorithm is as highly > interconnected as, e.g., naive blockwise multiplication. > > On Thu, May 5, 2016 at 1:50 PM, Dmitriy Lyubimov <dlie...@gmail.com> > wrote: > > > The mantra i keep hearing

Re: Matrix inversion

:50 PM, Dmitriy Lyubimov <dlie...@gmail.com> wrote: > The mantra i keep hearing is that if someone needs matrix inversion then > he/she must be doing something wrong. Not sure how true that is, but in all > cases i have encountered, people try to avoid matrix inversion one way or &g

Re: Matrix inversion

The mantra i keep hearing is that if someone needs matrix inversion then he/she must be doing something wrong. Not sure how true that is, but in all cases i have encountered, people try to avoid matrix inversion one way or another. Re: libraries: Mahout is more about apis now than any particular

Re: matrix inversion in plan ?

Unfortunately I do not know much details of these. The steps of these calculation is passed to me from a research team. I am helping them with coding part only. I myself is not good at math :-(  btw,  I think Mahout supports out-of-core SVD, am I correct ? If so, I can get inverse of matrix

Re: matrix inversion in plan ?

Go, if you'd like to put your research team in touch with the list we may be able to help work through a good approach; let us know. On Thu, Oct 8, 2015 at 3:58 PM, Dmitriy Lyubimov wrote: > Mahout translation (approximation, since ssvd is reduced-rank, not the true > thing):

Re: matrix inversion in plan ?

or pseudoinverse really, i guess On Thu, Oct 8, 2015 at 3:58 PM, Dmitriy Lyubimov wrote: > Mahout translation (approximation, since ssvd is reduced-rank, not the > true thing): > > val (drmU, drmV, s) = dssvd(drmA, k = 100) > val drmInvA = drmV %*% diagv(1 /=: s) %*% drmU.t >

Re: matrix inversion in plan ?

Yeah, nice trick Ted; here's a how-to for the list: http://www.cse.unr.edu/~bebis/CS791E/Notes/SVD.pdf On Thu, Oct 8, 2015 at 2:31 PM, Ted Dunning wrote: > Yes. You can get the inverse from an SVD or emulate its effect. > > Can you share the actual mathematical

Re: matrix inversion in plan ?

Totally an approximation; depends on why people are asking for the inverse and whether it'd do. On Thu, Oct 8, 2015 at 4:20 PM, Dmitriy Lyubimov wrote: > or pseudoinverse really, i guess > > On Thu, Oct 8, 2015 at 3:58 PM, Dmitriy Lyubimov > wrote: > > >

Re: matrix inversion in plan ?

Yes. You can get the inverse from an SVD or emulate its effect. Can you share the actual mathematical specification for your problem? If you can't, then there is little we can do to help. On Wed, Oct 7, 2015 at 11:35 PM, go canal wrote: > Unfortunately I do not

Re: matrix inversion in plan ?

Mahout translation (approximation, since ssvd is reduced-rank, not the true thing): val (drmU, drmV, s) = dssvd(drmA, k = 100) val drmInvA = drmV %*% diagv(1 /=: s) %*% drmU.t Still, technically, it is a right inverse as in reality m is rarely the same as n. Also, k must be k<= drmA.nrow min

Re: matrix inversion in plan ?

> > oh, it is so unfortunate that the first step of my project requires the > inversion of a very large matrix. will have to revert back to scalapack or > MR based solutions I guess. > > > thanks, canal > > > > > > > > >On Saturday, October 3, 201

Re: matrix inversion in plan ?

On Sun, Oct 4, 2015 at 10:32 PM, go canal wrote: > in fact i need to support both double and complex double for either > distributed memory based or out-of-core. Ahh... Well Mahout doesn't support complex anything. So this isn't going to help you.

Re: matrix inversion in plan ?

I will be more than interested to extend to complex double, when the solver is ready for double data type.  thanks, canal On Monday, October 5, 2015 2:02 PM, Ted Dunning wrote: On Sun, Oct 4, 2015 at 10:32 PM, go canal wrote: >

Re: matrix inversion in plan ?

That isn't enough detail. How do you mean to compute degrees of freedom? WHy do you need the inverse to do this? Where did you get this algorithm? Is this even appropriate at large scale? Is this a stable computation? On Sun, Oct 4, 2015 at 11:18 PM, go canal

Re: matrix inversion in plan ?

on of a very large matrix. will have to revert back to scalapack or MR based solutions I guess. > >  thanks, canal > > > > > >    On Saturday, October 3, 2015 11:31 PM, Ted Dunning < ted.dunn...@gmail.com> wrote: > > > > > > I doubt seriously that Samsara

Re: matrix inversion in plan ?

trix. will have to revert back to scalapack or > MR based solutions I guess. >>> thanks, canal >>> >>> >>> On Saturday, October 3, 2015 11:31 PM, Ted Dunning < > ted.dunn...@gmail.com> wrote: >>> >>> >>> I doubt serio

Re: matrix inversion in plan ?

roject requires the > inversion of a very large matrix. will have to revert back to scalapack or MR > based solutions I guess. > thanks, canal > > > On Saturday, October 3, 2015 11:31 PM, Ted Dunning > <ted.dunn...@gmail.com> wrote: > > > I doubt serio

Re: matrix inversion in plan ?

I guess. > > thanks, canal > > > > > > On Saturday, October 3, 2015 11:31 PM, Ted Dunning < ted.dunn...@gmail.com> wrote: > > > > > > I doubt seriously that Samsara will support matrix inversion per se. The > > problem is > > > >

Re: matrix inversion in plan ?

the > inversion of a very large matrix. will have to revert back to scalapack or > MR based solutions I guess. >>>  thanks, canal >>> >>> >>>    On Saturday, October 3, 2015 11:31 PM, Ted Dunning < > ted.dunn...@gmail.com> wrote: >>>

Re: matrix inversion in plan ?

I doubt seriously that Samsara will support matrix inversion per se. The problem is a) it densifies sparse matrices b) it is much more costly than solving a linear system Samsara is roughly memory based, but different back-ends will try to spill to disk if necessary. It is likely

Re: matrix inversion in plan ?

doubt seriously that Samsara will support matrix inversion per se. The problem is a) it densifies sparse matrices b) it is much more costly than solving a linear system Samsara is roughly memory based, but different back-ends will try to spill to disk if necessary.  It is likely that the res

Re: matrix inversion in plan ?

t; > > > > On Saturday, October 3, 2015 11:31 PM, Ted Dunning > > <ted.dunn...@gmail.com <javascript:;>> wrote: > > > > > > I doubt seriously that Samsara will support matrix inversion per se. > > The problem is > > > > a) it densifi

Re: Any utility to solve the matrix inversion in Map/Reduce Way

can try an academic package, such as PLASMA. (May beat MKL on matrix inversion, but still requires MKL for BLAS.) If you're going for the kill (performance-wise), you can try GPU acceleration with MAGMA. My guess is that MAGMA will invert a 20K x 20K matrix for you in the matter of seconds. On Mon

Re: Any utility to solve the matrix inversion in Map/Reduce Way

This is a relatively small matrix. You can decompose a matrix like that, at least approximately in a pretty short time. With stock R (has no BLAS acceleration), it takes 9ms to decompose a 100 x 100 dense matrix and 6.2 seconds to decompose a 1K x 1K dense matrix. Since this is so fast, there

Re: Any utility to solve the matrix inversion in Map/Reduce Way

Hi Koobas, I want the first one. Do you have any suggestions? Thank you, Colin On Fri, Jan 18, 2013 at 12:49 PM, Koobas koo...@gmail.com wrote: Martix inversion

Re: Any utility to solve the matrix inversion in Map/Reduce Way

Hi Sean, I start to realize that the full inverse may not be realistic. Matrix decomposition may be a better idea. But I want to have a try and to show it as negative example in Map/Reduce. Thank you, Colin On Fri, Jan 18, 2013 at 8:10 PM, Sean Owen sro...@gmail.com wrote: And, do you really

Re: Any utility to solve the matrix inversion in Map/Reduce Way

Hi Ted, Thank you for the valuable experience. Colin On Fri, Jan 18, 2013 at 7:58 PM, Ted Dunning ted.dunn...@gmail.com wrote: h O(n^2) cannot be scaled to arbitrary size n. Sparse systems with only k items on average per row can often be handled with o(n) com

Re: Any utility to solve the matrix inversion in Map/Reduce Way

Colin, I am more of an HPC guys. I am a Mahout noob myself. Are we talking about a dense matrix? What size? On Sun, Jan 20, 2013 at 9:34 PM, Colin Wang colin.bin.wang.mah...@gmail.com wrote: Hi Koobas, I want the first one. Do you have any suggestions? Thank you, Colin On Fri, Jan 18,

Re: Any utility to solve the matrix inversion in Map/Reduce Way

Left unsaid in this comment is the fact that matrix inversion of any sizable matrix is almost always a mistake because it is (a) inaccurate, (b) slow. In scalable numerics it is also commonly true that the only really scalable problems are sparse. The reason for that is that systems whose cost

Re: Any utility to solve the matrix inversion in Map/Reduce Way

: Left unsaid in this comment is the fact that matrix inversion of any sizable matrix is almost always a mistake because it is (a) inaccurate, (b) slow. In scalable numerics it is also commonly true that the only really scalable problems are sparse. The reason for that is that systems whose cost

Re: Any utility to solve the matrix inversion in Map/Reduce Way

Martix inversion, as in explicitly computing the inverse, e.g. computing variance / covariance, or matrix inversion, as in solving a linear system of equations? On Thu, Jan 17, 2013 at 7:49 PM, Colin Wang colin.bin.wang.mah...@gmail.com wrote: Hi All, I want to solve the matrix inversion