Re: Invert large matrix

[Yanwei Wayne Zhang]

Thanks for the advice. I figured out a way to solve this problem by avoiding 
the matrix representation.


Wayne



From: Sean Owen 
Sent: Thursday, December 29, 2016 1:52 PM
To: Yanwei Wayne Zhang; user
Subject: Re: Invert large matrix

I think the best advice is: don't do that. If you're trying to solve a linear 
system, solve the linear system without explicitly constructing a matrix 
inverse. Is that what you mean?

On Thu, Dec 29, 2016 at 2:22 AM Yanwei Wayne Zhang 
mailto:actuary_zh...@hotmail.com>> wrote:

Hi all,


I have a matrix X stored as RDD[SparseVector] that is high dimensional, say 800 
million rows and 2 million columns, and more 95% of the entries are zero.

Is there a way to invert (X'X + eye) efficiently, where X' is the transpose of 
X and eye is the identity matrix? I am thinking of using RowMatrix but not sure 
if it is feasible.

Any suggestion is highly appreciated.


Thanks.


Wayne

Re: Invert large matrix

[Sean Owen]

I think the best advice is: don't do that. If you're trying to solve a
linear system, solve the linear system without explicitly constructing a
matrix inverse. Is that what you mean?

On Thu, Dec 29, 2016 at 2:22 AM Yanwei Wayne Zhang <
actuary_zh...@hotmail.com> wrote:

> Hi all,
>
>
> I have a matrix X stored as RDD[SparseVector] that is high dimensional,
> say 800 million rows and 2 million columns, and more 95% of the entries are
> zero.
>
> Is there a way to invert (X'X + eye) efficiently, where X' is the
> transpose of X and eye is the identity matrix? I am thinking of using
> RowMatrix but not sure if it is feasible.
>
> Any suggestion is highly appreciated.
>
>
> Thanks.
>
>
> Wayne
>
>
>