a. Are you solving for X or for Y? b. In addition to knowing that there are only around 100 non-zero entries per row, do you know anything more about the matrix A? Is it a band matrix, for instance?
See in the dictionary the entry for $., in particular http://www.jsoftware.com/help/dictionary/d211.htm#sparse%20linear%20algebra However neither +/ .* nor %. in the public version of J are up to dealing with the matrix A you described. c. With arrays of this size you should avoid CSV or anything that requires repeated conversions into 4- or 8-byte machine integers and 8-byte machine floating point numbers. ----- Original Message ----- From: Nick Kostirya <[EMAIL PROTECTED]> Date: Thursday, January 3, 2008 5:22 Subject: Re: [Jgeneral] Successful stories To: General forum <[email protected]> > В Thu, 3 Jan 2008 17:58:59 +0800 > "Alex Rufon" <[EMAIL PROTECTED]> пишет: > > > ... > > > > Nick. Just tell us what you think of doing. Maybe we can help. There > > are a lot of brilliant people here on the list and maybe we > can help > > thing move along for you. ;) > > Thanks a lot. > > I have two goals. > > The first one adds up to doing a giant system of linear equations. > The matrix A in the equation Y=A*X is rather sparse, but large. > The size of matrix A is 100 million to 100 million, however, the > matrixrow will only have around 100 cells with the values > different from zero. > > The description of this matrix is located in the file with 3 columns. > The first two columns contain the cell's coordinates, the third column > contains the value. That is, the file contains 10 billion rows. I > haven't decided yet what format is better to store the data in one > file. It can be either fixed length rows, or CSV. What is better from > the J viewpoint ? > > That said, the task consists of the following: > 1) read the file and generate the matrix > 2) do a system of linear equations > 3) save the result > > In a word, it's simple , but for the newbie in J it's hard to define > the best way for now. Say, the process of "Connection matrix" > generation described here > http://www.jsoftware.com/help/dictionary/samp20.htm doesn't seem quite > optimal to me for the above task solution. > > The second task is connected to the factor analysis, however, > there are > less data here. So, I believe upon the first task solution I'll > acquirethe experience needed to solve the second one. ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
