Yes, what you have described is how the primitive ip=: +/ .* (inner product) is implemented on sparse arrays. I assume that the data would already be sorted by i and within that by j . It is the times when you need to be sorted by j then i that things get interesting.
If you need to implement ip for this project, take a look at the description on the vocabulary page for "." http://www.jsoftware.com/help/dictionary/d300.htm For +/ .* , the u@(v"(1+lv,_)) description simplifies to +/@(*"1 _) . This is more efficient than the traditional "row on the left vs. column on the right" formulation, and underlies the improvement described in http://www.jsoftware.com/help/release/pdt504.htm I assume you will be using the 64-bit version of J. You will have enough other things to worry about without running out of address space on vectors of length 10e9. It is good to have your original description of the problem y = A ip x , solving for x, because often it is easy to lose sight of the main goal in all the nasty details that come with dealing with a 1e8 1e8 matrix having 10e9 non-zero entries. p.s. Would the power method work? Then all that's needed is repeated inner products and you have already described how to do inner product in your msg. ----- Original Message ----- From: Nick Kostirya <[EMAIL PROTECTED]> Date: Thursday, January 10, 2008 2:03 Subject: Re: [Jgeneral] table (i,j,value) to matrix To: General forum <[email protected]> Cc: [EMAIL PROTECTED] > Thank you so much for your help! > > First I defined the size of matrix by i,.j - everything works > just fine. > > But an idea occured to my mind... Maybe I am not at the right > point when using the matrix formulation when solving my task? > > If you: > > a) sort the table T(i, j, v) by i value; > > b) find the beginning and the end of the ranges for each unique > value i (let's call them u); > > c) create a loop by u, and pull the necessary ranges with j and > v from the T table; > > d) apply necessary calculations to the data which are pulled > from the other places by j value. > > Suppose in case we arrange the things this way - this will not > be much slower > than the matrix formulation method, am I right? > > Cheers. > > > > > a. Currently, mapped files do not support the sparse datatypes. > > > > b. You can do this in one of two ways: > > s=: 1 $. 1e8 1e8 ; 0 1;-~0.5 > > s=: value (<"1 i,.j)}s > > Or > > s=: 1 $. 1e8 1e8 ; 0 1; (-~0.5) ; (i,.j) ; value > > (The latter is not yet documented but should work in > > J602 beta.) > > > > A sparse matrix can be converted to the equivalent dense > > matrix by d=: $.^:_1 s . > > > > Note that with your full problem, a 1e8 1e8 matrix with 10e9 > > non-zero atoms, there is no advantage to making it a sparse > > matrix, because as stated before the primitive %. and > > +/ .* are not up to dealing with an array of that size. > > Better to just map i and j and values separately > > (perhaps i,.j and value as two map files), and apply your > > iterative methods to these. > > > > Perhaps I should add the problem you described is a > > non-trivial application. > > > > > > > > ----- Original Message ----- > > From: Nick Kostirya <[EMAIL PROTECTED]> > > Date: Wednesday, January 9, 2008 23:04 > > Subject: [Jgeneral] table (i,j,value) to matrix > > To: [email protected] > > > > > Hello, > > > > > > I proceed with my studies of J, well I have a couple of > > > questions for > > > you. > > > > > > As far as I understand, а sparse matrix can't mapped as > file, am > > > I right? > > > > > > Second, what's the most optimal way of converting a table > > > (i,j,value) into matrix, preferably a sparse one? Can't > really get > > > the clue... I am also interested in the process of reverse > > > conversion... ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
