Thank you. Although the actual cause of my space problem was mixing sparse and dense arrays, the trick of creating a sparse array and then assigning into it will be useful to me. At first, I thought is would be complicated to figure out where to make the assignments since the non-sparse locations in the two arrays I am comparing are in different places.
I now see that is actually fairly straight forward. The goal is to create the sparse binary array ba =: sa1 <: sa2 by creating an empty ba with sparse element 1=1 and assigning a few 0's into it here and there. sa1 and sa2 are arrays of non-negative integers or floats with sparse element 0. All non-sparse elements are greater than 0. Since 0 <: 0, the sparse element of ba is 1. In addition, wherever sa1 is sparse, ba1 is 1 regardless of the value of sa2. Wherever sa1 has a non-0 value and sa2 does not, ba is 0. The only place the <: comparison actually has to be performed is the intersection of 4$.sa1 and 4$.sa2. This intersection should be small. -Michael -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fraser Jackson Sent: Saturday, December 16, 2006 10:34 PM To: Programming forum Subject: Re: [Jprogramming] Is there a way I can let J know whatthesparseelement ought to be for the result of a boolean comparison? Roger's comment provides a potential solution, but in a comment to me some time ago he suggested creating the array and then assigning elements to it. That way you can assign whatever 'zero element' you like, and it solved some difficult space problems for me. s =: 3 10 10 NB. array shape c =: { (<@i.) "0 s NB. catalogue of cell locs cr =: (15 ?. 300){,c NB. random non zero items sa=: 1 $. s;(i.#s);1 NB. create empty sparse array NB. with 1 as 'zero' element NB. create the array before NB. assigning values sa=:(10+i.15) cr} sa NB. assign some 'non zero' elements 0$. sa NB. displays as dense Fraser ----- Original Message ----- From: "Michael Berry" <[EMAIL PROTECTED]> To: "'Programming forum'" <[email protected]> Sent: Sunday, December 17, 2006 1:33 PM Subject: RE: [Jprogramming] Is there a way I can let J know what thesparseelement ought to be for the result of a boolean comparison? > Aha! This is the point I did not understand. My large array was sparse. > I > was comparing it (rank 1) to a much smaller dense vector. The smaller > vector > actually is dense, but by coercing it to sparse I get the behavior I was > hoping for. Thank you. (And thank everyone who tries to help a J > novice!) > > -Michael > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Roger Hui > Sent: Saturday, December 16, 2006 6:11 PM > To: Programming forum > Subject: Re: [Jprogramming] Is there a way I can let J know what the > sparseelement ought to be for the result of a boolean comparison? > > You have drawn incorrect conclusions from your > experiments, or you did not do sufficient experiments. > > x=: 4 (1000 [EMAIL PROTECTED] 2e9)} 1 $. 2e9 > y=: 4 (1000 [EMAIL PROTECTED] 2e9)} 1 $. 2e9 > > 3 $. x = y > 1 > 3 $. x ~: y > 0 > 3 $. 2 + x > 2 > 3 $. x - 1 > _1 > 3 $. 0 > x - 1 > 1 > 3 $. ^ x > 1 > 3 $. ^. x > __ > > If x and y are sparse and f is atomic, then > (3 $. f y) = f 3$.y > (3 $. x f y) = (3$.x) f 3$.y > > that is, the sparse element of f y is f applied > to the sparse element of y; likewise the sparse > element of x f y is (sparse element of x) f > (sparse element of y). > > You can use (3;e)$.y to specify that the sparse > element should be e and 8$.y to "remove" the > sparse items. However, both of these are "after > the fact". You should avoid dense f sparse and > sparse f dense if you want to keep everything > sparse. > > t=: (1e6 [EMAIL PROTECTED] 100) > $. 1e6{.1 > +/t > 989986 > t > 0 │ 1 > 1 │ 1 > 2 │ 1 > 3 │ 1 > 4 │ 1 > ... > q=: 8 $. (3;1) $. t > 7!:5 ;:'t q' > 5243072 320 > > > > ----- Original Message ----- > From: Michael Berry <[EMAIL PROTECTED]> > Date: Saturday, December 16, 2006 1:40 pm > Subject: [Jprogramming] Is there a way I can let J know what the sparse > element ought to be for the result of a boolean comparison? > >> Dear J experts, >> >> >> >> In the course of trying to figure out why I sometimes get "out of >> memory"errors when I didn't think I had done anything to cause my >> sparse arrays to >> become dense, I became curious about how J knows what the right sparse >> element will be for the result of a Boolean function where one >> argument is a >> sparse integer array. By experiment, I see that if I compare my >> sparsearray to a scalar argument, the resulting Boolean array has >> the appropriate >> sparse element, namely the result of comparing the integer array's >> sparseelement with the scalar argument: >> >> >> >> v1 =.$.100{.3 >> >> v2=.$._100{.3 >> >> b1=.v1<3 >> >> b2=.v2<3 >> >> b1 >> >> 0 | 0 >> >> b2 >> >> 99 | 0 >> >> b3=.v1>0 >> >> b4=.v2>0 >> >> b3 >> >> 0 | 1 >> >> b4 >> >> 99 | 1 >> >> >> >> If, on the other hand, I have an array argument, the sparse >> element of the >> resulting array is always 0: >> >> >> >> v5=. ?. 100$10 >> >> b1 =. v1>v5 >> >> b2 =. v1<v5 >> >> 3 $. b1 >> >> 0 >> >> 3 $. b2 >> >> 0 >> >> +/b1 >> >> 0 >> >> +/b2 >> >> 94 >> >> >> >> This explains my out of memory problem. b2 is mostly 1, but its >> sparseelement is 0. It is in the nature of the comparisons I am >> doing, that I >> expect my results to be either mostly 1 or mostly 0 and I know in >> advancewhich. I can, of course, rephrase my comparisons so that >> the "mostly" case >> is always 0, but this sometimes seems unnatural. I don't suppose >> there is a >> way I could whisper in J's ear to say I'm expecting mostly 1? > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
