Thanks Devon.
That was not the problem though because my mx and my were booleans. I
rewrote the whole thing to not use so much memory in the first place. It is
a bit slower but does the job. I used an adverb called region2D to do
anything to the data inside each region. Maybe there is a much faster way to
do this using the cut verb though?
If anyone is interested, code is below:
region2D =: 1 : 0
:
bx =. (__ , ( 0{::x )) ,. _ ,~ ( 0{::x ) NB. cut boundaries
by =. (__ , ( 1{::x )) ,. _ ,~ ( 1{::x ) NB. cut boundaries
b =. _4 ]\ , bx ,"_1 _"_ _1 by NB. corners of grid boxes
xx =. (0: {"1 [)"_
yy =. (1: {"1 [)"_
zz =. (2: {"1 [)"_
lx =. (0: { ])"_
ux =. (1: { ])"_
ly =. (2: { ])"_
uy =. (3: { ])"_
a1 =. xx >: lx NB. tests for point on side of line
a2 =. xx < ux
a3 =. yy >: ly
a4 =. yy < uy
reg =. ( u @ (zz #~ (a1 *. a2 *. a3 *. a4) ) )"_ 1 f. NB. test for point
in region and do u
(- # by ) ]\ ; y reg b
)
histogram2D =: 4 : 0 "_ 2
(% (+/@:+/)) x # region2D y
)
load 'viewmat'
load 'statsdist'
viewmat ((steps _4 4 50);(steps _4 4 50))histogram2D (((normalrand
200000),.(normalrand 200000)),.1)
On Tue, Jul 22, 2008 at 2:30 PM, Devon McCormick <[EMAIL PROTECTED]> wrote:
> Matthew - are you sure your arrays are of type boolean? If I try to create
> an array equivalent to one of yours this way:
>
> mx=. 1-?484 200000$2
> |out of memory
> | mx=.1 -?484 200000$2
>
> I run out of memory because, even though "mx" will be all ones and zeros,
> they will be integers because they are the result of an integer function.
>
> On the other hand, I have no problem creating this:
>
> mx=. -.?484 200000$2
>
> which we can see is type boolean:
>
> 3!:0 mx
> 1
>
> as opposed to this (note the reduction in size to avoid running out of
> memory):
>
> mx=. 1-?484 20000$2
> 3!:0 mx
> 4
>
> which is integer. You can coerce an array of ones and zeros to boolean
> like
> this:
>
> 3!:0 mx NB. is now integer
> 4
> ~.,mx NB. Is only zeros and ones
> 0 1
> mx=. 1=mx
> 3!:0 mx NB. is now boolean
> 1
>
> Hope this helps,
>
> Devon
>
>
>
> On 7/22/08, Matthew Brand <[EMAIL PROTECTED]> wrote:
> >
> > Hi All,
> >
> > I am writing a program to generate two dimensional histograms but find
> that
> > the "out of memory" error is generated for reasonable sized datasets. So
> > far
> > I have come up with the code below.
> >
> > Is there an alternative to *. which would avoid the out of memory error
> > without doing an explicit
> > for loop? Would the problem be solved by using J64?
> >
> > I just want to "and" each corresponding atom like in the following:
> >
> > $mx
> > 484 200000
> > $my
> > 484 200000
> > mx*.my
> > |out of memory
> > | mx *.my
> >
> > Also, how can I add x and y labels to the output of viewmat?
> >
> > Thanks,
> > Matthew.
> >
> > load 'viewmat'
> > histogram2D =: 4 : 0 "_ 2
> > bx =. (__ , ( 0{::x )) ,. _ ,~ ( 0{::x ) NB. cut boundaries
> > by =. (__ , ( 1{::x )) ,. _ ,~ ( 1{::x ) NB. cut boundaries
> > b =. _4 ]\ , bx ,"_1 _"_ _1 by NB. corners of grid boxes
> > lbx =. ([: 0&{ [) NB. verb to get lower bound from the pair
> of
> > bounds
> > ubx =. ([: 1&{ [) NB. verb to get upper bound from the pair
> > of
> > bounds
> > lby =. ([: 2&{ [) NB. verb to get lower bound from the pair
> of
> > bounds
> > uby =. ([: 3&{ [) NB. verb to get upper bound from the pair
> > of
> > bounds
> > mx =. b ( (] >: lbx ) *. ( ] < ubx) )"1 1 ( 0 {"1 y ) NB. mask for
> > datapoints in each boundaries
> > my =. b ( (] >: lby ) *. ( ] < uby) )"1 1 ( 1 {"1 y ) NB. mask for
> > datapoints in each boundaries
> > m =. mx *. my NB. each point is in one and only one box
> > hz =. (- # by ) ]\ ( [: (] % +/) +/"1 ) m
> > hx =. (2: %~ lbx + ubx)"1 b
> > hy =. (2: %~ lby + uby)"1 b
> > viewmat hz
> > hx;hy;hz
> > )
> > NB. example
> > ((steps _4 4 20);(steps _4 4 20))histogram2D (((normalrand
> > 100000),.(normalrand 100000)),.1)
> > NB. but this runs out of memory on line "m =. mx *. my "
> > ((steps _4 4 20);(steps _4 4 20))histogram2D (((normalrand
> > 200000),.(normalrand 200000)),.1)
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> >
>
>
>
> --
> Devon McCormick, CFA
> ^me^ at acm.
> org is my
> preferred e-mail
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
