I'm doing some analyses on EEG data and have run into a limit error for
vectors of > 4000 elements. I'm wondering if there's any way around it.
The code involves moving a fixed width window across a long vector of values
one element at a time, computing the outer product of the (reversed) window
of values with itself, and averaging over all the resulting matrices. I run
into the limit error for vectors greater than 4000 elements with a typical
window size of 276 - yes, I was naive not to expect this :-)
Here's the code that hits the limit:
NB. x = window size ; y = vector of EEG values
calcAvgOP =: [: (+/ % #) (*"0 1 |.)\
$M =: 276 calcAvgOP i. 4000
|limit error: calcAvgOP
| M=:276 calcAvgOP i.4000
The problem is having to hold onto the millions of intervening matrices
before averaging. I'd very much like to find a tacit solution to this
problem and I was wondering if there's a formulation that can circumvent it.
On the other hand, the rather inelegant explicit definition below works
fine.
NB. x = window size ; y = vector of EEG values
calcAvgOP2 =: dyad define
i =. 0
lim =. >: x -~ $y
acc =. 0 $~ x,x
while. i < lim do.
acc =. acc + (*"0 1 |.) x {. y
y =. }. y
i =. >: i
end.
acc % i
)
$M =: 276 calcAvgOP2 i. 4000
276 276
Thanks for any help.
Ronan
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
