Matthew
Here's a simple but not optimal improvement, based on the
fact that you're windowing only by day and not by time
series;
NB. starting approach
ts 'a1=:; (1 1,:400,25) (+/@:;);._3 (i. 400 20000)'
1.04556 6.9193e7
NB. if you can afford to precompute the column sums
ts 'a2=:; 25 (+/@:;);._3 TSsum' [ TSsum =: +/i.400 20000
0.0326386 1.34589e7
NB. if you want column sums on the fly
ts 'a3=:; 25 (+/@:;);._3 +/i.400 20000'
0.155832 8.06992e7
a1-:a2
1
That should help a bit.
A rather different approach is to avoid repeating all
those additions and just work on the cumulative sum.
However it could raise problems with precision with
real data:
ts'a4=:}:-//.((24&}.),:(_25&}.))+/\TSsum'
0.0122425 2.69056e6
a1-:a4
1
It's probably quicker to do the difference directly
rather than assemble two rows for an oblique difference
but I expect you can take it from here...
Mike
Matthew Brand wrote:
Hi All,
I have a bottleneck in my programs which is related to my use of ;. .
Does anyone have any advice on how I could change my data structure or
my usage of ;. to speed things up?
I am storing all of my time series (which I have aligned) in a 2x2
matrix where each row is a different series and each column is a
different day. So for me a shape 400 10000 array is 400 time series
with 10000 days each.
I am applying a moving time window of fixed duration over the series
and applying a function on the data inside the window using ;. .
The example below is generates sample data as i.400 10000 and adds up
the numbers in a window of duration 25 days.
ts '; (1 1,:400,25) (+/@:;);._3 (i. 400 20000)'
1.90589 6.91929e7
I can get a small speed up by rotating the input matrix:
ts '; (1 1,:25,400) (+/@:;);._3 (|: i. 400 20000)'
1.69103 1.02748e8
I can get another small speed up by using , instead of ; :
ts '; (1 1,:25,400) (+/@:,);._3 (|: i. 400 20000)'
0.614452 1.05468e8
Is there a way to speed this up further by changing the way I am
representing the data or any in any other way? Maybe ;. is not the
best way to do this?
Thanks,
Matthew.
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
------------------------------------------------------------------------
No virus found in this incoming message.
Checked by AVG - http://www.avg.com
Version: 8.0.175 / Virus Database: 270.9.0/1773 - Release Date: 07/11/2008 09:08
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
