If QR were to change in time, would ProjModel1 become a bit unwieldy, and
ProjModel 1a still remain palatable?
Probably, then if
QR=: ((>:/)~i.y)*"2 (0.00001*
(10000*i.2)+/(1000*i.y+4)+/(100*i.-y)+/1+i.-y)
the revisions would be:
test=: 3 : 0
POP=:((>:/)~i.y)*"2 (1000*i.2)+/(100*i.y)+/1+i.y NB. SERVICE AGE
if. x=6 do.
QR=: ((>:/)~i.y)*"2 (0.00001*
(10000*i.2)+/(1000*i.y+4)+/(100*i.-y)+/1+i.-y)
else.
QR=: ((>:/)~i.y)*"2 (0.0001* (1000*i.2)+/(100*i.-y)+/1+i.-y)
end.
NV=: (2, y+4,0) $,10000+(100*i.y+4)+/i.y NB. ANS AGE
projrotate`projoblique`projloop`projoutf`projmodel1`projmodel1a`[email protected]
POP;NV;QR
)
ProjModel1a1=: 3 : 0 NB. same as before but with for-loop
'pop new srv'=.y
srv=. 1 0 2 3|: srv
pop=.(i...@# |."0 1 |:)"2 pop
srv=.(i...@# |."0 1 |:)"2 srv
srv=.(* <:@# >+/~@:i...@#)"2 srv
'new'=. 1 0 2 |: new
z=. ,: pop
for_t. i.#new do.
z=.z, (t{new) ,"1 2 }:"2 (t{srv) * {:z
end.
1 0 3 2|:(-...@i.@# |."0 1 ])"2 z
)
but
[sc=:scores(":,.i.7)<@,"1 ' test 55'
0.207991 9.71514e7
0.859386 6.29322e7
0.0451834 1.68588e7
0.0391422 2.08553e7
0.0233755 1.69299e7
0.0218773 1.27962e7
0.061731 2.10537e7
And not only the timings change quite a bit and the memory usage nearly
doubles. However, this temporary increase in used memory is not likely to
be a problem.
Robert Cyr
On Wed, Apr 1, 2009 at 5:45 PM, Robert Cyr <[email protected]> wrote:
> In my view these are great solutions.
>
> My ultimate aim is to evaluate the consequence of the market tsunami on the
> projected improved indexing. It has been promised by unions and the plan
> sponsor but I doubt they will be able to deliver.
>
> Initially I was not concerned about the use of memory, but in the end I
> will have to keep an eye on it, and your techniques will be quite handy.
> The arrays tend to become rather large as I am trying to keep track of the
> year a unit of pension is credited, the aging of the contributing population
> and the characteristics of the retired as they age.
>
> Thank you.
>
> Robert Cyr
>
>
> On Tue, Mar 31, 2009 at 1:27 PM, R.E. Boss <[email protected]> wrote:
>
>> Some alternatives from remodeling the data are more efficient.
>>
>> test=: 3 : 0
>> NB. x: 0 for projection, 1 for rotation
>> NB. y: number of age and service
>> :
>> POP=:((>:/)~i.y)*"2 (1000*i.2)+/(100*i.y)+/1+i.y NB. SERVICE AGE
>> QR=: ((>:/)~i.y)*"2 (0.0001* (1000*i.2)+/(100*i.-y)+/1+i.-y) NB.
>> SERVICE
>> AGE
>> NV=: (2, y+4,0) $,10000+(100*i.y+4)+/i.y NB. ANS AGE
>> projrotate`projoblique`projloop`projoutf`projmodel1`[email protected]
>> POP;NV;QR
>> )
>>
>> ProjModel1=: 3 : 0
>> 'pop new srv'=.y
>> 'srv pop'=.(i...@# |."0 1 |:)"2 srv,:pop
>> srv=.(* <:@# >+/~@:i...@#)"2 srv
>> 'new'=. 1 0 2 |: new
>> 1 0 3 2|:(-...@i.@# |."0 1 ])"2>(,"1 2 [:}:"2 srv&*)&.>/\. &.|.pop;;/new
>> )
>>
>> ProjModel1a=: 3 : 0 NB. same as before but with for-loop
>> 'pop new srv'=.y
>> 'srv pop'=.(i...@# |."0 1 |:)"2 srv,:pop
>> srv=.(* <:@# >+/~@:i...@#)"2 srv
>> 'new'=. 1 0 2 |: new
>> z=. ,: pop
>> for_t. new do.
>> z=.z, t ,"1 2 }:"2 srv * {:z
>> end.
>> 1 0 3 2|:(-...@i.@# |."0 1 ])"2 z
>> )
>>
>> [sc=:scores(":,.i.6)<@,"1 ' test 55'
>> 0.33658643 76295872
>> 1.4320343 58728320
>> 0.082450888 16817472
>> 0.056301756 12590144
>> 0.040160138 16623744
>> 0.043919412 12588608
>>
>> [rn=:rnkng sc
>> 4 46.447614 8.3811073 6.0607076
>> 5 152.11316 35.658103 4.6651957
>> 3 2.507967 2.0530529 1.3359278
>> 2 1.2820897 1.4019313 1.000122
>> 1 1.2075074 1 1.3205387
>> 0 1 1.0936071 1
>>
>>
>>
>> ('ProjRotate';'ProjOblique';'ProjLoop';'ProjOutf';'ProjModel1';'ProjModel1a'
>> )dspl rn
>> +-----------+----+-------+------+------+
>> |verb |rank|rlt_prf|rlt_et|rlt_sz|
>> +-----------+----+-------+------+------+
>> |ProjRotate | 4 | 46.45 | 8.38 |6.06 |
>> +-----------+----+-------+------+------+
>> |ProjOblique| 5 |152.11 |35.66 |4.67 |
>> +-----------+----+-------+------+------+
>> |ProjLoop | 3 | 2.51 | 2.05 |1.34 |
>> +-----------+----+-------+------+------+
>> |ProjOutf | 2 | 1.28 | 1.40 |1.00 |
>> +-----------+----+-------+------+------+
>> |ProjModel1 | 1 | 1.21 | 1.00 |1.32 |
>> +-----------+----+-------+------+------+
>> |ProjModel1a| 0 | 1.00 | 1.09 |1.00 |
>> +-----------+----+-------+------+------+
>>
>> ProjModel1a is the overall champion, ProjModel1 is fastest.
>>
>> 2-:/\(i.6) test"0[ 55
>> 1 1 1 1 1
>>
>> Their output match.
>>
>>
>> ts=: 6!:2 , 7!:2...@]
>>
>> scores=: 5 ts&> ]
>>
>> rnkng=: 3 : '([:(,~/:@/:@{.) [:(%<./)"1(,~*/)) &.|: y'
>>
>> dspl=: 4 : 0
>> z=: '2.0,0.2,0.2,0.2'(8!:0) y
>> z=: ('rank';'rlt_prf';'rlt_et';'rlt_sz'),z
>> ('verb';x),.z
>> )
>>
>>
>> R.E. Boss
>>
>>
>> -----Oorspronkelijk bericht-----
>> Van: [email protected]
>> [mailto:[email protected]] Namens Robert Cyr
>> Verzonden: maandag 23 maart 2009 18:44
>> Aan: Programming forum
>> Onderwerp: Re: [Jprogramming] Oblique can be slow
>>
>> Obviously, thank you.
>>
>> I like the way J encourages the manipulation of these multidimensional
>> arrays.
>>
>> On Mon, Mar 23, 2009 at 11:10 AM, R.E. Boss <[email protected]> wrote:
>>
>> > Not very much faster but more elegant, replacing the for-loop:
>> >
>> > 10 ts 'loop test 55'
>> > 0.057558227 16816768
>> >
>> > 10 ts 'outf test 55'
>> > 0.052024896 20978048
>> >
>> > (outf test 55)-:loop test 55
>> > 1
>> >
>> > with
>> >
>> > 'rotate oblique loop outf' =: i.4
>> >
>> > and
>> >
>> > ProjOutf=: 3 : 0
>> > 'pop new srv'=:y
>> > new=. 1 0 2 |: new NB. it makes it easy to loop on the first dimention
>> > p=. pop,"3 4(new,"1(0)),"0 1"2 3(0,"2 srv)
>> > q=.(*(-1 1)|.!.1 "1 2 ])/\.&.|. p
>> > 1 0 2 3 |:}:"1}:"2 q
>> > )
>> >
>> >
>> > R.E. Boss
>> >
>> >
>> > -----Oorspronkelijk bericht-----
>> > Van: [email protected]
>> > [mailto:[email protected]] Namens Robert Cyr
>> > Verzonden: zaterdag 21 maart 2009 15:55
>> > Aan: Programming forum
>> > Onderwerp: Re: [Jprogramming] Oblique can be slow
>> >
>> > In this case, the simple looping solution is 20 times faster than the
>> > oblique solution, as it avoids some unnecessary calculations and
>> > manipulation of data. It is also makes for easier reading and
>> maintenance
>> >
>> > Noting that the population data is basically triangular(age service)
>> where
>> > age-service >= 0 is of little use as we cannot easily create create and
>> use
>> > a triangular matrix.
>> >
>> > try these:
>> >
>> > 'rotate oblique loop' =. i.3
>> >
>> > ts 'rotate test 55'
>> > 0.203641 9.71485e7
>> >
>> > ts 'oblique test 55'
>> > 0.854339 6.29294e7
>> >
>> > ----------------------------------------
>> > and the winner is...
>> > ----------------------------------------
>> > ts 'loop test 55'
>> > 0.0385865 1.68559e7
>> > ----------------------------------------
>> >
>> > test=: 3 : 0
>> > NB. x: 0 for projection, 1 for rotation
>> > NB. y: number of age and service
>> > :
>> > POP=:((>:/)~i.y)*"2 (1000*i.2)+/(100*i.y)+/1+i.y NB. SERVICE AGE
>> > QR=: ((>:/)~i.y)*"2 (0.0001* (1000*i.2)+/(100*i.-y)+/1+i.-y) NB.
>> SERVICE
>> > AGE
>> > NV=: (2, y+4,0) $,10000+(100*i.y+4)+/i.y NB. ANS AGE
>> > projrotate`projoblique`[email protected] POP;NV;QR
>> > )
>> >
>> > ProjLoop=: 3 : 0
>> > 'pop new srv'=.y
>> > new=. 1 0 2 |: new NB. it makes it easy to loop on the first dimention
>> > p=. pop,"3 4(new,"1(0)),"0 1"2 3(0,"2 srv)
>> > for_t. i._1+#p do.
>> > p=.(((t+1){p)*(-1 1)|.!.1 "1 2 t{p)(t+1)}p
>> > end.
>> > 1 0 2 3 |:}:"1}:"2 p
>> > )
>> >
>> > ProjRotate=: 3 : 0
>> > NB. Population projection using rotation
>> > NB. result by sex year age service will allow us to determine the
>> probable
>> > aging of our population.
>> > NB. pop: number of active employees by sex(2), age(55=16 to 70),
>> service(55
>> > = 0 to 54)
>> > NB. new: number of new employees by sex(2), year of employment(50=years
>> of
>> > projection), age(55)
>> > NB. srv: probability of survival (not dying,quitting or retiring) by
>> > sex(2),age(55),service(55)
>> > 'pop new srv'=.y
>> > p=. pop,"2 3 (new,"1 (0)),"0 1"1 2 (0,"2 srv) NB. setup a matrix :
>> > sex,year of projection, age,service
>> > 'a n'=. (1 0)+ 1}.$new
>> > r=. (,.)~i.a NB. to shift each slice to the left and to the
>> top
>> > by 1 for each year of projection
>> > t=.-(0,0,a,a)+$p NB. to make space for our rotation
>> > f=.2,a,1+n,n NB. to return to the original size
>> > }:"2}:"1(-f){.(-r)|.!.0"1 2"2 3 */\"3 r |.!.1"1 2"2 3 t{.!.1 p
>> > )
>> >
>> >
>> > ProjOblique=: 3 : 0
>> > 'pop new srv'=.y
>> > p=. pop,"2 3 (new,"1 (0)),"0 1"1 2 (0,"2 srv)
>> > 'a n'=. (1 0)+ 1}.$new
>> > r=. |."1|."2*//./.\"3|."2|."1 p NB. this is the 3d oblique
>> > cumulative product, by year of projection
>> > b=. (1+i.a)|."0 1 ((a,n)#0 1) NB. eliminate the unused rows of
>> > the oblique
>> > s=.|:n-(i.n)<./i.a NB. a rotation will position
>> the
>> > required results
>> > n{."1 s|."0 1 " 2 3 b#"1 2"2 3 r NB. we retain the correct number
>> of
>> > columns
>> > )
>> > ----------------------------------------------------------------------
>> > For information about J forums see http://www.jsoftware.com/forums.htm
>> >
>> > ----------------------------------------------------------------------
>> > For information about J forums see http://www.jsoftware.com/forums.htm
>> >
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
>>
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
>>
>
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
