OK folks, I think I got it down to a tee now:
goal =: _1 1
reward =: _1 _1
rates =: 0.5 0.2 0.5 NB. remember, jumble, learn
w =: 1&{::
io =: 0&{::
ifio =: 0&{::
ofio =: 1&{::
i =: ifio@io
o =: ofio@io
show =: ([(1!:2&2)@('-+'{~>:&0)@(i,o))
rand =: _1 1 {~ ?@:(2"0)
matmul =: +/ .*
avg =: %&2@:+
avgio =: avg &.>
decide =: _1 1 {~ >:&0
i2o =: */~&>/
env =: (rand)`(reward&[) @. (-:&goal)
think =: w (] ; decide@matmul) env@o
learn =: +/ @ (rates&*) @ (w@[ , rand@w@[ ,: i2o@(avgio io)~)
cycle =: ((] ; learn) think)
cycle@show ^:40 (; ; ,:)~ 0 0
Now for the hard bit: the driving game.
Adrian.
On 26 March 2011 02:08, Raul Miller <[email protected]> wrote:
> On Fri, Mar 25, 2011 at 11:14 AM, Adrian May
> <[email protected]> wrote:
> >> If I understand correctly, the sum you want is:
> >> (weightlearn y) + (jumblerate@rand w y) + w y
> >>
> >
> > In C syntax, I want
> > new_w =
> > learnrate*learn(old_w, old_i, old_o, new_i, new_o )
> > + jumblerate * random_matrix_of_-1s_and_1s
> > + rememberreate * old_w
> >
> > where the rates are constants. My idea was to put the three functions in
> a
> > list and sum over them, but I guess that's the Haskell talking. I did
> kinda
> > want to put the rates into some tidy place, but it's not essential.
>
> So.. ok... I do not have a definition for learn, yet. But I believe
> that old_w is a matrix of weights, so new_w should also be a matrix of
> weights. And learnrate, jumblerate and rememberrate I expect to all
> be scalars (just single numbers with no dimensions).
>
> So... I am going to guess
> learn=: */~&>/@((avgio io)~)@think
>
> And, for the random matrix of _1 and 1:
> jumble=: _1 1 {~ ? bind (2 2$2)
>
> And, some rates:
> learnrate=: 0.3&[
> jumblerate=: 0.2&[
> rememberrate=: 0.5&[
>
> That said, I am going to be careful to construct my sentences so you
> could use the more simplistic
> learnrate=: 0.3
> jumblerate=: 0.2
> rememberrate=: 0.5
>
> This leaves me with:
> ((learnrate * learn) + (jumblerate * jumble) + (rememberrate * w)) seed
>
> With this structure, using a sum might be meaningful:
>
> rates=: learnrate,jumblerate,rememberrate
> weights=: learn, jumble,: w
> +/ (rates * weights) seed
> or
> (rates +/ .* weights) seed
>
> Note that when I combined arrays, I needed to be careful to indicate
> which dimensions were "content dimensions" and which dimensions were
> "container dimensions". Thus, I combine the last pair with ,: (which
> introduces a new dimension) and the rest of the arrays are introduced
> using , (which does not introduce a new dimension).
>
> >> But if by sum you meant +/ I am not sure that this is a good idea. +/
> >> would make sense if you had a reason for these three things to be in a
> >> list already, but since you are computing each value independently,
> >> forming a list form them seems like a meaningless operation.
> >
> > That seems like a funny thing to say. I thought J loved making lists.
>
> It depends on what you are doing. For example, you can do:
> 1 + 2 + 3
> or you can do
> +/ 1 , 2 , 3
>
> If the only thing you are going to do with the items in that array is
> add them up, it's simpler to just add them when combining them.
>
>
> > I'd like to get it working both ways just as a learning exercise. My
> program
> > works already but it bugs me that I can't seem to get this function list
> > working when it clearly should work in principle given that I already
> made
> > it half work out of context. I got this far:
> >
> > a +/@((3 2 1)&*)"2@(+,-,*)"0 b
> > 15 28
> > 43 60
> >
> > but when I try to factor out the (3 2 1)&* it barfs.
>
> Note that binary operations can only combine two values, so you would
> need to combine a and b into a list before you can factor out 3 2 1.
>
> +/@((3 2 1)&*)"2@(+,-,*)"0/ a,:b
>
> Then you can take advantage of the fact that you are trying to find an
> inner product, and:
>
> 3 2 1 +/ .*"1 (+,-,*)"0/ a,:b
>
> or, rather than constructing an array of the results of (+,-,*) we
> could instead assemble those three results into an array like this:
>
> 3 2 1 +/ .* (+,-,:*)/ a,:b
>
> And, since we have factored out 3 2 1 and we do not need a and b in a list:
>
> 3 2 1 +/ .* a (+,-,:*) b
> or
> +/ 3 2 1 * a (+,-,:*) b
>
> But there might be simpler ways of getting here...
>
> And, now that I think I understand what you were trying to do, I see
> that you could also have done:
>
> +/"1 (3 2 1) *"1 a (+,-,*)"0 b
>
> FYI,
>
> --
> Raul
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
