That looks right (and is about 10x faster than my attempts). -- Raul
On Mon, Feb 27, 2017 at 11:05 AM, 'Mike Day' via Programming <[email protected]> wrote: > It's not clear what you had in mind for the dyadic function "softmax" . > > Anyway, is this what you require? > > I =: =/~@:i.@# NB. Identity matrix (Kronecker delta) > pdsm =: (*"1 (-~I))@:sm NB. Partial derivative (matrix) of sm > > pdsm a > 0.186192 _0.0676431 _0.0467234 _0.0718259 > _0.0676431 0.19866 _0.0516374 _0.0793799 > _0.0467234 _0.0516374 0.153191 _0.0548305 > _0.0718259 _0.0793799 _0.0548305 0.206036 > > This returns the whole matrix, but it's easy enough to save the output, > and extract the indexed elements you need. > > Or have I misunderstood the specification? > > Mike > > > > > On 27/02/2017 08:09, 'Jon Hough' via Programming wrote: >> >> Given an array, we can calculate the softmax function >> https://en.wikipedia.org/wiki/Softmax_function >> >> a =: 0.5 0.6 0.23 0.66 >> sm=:(] % +/ )@:^ NB. softmax >> >> sm a >> 0.247399 0.273418 0.188859 0.290325 >> >> The (partial) derivative of softmax is a little more complicated: >> >> If the array is of length N, we need an NxN matrix of partial derivatives >> where (in pseudo code) >> >> derivatives[i,j] = sm (array[i] ) *( 1 - sm(array[j]) if i == j >> or >> derivatives[i,j] = -1 * sm (array[i] ) * ( sm(array[j]) if i != j >> >> ( see here for the reasoning: >> http://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/ ) >> >> My implementation of the partial derivatives is this: >> >> >> NB. x value is index, y value is the whole array >> dsoftmax=: 4 : 0 >> idx=. x >> vals=. y >> smx=. idx softmax vals >> rx=. '' >> for_j. i.#vals do. >> if. j = idx do. rx=. rx , smx * (1 - smx) >> elseif. 1 do. rx=. rx ,(j softmax vals)* (0 - smx) end. >> end. >> rx >> ) >> >> >> Then, for example using above array a, >> >> (i.# a) dsoftmax"0 _ a >> >> gives the values, in a 4x4 matrix. >> >> This is quite slow. I have tried to do this without iterating and >> branching, but cannot figure out a way to do it. >> Any help appreciated. >> Thanks, >> >> Jon >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm > > > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
