Do you want to make a PR or of this? Le dim. 23 juil. 2017 08:20, Maxim Kochurov <maxim.v.kochu...@gmail.com> a écrit :

## Advertising

> class BatchedDiag(tt.Op): > """ > Fast BatchedDiag allocation > """ > __props__ = () > > def make_node(self, diag): > diag = tt.as_tensor_variable(diag) > if diag.type.ndim != 2: > raise TypeError('data argument must be a matrix', diag.type) > > return tt.Apply(self, [diag], [tt.tensor3(dtype=diag.dtype)]) > > def perform(self, node, ins, outs, params=None): > (C,) = ins > (z,) = outs > > bc = C.shape[0] > dim = C.shape[-1] > Cd = np.zeros((bc, dim, dim), C.dtype) > bidx = np.repeat(np.arange(bc), dim) > didx = np.tile(np.arange(dim), bc) > Cd[bidx, didx, didx] = C.flatten() > z[0] = Cd > > def grad(self, inputs, gout): > (gz,) = gout > idx = tt.arange(gz.shape[-1]) > return [gz[..., idx, idx]] > > def infer_shape(self, nodes, shapes): > return [(shapes[0][0], ) + (shapes[0][1],) * 2] > > Here is code for Custom Op that might work faster when taking gradients > > > суббота, 7 мая 2016 г., 16:00:54 UTC+3 пользователь Tambet Matiisen > написал: > >> OK, solved. I used Keras wrapper K.zeros(), but this created Numpy matrix >> of zeros, which failed with Theano expression as dimension. After switching >> to full Theano implementation the error went away. The final code looks >> like this: >> >> # initialize with zeros >> batch_size = x.shape[0] >> a = T.zeros((batch_size, num_actuators, num_actuators)) >> # set diagonal elements >> batch_idx = T.extra_ops.repeat(T.arange(batch_size), num_actuators) >> diag_idx = T.tile(T.arange(num_actuators), batch_size) >> b = T.set_subtensor(a[batch_idx, diag_idx, diag_idx], >> T.flatten(T.exp(x[:, :num_actuators]))) >> # set lower triangle >> cols = np.concatenate([np.array(range(i), dtype=np.uint) for i in >> xrange(num_actuators)]) >> rows = np.concatenate([np.array([i]*i, dtype=np.uint) for i in >> xrange(num_actuators)]) >> cols_idx = T.tile(T.as_tensor_variable(cols), batch_size) >> rows_idx = T.tile(T.as_tensor_variable(rows), batch_size) >> batch_idx = T.extra_ops.repeat(T.arange(batch_size), len(cols)) >> c = T.set_subtensor(b[batch_idx, rows_idx, cols_idx], T.flatten(x[:, >> num_actuators:])) >> >> Thanks injecting me belief that it is possible! >> >> Tambet >> >> reede, 6. mai 2016 17:57.02 UTC+3 kirjutas nouiz: >>> >>> what error do you get? >>> >>> >>> On Fri, May 6, 2016 at 10:54 AM, Tambet Matiisen <tambet....@gmail.com> >>> wrote: >>> >>>> I could not figure out how make broadcasting work here, so I >>>> implemented option 2. >>>> >>>> num_actuators=4 >>>> x = K.variable([range(num_actuators*(num_actuators+1)/2)]*5) >>>> >>>> batch_size = K.shape(x)[0] >>>> a = K.zeros((batch_size.eval(), num_actuators, num_actuators)) >>>> >>>> # populate diagonal >>>> batch_idx = T.extra_ops.repeat(T.arange(batch_size), num_actuators) >>>> diag_idx = T.tile(T.arange(num_actuators), batch_size) >>>> b = T.set_subtensor(a[batch_idx, diag_idx, diag_idx], >>>> T.flatten(K.exp(x[:, :num_actuators]))) >>>> >>>> # populate lower triangle >>>> cols = np.concatenate([np.array(range(i), dtype=np.uint) for i in >>>> xrange(num_actuators)]) >>>> rows = np.concatenate([np.array([i]*i, dtype=np.uint) for i in >>>> xrange(num_actuators)]) >>>> cols_idx = T.tile(K.variable(cols, dtype=int), batch_size) >>>> rows_idx = T.tile(K.variable(rows, dtype=int), batch_size) >>>> batch_idx = T.extra_ops.repeat(T.arange(batch_size), len(cols)) >>>> c = T.set_subtensor(b[batch_idx, rows_idx, cols_idx], T.flatten(x[:, >>>> num_actuators:])) >>>> >>>> It works nicely, but only because I eval() batch_size when creating all >>>> zeros array. In real application I don't know the batch size beforehand and >>>> using it without eval() gives an error. So the question is - can you create >>>> a matrix in Theano dynamically, depending on some value in computational >>>> graph? >>>> >>>> Tambet >>>> >>>> reede, 6. mai 2016 16:14.59 UTC+3 kirjutas nouiz: >>>>> >>>>> broadcasting could be in theory more efficient. So this would request >>>>> that you try option 1. >>>>> >>>>> Otherwise, both should work. >>>>> >>>>> Fred >>>>> >>>>> On Fri, May 6, 2016 at 9:12 AM, Tambet Matiisen <tambet....@gmail.com> >>>>> wrote: >>>>> >>>>>> Actually I know the dimensions of the matrix beforehand, so I can do >>>>>> those calculations in Python+Numpy. Following seems to do the trick: >>>>>> >>>>>> num_actuators = 3 >>>>>> x = [1,2,3,4,5,6] >>>>>> a = K.zeros((num_actuators, num_actuators)) >>>>>> >>>>>> # set diagonal elements >>>>>> b = T.set_subtensor(a[range(num_actuators), range(num_actuators)], >>>>>> K.exp(x[:num_actuators])) >>>>>> >>>>>> # set lower triangle >>>>>> cols = np.concatenate([np.array(range(i), dtype=np.uint) for i in >>>>>> xrange(num_actuators)]) >>>>>> rows = np.concatenate([np.array([i]*i, dtype=np.uint) for i in >>>>>> xrange(num_actuators)]) >>>>>> c = T.set_subtensor(b[rows, cols], x[num_actuators:]) >>>>>> >>>>>> K.eval(c) >>>>>> >>>>>> >>>>>> array([[ 2.71828175, 0. , 0. ], >>>>>> [ 4. , 7.38905621, 0. ], >>>>>> [ 5. , 6. , 20.08553696]], dtype=float32) >>>>>> >>>>>> >>>>>> (I'm mixing Keras and Theano functions here, but I guess you >>>>>> understand the idea.) >>>>>> >>>>>> Now the problem is the following - actually x is not 1D, but 2D, the >>>>>> first dimension is batch size. So I would like to kind of broadcast this >>>>>> operation over first dimension of x. Is there any way to do it? >>>>>> >>>>>> An alternative would be to >>>>>> 1. construct a to be 3D, first dimension batch size, >>>>>> 2. repeat all index ranges batch size times. >>>>>> Sounds quite inefficient, but I guess doable. >>>>>> >>>>>> Tambet >>>>>> >>>>>> reede, 6. mai 2016 1:25.02 UTC+3 kirjutas nouiz: >>>>>> >>>>>>> >>>>>>> Le 5 mai 2016 16:18, "Tambet Matiisen" <tambet....@gmail.com> a >>>>>>> écrit : >>>>>>> > >>>>>>> > Thanks Fred for a hint! Following seems to work (I'm using K is >>>>>>> the Keras equivalent of theano.tensor): >>>>>>> > >>>>>>> > a = K.zeros((3,3)) >>>>>>> > K.eval(a) >>>>>>> > >>>>>>> > array([[ 0., 0., 0.], >>>>>>> > [ 0., 0., 0.], >>>>>>> > [ 0., 0., 0.]], dtype=float32) >>>>>>> > >>>>>>> > <br><br><br> >>>>>>> > >>>>>>> > b = T.set_subtensor(a[[0,1,2],[0,1,2]], [1,2,3]) >>>>>>> > >>>>>>> > K.eval(b) >>>>>>> > >>>>>>> > >>>>>>> > array([[ 1., 0., 0.], >>>>>>> > [ 0., 2., 0.], >>>>>>> > [ 0., 0., 3.]], dtype=float32) >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > But what if I don't know the matrix dimensions beforehand? Can I >>>>>>> produce the list of indexes to assign using just Theano arithmetics? >>>>>>> >>>>>>> Yes. There is T.a range(...) That you can use. >>>>>>> >>>>>>> Fred >>>>>>> > >>>>>>> > Tambet >>>>>>> > >>>>>>> > >>>>>>> > neljapäev, 5. mai 2016 17:30.08 UTC+3 kirjutas nouiz: >>>>>>> >> >>>>>>> >> The first idea I have is to init a vector of zeros of 9 element >>>>>>> and use set_subtensor to set the indices to the value you want. Then >>>>>>> reshape to a matrix. >>>>>>> >> >>>>>>> >> Fred >>>>>>> >> >>>>>>> >> On Thu, May 5, 2016 at 5:07 AM, Tambet Matiisen < >>>>>>> tambet....@gmail.com> wrote: >>>>>>> >>> >>>>>>> >>> Hi everyone! >>>>>>> >>> >>>>>>> >>> I'm trying to apply T.diag() and T.tril() operations over batch >>>>>>> of matrices, so that first dimension is preserved. Theano doesn't seem >>>>>>> to >>>>>>> provide built-in function to do that. Is there any other way to achieve >>>>>>> the >>>>>>> same? >>>>>>> >>> >>>>>>> >>> Basically I need to turn bunch of numbers x1, ... , x6 into a >>>>>>> matrix like this: >>>>>>> >>> >>>>>>> >>> | e^x1 0 0 | >>>>>>> >>> | x2 e^x3 0 | >>>>>>> >>> | x4 x5 e^x4 | >>>>>>> >>> >>>>>>> >>> i.e. diagonal must be filled with e^xi and lower triangle must >>>>>>> be filled with just xi. Order of x-s is not particularly important, as >>>>>>> these are learned weights anyway. >>>>>>> >>> >>>>>>> >>> Thanks! >>>>>>> >>> Tambet >>>>>>> >>> >>>>>>> >>> -- >>>>>>> >>> >>>>>>> >>> --- >>>>>>> >>> You received this message because you are subscribed to the >>>>>>> Google Groups "theano-users" group. >>>>>>> >>> To unsubscribe from this group and stop receiving emails from >>>>>>> it, send an email to theano-users...@googlegroups.com. >>>>>>> >>> >>>>>>> >>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >> >>>>>>> >> >>>>>>> > -- >>>>>>> > >>>>>>> > --- >>>>>>> > You received this message because you are subscribed to the Google >>>>>>> Groups "theano-users" group. >>>>>>> > To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to theano-users...@googlegroups.com. >>>>>>> > For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>>> >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "theano-users" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to theano-users...@googlegroups.com. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "theano-users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to theano-users...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to theano-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.