Do you want to make a PR or of this?

Le dim. 23 juil. 2017 08:20, Maxim Kochurov <maxim.v.kochu...@gmail.com> a
écrit :

> class BatchedDiag(tt.Op):
>     """
>     Fast BatchedDiag allocation
>     """
>     __props__ = ()
>
>     def make_node(self, diag):
>         diag = tt.as_tensor_variable(diag)
>         if diag.type.ndim != 2:
>             raise TypeError('data argument must be a matrix', diag.type)
>
>         return tt.Apply(self, [diag], [tt.tensor3(dtype=diag.dtype)])
>
>     def perform(self, node, ins, outs, params=None):
>         (C,) = ins
>         (z,) = outs
>
>         bc = C.shape[0]
>         dim = C.shape[-1]
>         Cd = np.zeros((bc, dim, dim), C.dtype)
>         bidx = np.repeat(np.arange(bc), dim)
>         didx = np.tile(np.arange(dim), bc)
>         Cd[bidx, didx, didx] = C.flatten()
>         z[0] = Cd
>
>     def grad(self, inputs, gout):
>         (gz,) = gout
>         idx = tt.arange(gz.shape[-1])
>         return [gz[..., idx, idx]]
>
>     def infer_shape(self, nodes, shapes):
>         return [(shapes[0][0], ) + (shapes[0][1],) * 2]
>
> Here is code for Custom Op that might work faster when taking gradients
>
>
> суббота, 7 мая 2016 г., 16:00:54 UTC+3 пользователь Tambet Matiisen
> написал:
>
>> OK, solved. I used Keras wrapper K.zeros(), but this created Numpy matrix
>> of zeros, which failed with Theano expression as dimension. After switching
>> to full Theano implementation the error went away. The final code looks
>> like this:
>>
>>     # initialize with zeros
>>     batch_size = x.shape[0]
>>     a = T.zeros((batch_size, num_actuators, num_actuators))
>>     # set diagonal elements
>>     batch_idx = T.extra_ops.repeat(T.arange(batch_size), num_actuators)
>>     diag_idx = T.tile(T.arange(num_actuators), batch_size)
>>     b = T.set_subtensor(a[batch_idx, diag_idx, diag_idx],
>> T.flatten(T.exp(x[:, :num_actuators])))
>>     # set lower triangle
>>     cols = np.concatenate([np.array(range(i), dtype=np.uint) for i in
>> xrange(num_actuators)])
>>     rows = np.concatenate([np.array([i]*i, dtype=np.uint) for i in
>> xrange(num_actuators)])
>>     cols_idx = T.tile(T.as_tensor_variable(cols), batch_size)
>>     rows_idx = T.tile(T.as_tensor_variable(rows), batch_size)
>>     batch_idx = T.extra_ops.repeat(T.arange(batch_size), len(cols))
>>     c = T.set_subtensor(b[batch_idx, rows_idx, cols_idx], T.flatten(x[:,
>> num_actuators:]))
>>
>> Thanks injecting me belief that it is possible!
>>
>>   Tambet
>>
>> reede, 6. mai 2016 17:57.02 UTC+3 kirjutas nouiz:
>>>
>>> what error do you get?
>>>
>>>
>>> On Fri, May 6, 2016 at 10:54 AM, Tambet Matiisen <tambet....@gmail.com>
>>> wrote:
>>>
>>>> I could not figure out how make broadcasting work here, so I
>>>> implemented option 2.
>>>>
>>>> num_actuators=4
>>>> x = K.variable([range(num_actuators*(num_actuators+1)/2)]*5)
>>>>
>>>> batch_size = K.shape(x)[0]
>>>> a = K.zeros((batch_size.eval(), num_actuators, num_actuators))
>>>>
>>>> # populate diagonal
>>>> batch_idx = T.extra_ops.repeat(T.arange(batch_size), num_actuators)
>>>> diag_idx = T.tile(T.arange(num_actuators), batch_size)
>>>> b = T.set_subtensor(a[batch_idx, diag_idx, diag_idx],
>>>> T.flatten(K.exp(x[:, :num_actuators])))
>>>>
>>>> # populate lower triangle
>>>> cols = np.concatenate([np.array(range(i), dtype=np.uint) for i in
>>>> xrange(num_actuators)])
>>>> rows = np.concatenate([np.array([i]*i, dtype=np.uint) for i in
>>>> xrange(num_actuators)])
>>>> cols_idx = T.tile(K.variable(cols, dtype=int), batch_size)
>>>> rows_idx = T.tile(K.variable(rows, dtype=int), batch_size)
>>>> batch_idx = T.extra_ops.repeat(T.arange(batch_size), len(cols))
>>>> c = T.set_subtensor(b[batch_idx, rows_idx, cols_idx], T.flatten(x[:,
>>>> num_actuators:]))
>>>>
>>>> It works nicely, but only because I eval() batch_size when creating all
>>>> zeros array. In real application I don't know the batch size beforehand and
>>>> using it without eval() gives an error. So the question is - can you create
>>>> a matrix in Theano dynamically, depending on some value in computational
>>>> graph?
>>>>
>>>>   Tambet
>>>>
>>>> reede, 6. mai 2016 16:14.59 UTC+3 kirjutas nouiz:
>>>>>
>>>>> broadcasting could be in theory more efficient. So this would request
>>>>> that you try option 1.
>>>>>
>>>>> Otherwise, both should work.
>>>>>
>>>>> Fred
>>>>>
>>>>> On Fri, May 6, 2016 at 9:12 AM, Tambet Matiisen <tambet....@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Actually I know the dimensions of the matrix beforehand, so I can do
>>>>>> those calculations in Python+Numpy. Following seems to do the trick:
>>>>>>
>>>>>> num_actuators = 3
>>>>>> x = [1,2,3,4,5,6]
>>>>>> a = K.zeros((num_actuators, num_actuators))
>>>>>>
>>>>>> # set diagonal elements
>>>>>> b = T.set_subtensor(a[range(num_actuators), range(num_actuators)],
>>>>>> K.exp(x[:num_actuators]))
>>>>>>
>>>>>> # set lower triangle
>>>>>> cols = np.concatenate([np.array(range(i), dtype=np.uint) for i in
>>>>>> xrange(num_actuators)])
>>>>>> rows = np.concatenate([np.array([i]*i, dtype=np.uint) for i in
>>>>>> xrange(num_actuators)])
>>>>>> c = T.set_subtensor(b[rows, cols], x[num_actuators:])
>>>>>>
>>>>>> K.eval(c)
>>>>>>
>>>>>>
>>>>>> array([[  2.71828175,   0.        ,   0.        ],
>>>>>>        [  4.        ,   7.38905621,   0.        ],
>>>>>>        [  5.        ,   6.        ,  20.08553696]], dtype=float32)
>>>>>>
>>>>>>
>>>>>> (I'm mixing Keras and Theano functions here, but I guess you
>>>>>> understand the idea.)
>>>>>>
>>>>>> Now the problem is the following - actually x is not 1D, but 2D, the
>>>>>> first dimension is batch size. So I would like to kind of broadcast this
>>>>>> operation over first dimension of x. Is there any way to do it?
>>>>>>
>>>>>> An alternative would be to
>>>>>> 1. construct a to be 3D, first dimension batch size,
>>>>>> 2. repeat all index ranges batch size times.
>>>>>> Sounds quite inefficient, but I guess doable.
>>>>>>
>>>>>>   Tambet
>>>>>>
>>>>>> reede, 6. mai 2016 1:25.02 UTC+3 kirjutas nouiz:
>>>>>>
>>>>>>>
>>>>>>> Le 5 mai 2016 16:18, "Tambet Matiisen" <tambet....@gmail.com> a
>>>>>>> écrit :
>>>>>>> >
>>>>>>> > Thanks Fred for a hint! Following seems to work (I'm using K is
>>>>>>> the Keras equivalent of theano.tensor):
>>>>>>> >
>>>>>>> > a = K.zeros((3,3))
>>>>>>> > K.eval(a)
>>>>>>> >
>>>>>>> > array([[ 0.,  0.,  0.],
>>>>>>> >        [ 0.,  0.,  0.],
>>>>>>> >        [ 0.,  0.,  0.]], dtype=float32)
>>>>>>> >
>>>>>>> > <br><br><br>
>>>>>>> >
>>>>>>> > b = T.set_subtensor(a[[0,1,2],[0,1,2]], [1,2,3])
>>>>>>> >
>>>>>>> > K.eval(b)
>>>>>>> >
>>>>>>> >
>>>>>>> > array([[ 1.,  0.,  0.],
>>>>>>> >        [ 0.,  2.,  0.],
>>>>>>> >        [ 0.,  0.,  3.]], dtype=float32)
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > But what if I don't know the matrix dimensions beforehand? Can I
>>>>>>> produce the list of indexes to assign using just Theano arithmetics?
>>>>>>>
>>>>>>> Yes. There is T.a range(...) That you can use.
>>>>>>>
>>>>>>> Fred
>>>>>>> >
>>>>>>> >   Tambet
>>>>>>> >
>>>>>>> >
>>>>>>> > neljapäev, 5. mai 2016 17:30.08 UTC+3 kirjutas nouiz:
>>>>>>> >>
>>>>>>> >> The first idea I have is to init a vector of zeros of 9 element
>>>>>>> and use set_subtensor to set the indices to the value you want. Then
>>>>>>> reshape to a matrix.
>>>>>>> >>
>>>>>>> >> Fred
>>>>>>> >>
>>>>>>> >> On Thu, May 5, 2016 at 5:07 AM, Tambet Matiisen <
>>>>>>> tambet....@gmail.com> wrote:
>>>>>>> >>>
>>>>>>> >>> Hi everyone!
>>>>>>> >>>
>>>>>>> >>> I'm trying to apply T.diag() and T.tril() operations over batch
>>>>>>> of matrices, so that first dimension is preserved. Theano doesn't seem 
>>>>>>> to
>>>>>>> provide built-in function to do that. Is there any other way to achieve 
>>>>>>> the
>>>>>>> same?
>>>>>>> >>>
>>>>>>> >>> Basically I need to turn bunch of numbers x1, ... , x6 into a
>>>>>>> matrix like this:
>>>>>>> >>>
>>>>>>> >>> | e^x1    0    0 |
>>>>>>> >>> |   x2 e^x3    0 |
>>>>>>> >>> |   x4   x5 e^x4 |
>>>>>>> >>>
>>>>>>> >>> i.e. diagonal must be filled with e^xi and lower triangle must
>>>>>>> be filled with just xi. Order of x-s is not particularly important, as
>>>>>>> these are learned weights anyway.
>>>>>>> >>>
>>>>>>> >>> Thanks!
>>>>>>> >>> Tambet
>>>>>>> >>>
>>>>>>> >>> --
>>>>>>> >>>
>>>>>>> >>> ---
>>>>>>> >>> You received this message because you are subscribed to the
>>>>>>> Google Groups "theano-users" group.
>>>>>>> >>> To unsubscribe from this group and stop receiving emails from
>>>>>>> it, send an email to theano-users...@googlegroups.com.
>>>>>>> >>>
>>>>>>> >>> For more options, visit https://groups.google.com/d/optout.
>>>>>>> >>
>>>>>>> >>
>>>>>>> > --
>>>>>>> >
>>>>>>> > ---
>>>>>>> > You received this message because you are subscribed to the Google
>>>>>>> Groups "theano-users" group.
>>>>>>> > To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to theano-users...@googlegroups.com.
>>>>>>> > For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>> --
>>>>>>
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "theano-users" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to theano-users...@googlegroups.com.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>>
>>>> --
>>>>
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "theano-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to theano-users...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to