Hi,
We are currently working on a project porting complex gradients to theano
using the Wirtinger Calculus.
So far we have written a couple of ops performing complex math. operations
and
gradient calculations using real valued cost functions. All gradients of
these functions are numerically tested.
At the moment we are interested in making those ops faster, as in the
forward path only python is used.
We want to use gpu calculations also in forward mode. Most importantly, we
will code CUDA code only for
selected ops, as most operations will be runnable using theano's base math
functionality (T.xx()).
The problem arises when I want to use those base GPU ops. Using them in
perform() will not solve the problem as we are in
np format. I was not able to figure out how to use make_thunk in
combination with T.xx(). My solution for this problem is to calculate the
T.xx()
in make_node and pass it to Apply(). Theno the theano.op will be calculated
on the gpu (if supported) and will be passed to perform. I really don't
like this solution, so my question is: Is there a better way to use theano
math (T.xx()) ops in forward() mode when writing theano.ops? I think there
are only code examples using CUDA code
#-------------------------------------------------------------------------------
import theano
import theano.tensor as T
import numpy as np
class cDot (theano.Op):
__props__ = ()
def make_node(self, x, y):
x = theano.tensor.as_tensor_variable(x)
y = theano.tensor.as_tensor_variable(y)
z = T.dot(x,y)
return theano.Apply(self, [x, y, z], [x.type()])
def perform(self, node, inputs, output_storage):
x,y,z = inputs
z[0] = output_storage[0]
z[0] = z
'''
# standard np calculation
x,y,_ = inputs
z = output_storage[0]
z[0] = np.dot(x,y)
'''
def grad(self, inputs, output_grads):
x, y, _ = inputs
g = output_grads[0]
#see 4.42 and 4.43
x_out = T.dot(g, T.conj(y.T))
y_out = T.dot(T.conj(x.T), g)
return [T.cast((x_out), 'complex64'), T.cast((y_out), 'complex64'),
T.zeros_like(x)]
cdot = cDot()
#-------------------------------------------------------------------------------
Thanks in advance
Matthias
ps.: we will share complex op code after submitting the paper. Maybe we
could merge it into the master branch, but a lot of work has to be done
before regarding interfaces etc.
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
