Thank you Fred for your help, your advises and resources really helped me
learn. I implemented n as a prop and rewrote grad() without the second
element which helped streamline the code.
Also I realized that I was using [g] instead of [g[0]] when calculating the
gradient which was causing the original error.
A follow up question. I am unfamiliar with c_code implementation, and I
tried to use the examples in scalar_ops as a template for my own, below is
what I have so far. I set theano to use GPU, but when I ran it I didn't see
"test" in terminal so I'm pretty sure the c_code did not run. I's there a
way to test/debug this? I apologize if this is too broad of a question, I
am rather new at theano.
def c_code(self, node, name, (x, ), (x_quantized, ), sub):
if node.inputs[0].type in float_types:
n = self.n
print "test"
return """
double exponent = %(n)s
%(x_quantized)s = round(( %(x)s * pow( 2.0, exponent ) / pow( 2.0,
exponent );""" % locals()
raise NotImplementedError('only floating point is implemented')
Thank you again for your help/time.
Kan
On Monday, July 25, 2016 at 9:43:16 PM UTC-7, nouiz wrote:
>
> The problem is the grad method.
>
> The second element it return must be a Theano variable and not a python
> object. If you really want to return 0, you can reutrn
> theano.tensor.constant(0, dtype=...) instead.
>
> Also, I'm not sure it return the right think. Should it be grad_undefined
> instead?
>
> http://deeplearning.net/software/theano/extending/op.html#grad
>
> Also, do you want that operator to also work on GPU? If so, you are lucky,
> you don't need to write GPU code! You can reuse the Elemwise code generator
> and just write the code for 1 element. This will get also optimized by in
> many cases too (CPU and GPU) like by doing implace version.. See:
>
> http://deeplearning.net/software/theano/extending/other_ops.html#scalar-ops
>
> Here, you will need the n-bit parameter to not be an input of the node,
> but a __props__:
>
>
> http://deeplearning.net/software/theano/extending/extending_theano.html#example-props-definition
>
> Fred
>
> On Mon, Jul 25, 2016 at 7:37 PM, Kan Kawabata <[email protected]
> <javascript:>> wrote:
>
>> *Hello, I am trying to create a custom operation that would round a tensor
>> variable to n-bits. (e.g. f(0.5238472) returns .5 for 2bit rounding)
>>
>> I was able to do this but I want to define the gradient of this operation to
>> be the same as the un-rounded expression.
>>
>> Below is the code and test that I performed. I get an attributeError when I
>> try to compute dy = T.grad(y,x) (please see the full error below).
>>
>>
>> *
>>
>> import theano
>> import theano.tensor as T
>> import numpy
>> #theano.config.exception_verbosity = 'high'
>>
>>
>> class Quantize(theano.Op):
>> # Properties attribute
>> __props__ = ()
>>
>> #itypes and otypes attributes are
>> #compulsory if make_node method is not defined.
>> #They're the type of input and output respectively
>> # itypes = [theano.tensor.scalar, theano.tensor.iscalar]
>> # otypes = [theano.tensor.scalar]
>>
>>
>> #Compulsory if itypes and otypes are not defined
>> def make_node(self, x, n):
>> x = T.as_tensor_variable(x)
>> return theano.Apply(self, [x, n], [x.type.make_variable()])
>>
>> # Python implementation:
>> def perform(self, node, inputs_storage, output_storage):
>> x, n = inputs_storage
>> x_quantized, = output_storage
>> x_quantized[0] = numpy.round(x*2**n)/2**n
>>
>> def grad(self, inputs, g):
>> return [g, 0]
>>
>> def infer_shape(self, node, input_shapes):
>> return [input_shapes[0]]
>>
>> if __name__ == "__main__":
>> x = T.dscalar()
>> n = T.iscalar()
>> y = Quantize()(x, n)
>> f = theano.function([x, n], y)
>>
>> a = numpy.random.rand()
>>
>> print a
>>
>> print f(a, 4)
>>
>> dy = T.grad(y, x)
>> df = theano.function([x], dy)
>>
>> print df(a)
>>
>>
>> error below
>>
>> 0.0151228405317
>> 0.0
>> Traceback (most recent call last):
>> File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition
>> 2016.1.4\helpers\pydev\pydevd.py", line 1531, in <module>
>> globals = debugger.run(setup['file'], None, None, is_module)
>> File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition
>> 2016.1.4\helpers\pydev\pydevd.py", line 938, in run
>> pydev_imports.execfile(file, globals, locals) # execute the script
>> File "C:/Users/Coor3427_5JYQJB2/Dropbox
>> (ASU)/MachineLearning/Pseudo-Ensembles/quantize.py", line 49, in <module>
>> dy = T.grad(y, x)
>> File "C:\Anaconda2\lib\site-packages\theano\gradient.py", line 561, in grad
>> grad_dict, wrt, cost_name)
>> File "C:\Anaconda2\lib\site-packages\theano\gradient.py", line 1324, in
>> _populate_grad_dict
>> rval = [access_grad_cache(elem) for elem in wrt]
>> File "C:\Anaconda2\lib\site-packages\theano\gradient.py", line 1279, in
>> access_grad_cache
>> term = access_term_cache(node)[idx]
>> File "C:\Anaconda2\lib\site-packages\theano\gradient.py", line 1190, in
>> access_term_cache
>> if not isinstance(term.type,
>> AttributeError: 'list' object has no attribute 'type'
>>
>>
>> Thank you in advance,
>>
>> Kan Kawabata
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "theano-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
