[theano-users] import theano issues in PCharm community edition

[Maradani Bhavana]

Hello All,

I am using PyCharm IDE and anaconda (python 3.5.2). I have installed theano 
using conda. I tested with a small program in PyCharm as well as Spyder. 
The program runs well in spyder but gives issues in PyCharm IDE. I tried to 
install theano package in pyCharm IDE, but doesn't help. Below is the error 
which I get in pyCharm:


C:\Users\Bhavana\Anaconda3\python.exe 
C:/Users/Bhavana/PycharmProjects/untitled/1.py
===
1 #include 
2 #include 
3 #include "theano_mod_helper.h"
4 #include 
5 #include 
6 #include 
7 #include 
8 #include 
9 //
00010   Support Code
00011 //
00012 
00013namespace {
00014struct __struct_compiled_op_mb7d4d181798f8e2bf5cb5c85493f6707 {
00015PyObject* __ERROR;
00016 
00017PyObject* storage_V3;
00018 PyObject* storage_V5;
00019 PyObject* storage_V1;
00020
00021 
00022__struct_compiled_op_mb7d4d181798f8e2bf5cb5c85493f6707() {
00023// This is only somewhat safe because we:
00024//  1) Are not a virtual class
00025//  2) Do not use any virtual classes in the members
00026//  3) Deal with mostly POD and pointers
00027 
00028// If this changes, we would have to revise this, but for
00029// now I am tired of chasing segfaults because
00030// initialization code had an error and some pointer has
00031// a junk value.
00032memset(this, 0, sizeof(*this));
00033}
00034~__struct_compiled_op_mb7d4d181798f8e2bf5cb5c85493f6707(void) {
00035cleanup();
00036}
00037 
00038int init(PyObject* __ERROR, PyObject* storage_V3, PyObject* 
storage_V5, PyObject* storage_V1) {
00039Py_XINCREF(storage_V3);
00040 Py_XINCREF(storage_V5);
00041 Py_XINCREF(storage_V1);
00042this->storage_V3 = storage_V3;
00043 this->storage_V5 = storage_V5;
00044 this->storage_V1 = storage_V1;
00045
00046 
00047 
00048 
00049 
00050this->__ERROR = __ERROR;
00051return 0;
00052}
00053void cleanup(void) {
00054__label_1:
00055 
00056 double __DUMMY_1;
00057 __label_3:
00058 
00059 double __DUMMY_3;
00060 __label_5:
00061 
00062 double __DUMMY_5;
00063 __label_8:
00064 
00065 double __DUMMY_8;
00066 
00067Py_XDECREF(this->storage_V3);
00068 Py_XDECREF(this->storage_V5);
00069 Py_XDECREF(this->storage_V1);
00070}
00071int run(void) {
00072int __failure = 0;
00073
00074PyObject* py_V1;
00075
00076PyArrayObject* V1;
00077
00078typedef npy_float64 dtype_V1;
00079
00080PyObject* py_V3;
00081
00082PyArrayObject* V3;
00083
00084typedef npy_float64 dtype_V3;
00085
00086PyObject* py_V5;
00087
00088PyArrayObject* V5;
00089
00090typedef npy_float64 dtype_V5;
00091
00092 {
00093 
00094py_V1 = Py_None;
00095{Py_XINCREF(py_V1);}
00096
00097V1 = NULL;
00098
00099 {
00100 
00101py_V3 = PyList_GET_ITEM(storage_V3, 0);
00102{Py_XINCREF(py_V3);}
00103
00104V3 = NULL;
00105if (py_V3 == Py_None) {
00106// We can either fail here or set V3 to NULL and rely 
on Ops
00107// using tensors to handle the NULL case, but if they 
fail to do so
00108// they'll end up with nasty segfaults, so this is 
public service.
00109PyErr_SetString(PyExc_ValueError, "expected an 
ndarray, not None");
00110{
00111__failure = 4;
00112if (!PyErr_Occurred()) {
00113PyErr_SetString(PyExc_RuntimeError,
00114"Unexpected error in an Op's C code. "
00115"No Python exception was set.");
00116}
00117goto __label_4;}
00118}
00119if (!PyArray_Check(py_V3)) {
00120PyErr_SetString(PyExc_ValueError, "expected an 
ndarray");
00121{
00122__failure = 4;
00123if (!PyErr_Occurred()) {
00124PyErr_SetString(PyExc_RuntimeError,
00125"Unexpected error in an Op's C code. "
00126"No Python exception was set.");
00127}
00128goto __label_4;}
00129}
00130// We expect NPY_FLOAT64
00131if (!PyArray_ISALIGNED((PyArrayObject*) py_V3)) {
00132PyArrayObject * tmp = (PyArrayObject*) py_V3;
00133PyErr_Format(PyExc_NotImplementedError,
00134 "expected an aligned array of type %ld "
00135 "(NPY_FLOAT64), got non-aligned array of 
type %ld"
00136 " with %ld dimensions, with 3 last dims "
00137 "%ld, %ld, 

Re: [theano-users] pygpu.gpuarray.GpuArrayException: Unknown device error: -1

[Øystein Schønning-Johansen]

Thanks for answering, Pascal.

Just before I ran deviceQuery and nvidia-smi, I did a reboot of the computer, 
and guess what?
[oystein@jupiter release]$ *python*
Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
[GCC 6.3.1 20170109] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> *import theano*
*Using cuDNN version 5105 on context None*
*Mapped name None to device cuda: GeForce GTX 1080 (:02:00.0)*

It works again! I guess the reboot helped.

I've no idea what triggered the bug in the first place. I might have had a 
nvidia driver upgrade pending a reboot or something.

Thanks anyway,
-Øystein

On Thursday, March 30, 2017 at 8:26:05 PM UTC+2, Pascal Lamblin wrote:
>
> Are you able to run other CUDA programs on that GPU, for instance 
> devicQuery (source included in cuda SDK)? 
> What does deviceQuery return, vs nvidia-smi? 
>
> On Thu, Mar 30, 2017, Øystein Schønning-Johansen wrote: 
> > 
> > 
> > Hi all! 
> > 
> > Today I got the 'strange' error message when importing theano. This has 
> worked perfectly before, 
> > 
> > I've built theano and libgpuarray from git sources. I've even recompiled 
> from the today's repo code, but I get the same error every time. 
> > 
> > Any hints? 
> > 
> > 
> > Thanks, 
> > 
> > -Øystein 
> > 
> > 
> > [oystein@jupiter libgpuarray-git]$ python 
> > Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
> > [GCC 6.3.1 20170109] on linux 
> > Type "help", "copyright", "credits" or "license" for more information. 
> > >>> import theano 
> > ERROR (theano.gpuarray): Could not initialize pygpu, support disabled 
> > Traceback (most recent call last): 
> >   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", 
> line 164, in  
> > use(config.device) 
> >   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", 
> line 151, in use 
> > init_dev(device) 
> >   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", 
> line 60, in init_dev 
> > sched=config.gpuarray.sched) 
> >   File "pygpu/gpuarray.pyx", line 614, in pygpu.gpuarray.init 
> (pygpu/gpuarray.c:9419) 
> >   File "pygpu/gpuarray.pyx", line 566, in pygpu.gpuarray.pygpu_init 
> (pygpu/gpuarray.c:9110) 
> >   File "pygpu/gpuarray.pyx", line 1021, in 
> pygpu.gpuarray.GpuContext.__cinit__ (pygpu/gpuarray.c:13472) 
> > pygpu.gpuarray.GpuArrayException: Unknown device error: -1 
> > >>> 
> > 
> > [oystein@jupiter libgpuarray-git]$ cat ~/.theanorc 
> > [global] 
> > floatX = float32 
> > device = cuda 
> > allow_gc = False 
> > [lib] 
> > cnmem=1.0 
> > 
> > 
> > -- 
> > 
> > --- 
> > You received this message because you are subscribed to the Google 
> Groups "theano-users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to theano-users...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
> -- 
> Pascal 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] [BUG or FEATURE?] gradient wrt shared after changing shape of storage

[Pascal Lamblin]

The problem is that your test values are no longer consistent with each other.

On Fri, Mar 24, 2017, Maxim Kochurov wrote:
> Hi, I'm trying to get gradient wrt shared variable after setting new value 
> and get an error. Am I doing something wrong?
> Minimal example to play with
> 
> from theano import theano, tensor as tt
> abc = tt.fvector()
> abc.tag.test_value = np.array([1.,2.,3.], dtype='float32')
> X = theano.shared(np.ones((4,3), 'float32'))
> y = tt.dot(X, abc)
> X.set_value(np.ones((1, 3), 'float32'))
> y.sum().eval({abc:np.array([1.,2.,3.], dtype='float32')}) # works
> g = theano.grad(y.sum(), X)
> 
> ...
> 
> ValueError: shapes (3,1) and (4,) not aligned: 1 (dim 1) != 4 (dim 0)
> 
> 
> theano.version 0.9.0.dev-dded8355679b510acf392ce35753cbc7856ba69f
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Error while executing Alexnet with theano v0.9.0

[Pascal Lamblin]

Hi,

You seem to be initializing both the new back-end and the old back-end
at the same time, which is not really supported.

Regarding the NotImplementedError, it is raised when calling T.min()
on something that is neither a signed int nor a float, as it currently
instantiates (-T.max(-x)), which may not be safe.

Here, the argument is the output of T.neq(...), which used to return an
int8, but now returns a bool.

I think the easiest fix for that code would be to use boolean operations
instead of min(), since the equivalent would be all(...).
The following should all work:

  T.mean(T.neq(y_pred_top_x, y_top_x).all(axis=1))
  T.mean(T.min(T.neq(y_pred_top_x, y_top_x).astype('int8'), axis=1))

On Tue, Mar 28, 2017, Deepali Patel wrote:
> Hello,
> 
> While executing alexnet with theano v0.9.0 I observed the following error:
> 
> THEANO_FLAGS=mode=FAST_RUN,floatX=float32 python train.py
> WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be 
> removed in the next release (v0.10).  Please switch to the gpuarray 
> backend. You can get more information about how to switch at this URL:
>  
> https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
> 
> Using cuDNN version 5105 on context None
> Mapped name None to device cuda: Tesla K80 (0004:03:00.0)
> Using gpu device 0: Tesla K80 (CNMeM is disabled, cuDNN 5105)
> ... building the model
> conv (cudnn) layer with shape_in: (3, 227, 227, 128)
> conv (cudnn) layer with shape_in: (96, 27, 27, 128)
> conv (cudnn) layer with shape_in: (256, 13, 13, 128)
> conv (cudnn) layer with shape_in: (384, 13, 13, 128)
> conv (cudnn) layer with shape_in: (384, 13, 13, 128)
> fc layer with num_in: 9216 num_out: 4096
> dropout layer with P_drop: 0.5
> fc layer with num_in: 4096 num_out: 4096
> dropout layer with P_drop: 0.5
> softmax layer with num_in: 4096 num_out: 1000
> Process Process-1:
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in 
> _bootstrap
> self.run()
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
> self._target(*self._args, **self._kwargs)
>   File "train.py", line 54, in train_net
> model = AlexNet(config)
>   File "/alexnet-theano/alex_net.py", line 130, in __init__
> self.errors_top_5 = softmax_layer8.errors_top_x(y, 5)
>   File "./lib/layers.py", line 312, in errors_top_x
> return T.mean(T.min(T.neq(y_pred_top_x, y_top_x), axis=1))
>   File "/site-packages/theano/tensor/basic.py", line 1742, in min
> raise NotImplementedError()
> NotImplementedError
> 
> 
> Please let me know if there is a known solution to this issue.
> 
> Regards
> Deepali 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] pygpu.gpuarray.GpuArrayException: Unknown device error: -1

[Pascal Lamblin]

Are you able to run other CUDA programs on that GPU, for instance devicQuery 
(source included in cuda SDK)?
What does deviceQuery return, vs nvidia-smi?

On Thu, Mar 30, 2017, Øystein Schønning-Johansen wrote:
> 
> 
> Hi all!
> 
> Today I got the 'strange' error message when importing theano. This has 
> worked perfectly before,
> 
> I've built theano and libgpuarray from git sources. I've even recompiled from 
> the today's repo code, but I get the same error every time.
> 
> Any hints?
> 
> 
> Thanks,
> 
> -Øystein
> 
> 
> [oystein@jupiter libgpuarray-git]$ python
> Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
> [GCC 6.3.1 20170109] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import theano
> ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
> Traceback (most recent call last):
>   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 
> 164, in 
> use(config.device)
>   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 
> 151, in use
> init_dev(device)
>   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 
> 60, in init_dev
> sched=config.gpuarray.sched)
>   File "pygpu/gpuarray.pyx", line 614, in pygpu.gpuarray.init 
> (pygpu/gpuarray.c:9419)
>   File "pygpu/gpuarray.pyx", line 566, in pygpu.gpuarray.pygpu_init 
> (pygpu/gpuarray.c:9110)
>   File "pygpu/gpuarray.pyx", line 1021, in 
> pygpu.gpuarray.GpuContext.__cinit__ (pygpu/gpuarray.c:13472)
> pygpu.gpuarray.GpuArrayException: Unknown device error: -1
> >>> 
> 
> [oystein@jupiter libgpuarray-git]$ cat ~/.theanorc
> [global]
> floatX = float32
> device = cuda
> allow_gc = False
> [lib]
> cnmem=1.0
> 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] pygpu.gpuarray.GpuArrayException: Unknown device error: -1

[Øystein Schønning-Johansen]

Hi all!

Today I got the 'strange' error message when importing theano. This has worked 
perfectly before,

I've built theano and libgpuarray from git sources. I've even recompiled from 
the today's repo code, but I get the same error every time.

Any hints?


Thanks,

-Øystein


[oystein@jupiter libgpuarray-git]$ python
Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
[GCC 6.3.1 20170109] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 
164, in 
use(config.device)
  File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 
151, in use
init_dev(device)
  File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 60, 
in init_dev
sched=config.gpuarray.sched)
  File "pygpu/gpuarray.pyx", line 614, in pygpu.gpuarray.init 
(pygpu/gpuarray.c:9419)
  File "pygpu/gpuarray.pyx", line 566, in pygpu.gpuarray.pygpu_init 
(pygpu/gpuarray.c:9110)
  File "pygpu/gpuarray.pyx", line 1021, in pygpu.gpuarray.GpuContext.__cinit__ 
(pygpu/gpuarray.c:13472)
pygpu.gpuarray.GpuArrayException: Unknown device error: -1
>>> 

[oystein@jupiter libgpuarray-git]$ cat ~/.theanorc
[global]
floatX = float32
device = cuda
allow_gc = False
[lib]
cnmem=1.0


-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] TypeError: ARange{dtype='int64'}.grad illegally returned an integer-valued variable

[Pascal Lamblin]

Right, that's a bug (https://github.com/Theano/Theano/issues/5790).

However, for some reason `seq` seems to be defined as int64, which I
doubt is what you want since you are taking a gradient wrt it, which
would be 0.

Moreover, the expression for the gradient does not look right. It
currently returns 0 for the boundaries, which is not correct.
It should be 0 for param2 (changing it by epsilon would not change
the output most of the time), and the sum of the output gradients for
param1, at least when step > 0, which is your case.

On Mon, Mar 27, 2017, Julia K wrote:
> Hi,
> 
> I'm getting an error when trying to get the gradient of a function. 
> 
> Basically, I want to use two parameters to define the lower and upper bound 
> of an iterator.
> For easier understanding I simplified my code:
> 
> from theano import theano, tensor as tt
> 
> def get_sum(param1, param2):
> seq = tt.arange(param1, param2)
> 
> components, _ = theano.scan(lambda index: index**2,
>outputs_info=None,
>sequences=[seq])
> return tt.sum(components)
> 
> 
> param1 = tt.dscalar('param1')
> param1.tag.test_value = 0.0
> 
> param2 = tt.dscalar('param2')
> param2.tag.test_value = 10.0
> 
> objective = get_sum(param1, param2)
> 
> obj_function = theano.function(inputs=[param1, param2],outputs=objective)
> 
> obj_gradient = theano.gradient.jacobian(objective,[param1])
> gradient_function = theano.function(inputs=[param1, param2], 
> outputs=obj_gradient)
> 
> ---
> 
> ...
> 
> TypeError: ARange{dtype='int64'}.grad illegally  returned an integer-valued 
> variable. (Input index 0, dtype int64)
> 
> 
> The function that is called in theano.scan (which is index**2 in the 
> example) could be any function that uses the index as one of its parameters.
> 
> What do I need to change about the code to make it work?
> 
> The same error occurs when using theano version 0.8.2 or 
> 0.9.0.dev-b3d19ce8e8d7cdf87c3f84599be81647803aa1f6
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] How to get gradients from Theano in numpy type?

[Pascal Lamblin]

If you enabled the GPU back-end, then shared variables will live on the GPU, 
and the gradients will have the same type.

You have two options at this time:
- either call `.transfer('cpu')` on the symbolic gradients before using them as 
outputs
- call numpy.asarray() on the GpuArrays returned by the Theano function

On Mon, Mar 27, 2017, duy kien nguyen wrote:
> 
> Hi all,
> 
> I tried to get the gradients from Theano in numpy type. But, when I defined 
> the function, it returned pygpu.gpuarray.GpuArray. How can I solve this 
> problem?
> 
> Here is my function:
> 
> cost  = classifier.negative_log_likelihood(answer, answer_mask)
> grads = T.grad(cost, list(classifier.tparams.values()))
> 
> optim = optimizer()
> # updates = optim(itemlist(classifier.tparams), grads)
> 
> train_model = theano.function(
> inputs=[],
> outputs=[cost] + grads,
> # updates=updates,
> givens={
> img: shared_img, ques: shared_ques_casted, ans: shared_ans_casted[:-1],
> m_ques: shared_m_ques, m_ans: shared_m_ans[:-1],
> answer: shared_ans_casted[1:], answer_mask: shared_m_ans[1:]
> }
> )
> 
> test_model = theano.function(
> inputs=[],
> outputs=classifier.samples(length_answer-1),
> givens={
> img: shared_img, ques: shared_ques_casted,
> m_ques: shared_m_ques
> }
> )
> --
> 
> Here is the type of the returned gradients (the first one is in numpy type 
> but the other are not):
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Thank you in advance
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Differentiating w.r.t matrix row results in DisconnectedInputError

[Pascal Lamblin]

You can take a subtensor of the whole gradient instead, it should be 
mathematically equivalent.

It may not get completely optimized, though, and parts of the whole gradient 
may be computed and end up not being used.

On Thu, Mar 30, 2017, Jan Kukačka wrote:
> Dear all,
> 
> I have the following problem: When I want to differentiate a row of an 
> input matrix w.r.t. a row of an output matrix, I get a 
> DisconnectedInputError. The minimum scenario is in the attached file. I 
> suspect theano does not consider theta and theta[i] to be the same 
> variable, rather take theta[i] to be a different variable depending on 
> theta.
> 
> Long story:
> I have a neural network, using M-dimensional input and N-dimensional output 
> and having batch size B. Input matrix for my net is then (B,M) dimensional 
> and the output matrix is (B,N) dimensional. For every batch element I want 
> to compute the jacobbian of the network, that is, M*N matrix with gradients 
> of element outputs depending on the element inputs. In the scan function I 
> can compute jacobbians of individual rows wrt. whole input (thus having B 
> matrices of size (B*M, N) but when I try computing the jacobbian only wrt a 
> row of the input, I get the DisconnectedInputError.
> 
> The question is:
> Is there a way to differentiate w.r.t. a subtensor?
> 
> Best regards,
> Jan
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

> import theano
> from theano import tensor as T
> import numpy as np
> 
> n_samples = 10
> theta_single = T.as_tensor_variable([0.2, 0.15, 0.4, 0.05, 0.2])
> theta = T.tile(theta_single,(n_samples, 1))
> print theta_single.eval().shape
> print theta.eval().shape
> 
> # differentiating with the original theta_single
> J1, _ = theano.scan(lambda i, theta, theta_single : 
> theano.gradient.jacobian(theta[i,:], theta_single),
> sequences=T.arange(theta.shape[0]),
> non_sequences=[theta, theta_single])
> J1.eval()
> 
> # same as before, just differentiating wrt to single row of theta taken from 
> that same array\n",
> J2, _ = theano.scan(lambda i, theta, theta_single : 
> theano.gradient.jacobian(theta[i,:], theta[i,:]),
> sequences=T.arange(theta.shape[0]),
> non_sequences=[theta, theta_single])
> J2.eval()


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: Generating my own data like mnist.pkl.gz

[Frédéric Bastien]

What is set_x and rp? If set_x is a scalar, you can't index into it.

Fred

On Thu, Mar 12, 2015 at 12:25 PM Aloïs  wrote:

> Hi,
>
> I chose to use H5 files.
> Here's an example script to generate and read these files :
> https://gist.github.com/aloisg/ac83160edf8a543b5ee6
>
> The generate_h5.py should work without any changes on your data.
>
> Aloïs
>
>
> Le jeudi 12 mars 2015 15:02:21 UTC+1, Henrik Pedersen a écrit :
>
> Hi,
> Say, I have a set of image files divided into different classes. The class
> of each image could be indicated by the filename or the name of the folder
> in which the image is located.
>
> What is the easiest way to generate a data file like mnist.pkl.gz?
>
> Thx.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: How to improve resolution reduction of frequency axis of FFT data

[Robb Brown]

I think you may have difficulty getting the result you want with your current 
approach. The signal and the spectrum are really the same thing, but 
transformed using the FFT. That means that any operation on one implies a dual 
operation on the other.  Downsampling the frequency spectrum implies decreasing 
the field of view of the signal.  So effectively what you’re doing is limiting 
the ANN’s view of the signal to some constant sized window, probably in the 
centre of the signal. I assume you want to noise filter the entire signal, not 
just the middle of it.

There are two approaches I'd consider:  1) you could use a fully convolutional 
ANN, which can handle different sized input. 2) you could perform a short time 
Fourier transform with fixed window sizes, and have your fully connected ANN 
operate on the individual STFT frames.

For audio recordings, I would probably use the STFT approach if you want to 
operate in the frequency domain, or just implement a fully convolutional 
network in the signal domain.  Because your audio recording is not stationary 
(the frequency content changes over time) it will be difficult to effectively 
filter the global frequency spectrum (as opposed to the local ones provided by 
the STFT).


> On Mar 30, 2017, at 4:00 AM, Torsten Knodt  
> wrote:
> 
> Hello Rob,
> sorry for answering late. Somehow I did not see the notification.
> The signal comes from audio recordings of very different length and sample 
> rate. I am not really sure whether this very simple way of fingerpinting 
> works at the end or I need a sliding window over time at the end which would 
> also result in a fixed resolution. As this would increase the amount of data 
> and complicate the further prpcessing, I first chose the other way un the 
> hope the result is good enough.
> At tje end the data shall pre-train an adaptive notch filter to remove 
> (changing) background noise. The filter will run on an low power embedded 
> system.
> So far this is just a vague idea. I had several applications in the past 
> where I would have required the solution, outside of audio.
> 
> Robb Brown > schrieb am 
> Mo., 27. März 2017 23:50:
> This seems like a very odd thing to be doing.  Downsampling the frequency 
> spectrum is (in the ideal case) equivalent to decreasing the field of view of 
> the FFT on the original signal.  Why don't you just decrease the size of your 
> original FFT?  That will make the FFT faster, and make your downsampling 
> operation unnecessary.
> 
> 
> On Monday, March 27, 2017 at 12:26:37 PM UTC-4, Torsten Knodt wrote:
> Hello,
> I am totally new to theano and have a first implementation to reduce the 
> resolution of the frequency axis of FFT data. My function basically sums 
> multiple consecutive bins to one, but also handles cases where the target 
> resolution is no integer multiple of the source resolution.
> from math import log2
> from numpy import 
> array,arange,asarray,ceil,clip,empty,floor,trunc,frombuffer,save
> from theano import shared,function
> from theano.tensor import matrix,scalar,vector
> 
> def resample(self, input_data, fsi, fso, fdo):
>   fsi = float(fsi)
>   fdi = fsi/len(input_data)
>   output_data = empty(shape=int(ceil(fso/fdo)))
>   for o in range(len(output_data)):
> fmino = o*fdo
> fnexto = (o+1)*fdo
> _v = vector()
> _i = scalar(dtype='int64')
> _fdi = scalar()
> _fmino = scalar()
> _fnexto = scalar()
> _f1 = 
> function([_i,_fdi,_fmino,_fnexto,_v],(((_i+1)*_fdi).clip(_fmino,_fnexto)-(_i*_fdi).clip(_fmino,_fnexto))/_fdi*_v[_i])
> output_data[o] = sum([_f1(i,fdi,fmino,fnexto,input_data) for i in 
> range(int(clip(floor(fmino/fdi),0,len(input_data)-1)),int(clip(ceil(fnexto/fdi),
>  1, len(input_data])
>   output_data = output_data / output_data.sum()
>   return output_data
> 
> The background is, that I need fixed size FFT data to train a neuronal 
> network which shall do some classification of the input.
> What would be your recommendations to improve the performance of the 
> function? Improvements to efficiently handle multiple datasets of different 
> source frequency resolutions would be of special interest. Currently I only 
> work on my CPU, but plan to move to GPU later.
> input_data is already a numpy array. fsi specifies the maximum input 
> frequency, while fso specifies the maximum output frequency. fdo specifies 
> the requested resolution of the output.
> 
> Thanks in Advance
>  Torsten Knodt
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to a topic in the Google 
> Groups "theano-users" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/theano-users/cmJB8_0HB-k/unsubscribe 
> .
> To unsubscribe from this group and all its topics, send an email to 
> 

[theano-users] Re: AttributeError: 'TensorType' object has no attribute 'shape'

[司马井上]

has been solved

On Thursday, 30 March 2017 15:11:52 UTC+2, 司马井上 wrote:
>
> import theano.tensor as tt
>
> var=tt.itensor3
> cvar=tt.flatten(var,outdim=2)
>
> I run it and get AttributeError: 'TensorType' object has no attribute 
> 'shape'
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] AttributeError: 'TensorType' object has no attribute 'shape'

[司马井上]

import theano.tensor as tt

var=tt.itensor3
cvar=tt.flatten(var,outdim=2)

I run it and get AttributeError: 'TensorType' object has no attribute 
'shape'

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Differentiating w.r.t matrix row results in DisconnectedInputError

[Jan Kukačka]

Dear all,

I have the following problem: When I want to differentiate a row of an 
input matrix w.r.t. a row of an output matrix, I get a 
DisconnectedInputError. The minimum scenario is in the attached file. I 
suspect theano does not consider theta and theta[i] to be the same 
variable, rather take theta[i] to be a different variable depending on 
theta.

Long story:
I have a neural network, using M-dimensional input and N-dimensional output 
and having batch size B. Input matrix for my net is then (B,M) dimensional 
and the output matrix is (B,N) dimensional. For every batch element I want 
to compute the jacobbian of the network, that is, M*N matrix with gradients 
of element outputs depending on the element inputs. In the scan function I 
can compute jacobbians of individual rows wrt. whole input (thus having B 
matrices of size (B*M, N) but when I try computing the jacobbian only wrt a 
row of the input, I get the DisconnectedInputError.

The question is:
Is there a way to differentiate w.r.t. a subtensor?

Best regards,
Jan

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
import theano
from theano import tensor as T
import numpy as np

n_samples = 10
theta_single = T.as_tensor_variable([0.2, 0.15, 0.4, 0.05, 0.2])
theta = T.tile(theta_single,(n_samples, 1))
print theta_single.eval().shape
print theta.eval().shape

# differentiating with the original theta_single
J1, _ = theano.scan(lambda i, theta, theta_single : theano.gradient.jacobian(theta[i,:], theta_single),
sequences=T.arange(theta.shape[0]),
non_sequences=[theta, theta_single])
J1.eval()

# same as before, just differentiating wrt to single row of theta taken from that same array\n",
J2, _ = theano.scan(lambda i, theta, theta_single : theano.gradient.jacobian(theta[i,:], theta[i,:]),
sequences=T.arange(theta.shape[0]),
non_sequences=[theta, theta_single])
J2.eval()

Re: [theano-users] Straight-through gradient for stochastic node in theano

[Yoh Okuno]

Thank you! I will try it.

On Thursday, March 30, 2017 at 12:58:34 AM UTC+1, Pascal Lamblin wrote:
>
> You can try OpFromGraph with inline=True, and specify 
> override_gradients=... with the right expression. 
> This is still experimental. 
>
> On Wed, Mar 29, 2017, nokun...@gmail.com  wrote: 
> > Hi guys! 
> > 
> > I recently started using theano and am struggling to implement custom 
> > gradient for stochastic node. Can anyone help me? 
> > 
> > What I want is an op that produces one hot vector whose hot element is 
> > sampled from softmax distribution. 
> > The op is not differentiable, but I want to "fake" as if its gradient is 
> > softmax's one ("straight through estimator"). 
> > Below is the minimum code that perform forward path, which raises 
> > DisconnectedInputError due to missing gradient. 
> > 
> > 
> > import theano 
> > 
> > import theano.tensor as T 
> > 
> > import numpy as np 
> > 
> > 
> > logits_values = np.random.uniform(-1, 1, size=3) 
> > 
> > logits = theano.shared(logits_values, 'logits') 
> > 
> > probabilities = T.nnet.softmax(logits) 
> > 
> > print('probabilities', probabilities.eval()) 
> > 
> > # result: probabilities [[ 0.55155489  0.2907730.15767211]] 
> > 
> > 
> > random_streams = T.shared_randomstreams.RandomStreams() 
> > 
> > index = random_streams.choice(size=(1,), a=3, p=probabilities[0]) 
> > 
> > samples = T.extra_ops.to_one_hot(index, logits.shape[-1]) 
> > 
> > print('samples', samples.eval()) 
> > 
> > # result: samples [[ 1.  0.  0.]] 
> > 
> > 
> > # We want to use gradient of probabilities instead of samples! 
> > 
> > samples_grad = T.grad(samples[0][0], logits) 
> > 
> > # result: raise DisconnectedInputError 
> > 
> > 
> > The node is not the final layer, so I can't use categorical cross 
> entropy loss for training it. 
> > 
> > I am trying to implement custom op (see attached stochastic_softmax.py), 
> but it is not working in practice. 
> > 
> > Since I have working expression for forward path, can I simply override 
> gradient of existing expression? 
> > 
> > -- 
> > 
> > --- 
> > You received this message because you are subscribed to the Google 
> Groups "theano-users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to theano-users...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>
> > import numpy as np 
> > import theano 
> > import theano.tensor as T 
> > 
> > 
> > class StochasticSoftmax(theano.Op): 
> > def __init__(self, random_state=np.random.RandomState()): 
> > self.random_state = random_state 
> > 
> > def make_node(self, x): 
> > x = T.as_tensor_variable(x) 
> > return theano.Apply(self, [x], [x.type()]) 
> > 
> > def perform(self, node, inputs, output_storage): 
> > # Gumbel-max trick 
> > x, = inputs 
> > z = self.random_state.gumbel(loc=0, scale=1, size=x.shape) 
> > indices = (x + z).argmax(axis=-1) 
> > y = np.eye(x.shape[-1], dtype=np.float32)[indices] 
> > output_storage[0][0] = y 
> > 
> > def grad(self, inp, grads): 
> > x, = inp 
> > g_sm, = grads 
> > 
> > sm = T.nnet.softmax(x) 
> > return [T.nnet.softmax_grad(g_sm, sm)] 
> > 
> > def infer_shape(self, node, i0_shapes): 
> > return i0_shapes 
>
>
> -- 
> Pascal 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: How to improve resolution reduction of frequency axis of FFT data

[Torsten Knodt]

Hello Rob,
sorry for answering late. Somehow I did not see the notification.
The signal comes from audio recordings of very different length and sample
rate. I am not really sure whether this very simple way of fingerpinting
works at the end or I need a sliding window over time at the end which
would also result in a fixed resolution. As this would increase the amount
of data and complicate the further prpcessing, I first chose the other way
un the hope the result is good enough.
At tje end the data shall pre-train an adaptive notch filter to remove
(changing) background noise. The filter will run on an low power embedded
system.
So far this is just a vague idea. I had several applications in the past
where I would have required the solution, outside of audio.

Robb Brown  schrieb am Mo., 27. März 2017 23:50:

> This seems like a very odd thing to be doing.  Downsampling the frequency
> spectrum is (in the ideal case) equivalent to decreasing the field of view
> of the FFT on the original signal.  Why don't you just decrease the size of
> your original FFT?  That will make the FFT faster, and make your
> downsampling operation unnecessary.
>
>
> On Monday, March 27, 2017 at 12:26:37 PM UTC-4, Torsten Knodt wrote:
>
> Hello,
> I am totally new to theano and have a first implementation to reduce the
> resolution of the frequency axis of FFT data. My function basically sums
> multiple consecutive bins to one, but also handles cases where the target
> resolution is no integer multiple of the source resolution.
> from math import log2
> from numpy import array,arange,asarray,ceil,clip,empty,floor,trunc,
> frombuffer,save
> from theano import shared,function
> from theano.tensor import matrix,scalar,vector
>
> def resample(self, input_data, fsi, fso, fdo):
>   fsi = float(fsi)
>   fdi = fsi/len(input_data)
>   output_data = empty(shape=int(ceil(fso/fdo)))
>   for o in range(len(output_data)):
> fmino = o*fdo
> fnexto = (o+1)*fdo
> _v = vector()
> _i = scalar(dtype='int64')
> _fdi = scalar()
> _fmino = scalar()
> _fnexto = scalar()
> _f1 = function([_i,_fdi,_fmino,_fnexto,_v],(((_i+1)*_fdi).clip(_fmino,
> _fnexto)-(_i*_fdi).clip(_fmino,_fnexto))/_fdi*_v[_i])
> output_data[o] = sum([_f1(i,fdi,fmino,fnexto,input_data) for i in
> range(int(clip(floor(fmino/fdi),0,len(input_data)-1)),int(clip(ceil(fnexto
> /fdi), 1, len(input_data])
>   output_data = output_data / output_data.sum()
>   return output_data
>
> The background is, that I need fixed size FFT data to train a neuronal
> network which shall do some classification of the input.
> What would be your recommendations to improve the performance of the
> function? Improvements to efficiently handle multiple datasets of different
> source frequency resolutions would be of special interest. Currently I only
> work on my CPU, but plan to move to GPU later.
> input_data is already a numpy array. fsi specifies the maximum input
> frequency, while fso specifies the maximum output frequency. fdo specifies
> the requested resolution of the output.
>
> Thanks in Advance
>  Torsten Knodt
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "theano-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/theano-users/cmJB8_0HB-k/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.