Re: [theano-users] Re: Cannot do a simple theano install (Python 2.7, Ubuntu 16.04, Theano 0.9, Cuda 8.0, TitanX GPU) due to pygpu errors

[Pascal Lamblin]

On Friday, June 30, 2017 at 9:50:58 PM UTC-4, Daniel Seita wrote:
>
> your plan worked!
>

Nice to hear that :)
 

> Pascal and nouiz, just one last thing, this GPU backend is the first time 
> that Theano can run float64 with the GPU, right? I'm hoping to take 
> existing code which uses float64 only and running it on the GPU. (That's 
> why I've been using the CPU recently, to use float64.)
>

Yes, indeed.
Note that float64 performance can be much slower than float32, depending on 
the GPU. For instance, on a Titan X (Pascal), it would be 32x slower than 
float32, K40 and K80 should be only 3x slower, and P100 / GP100 2x slower.
 

>
>
>
> On Friday, June 30, 2017 at 6:48:08 PM UTC-7, Daniel Seita wrote:
>>
>> Hi nouiz and Pascal, thanks for the responses. I've been busy using the 
>> CPU version of Theano in the meantime, so sorry for the delay in responding.
>>
>> nouiz:
>>
>> I actually had my `cudnn.h` file in both a `lib64` directory and an 
>> `include` directory:
>>
>> ~$ ls -lh /usr/local/cuda-8.0/include/cudnn.h 
>> -r--r--r-- 1 root root 98K Oct 17  2016 /usr/local/cuda-8.0/include/cudnn
>> .h
>> ~$ ls -lh /usr/local/cuda-8.0/lib64/cudnn.h 
>> -r--r--r-- 1 root root 98K Oct 17  2016 /usr/local/cuda-8.0/lib64/cudnn.h
>>
>> I must have copied them to both when I was installing it. Also, here is 
>> my error message in full, assuming that my `~/.theanorc` file is
>>
>> ~$ cat ~/.theanorc
>> [global]
>> device = cuda
>> floatX = float64
>>
>> [cuda]
>> root = /usr/local/cuda-8.0
>> ~$ ipython
>> Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) 
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 5.3.0 -- An enhanced Interactive Python.
>> ? -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help  -> Python's own help system.
>> object?   -> Details about 'object', use 'object??' for extra details.
>>
>> In [1]: import theano
>> ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
>> Traceback (most recent call last):
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
>> , line 164, in 
>> use(config.device)
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
>> , line 151, in use
>> init_dev(device)
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
>> , line 68, in init_dev
>> context.cudnn_handle = dnn._make_handle(context)
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
>> , line 80, in _make_handle
>> cudnn = _dnn_lib()
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
>> , line 67, in _dnn_lib
>> raise RuntimeError('Could not find cudnn library (looked for v5[.1])'
>> )
>> RuntimeError: Could not find cudnn library (looked for v5[.1])
>>
>> This happens if I also set the device to be `cuda0` instead of `cuda`, 
>> and trying with `float32` instead of `float64`.
>>
>>
>>
>>
>> On Friday, June 30, 2017 at 5:43:35 PM UTC-7, Pascal Lamblin wrote:
>>>
>>> Something weird seems to be happening:
>>> - theano detects that cuDNN is available, in fact it called 
>>> _dnn_check_compile() and _dnn_check_version() successfully
>>> - however, calling _dnn_lib() failed, which means ctypes did not manage 
>>> to find the cudnn library.
>>>
>>> Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to 
>>> LD_LIBRARY_PATH)?
>>>
>>> On Monday, June 19, 2017 at 6:16:37 PM UTC-4, nouiz wrote:

 Your cudnn.h file should not be in the lib64 directory, but in an 
 include directory. Tensorflow does none standard stuff related to import 
 and cause problem in other setup, but it seem to tolerate your non 
 standard 
 setup. Theano does the standard setup.

 You can use the Theano flag dnn.include_path and dnn.library_path to 
 tell Theano where your cudnn.h and cudnn.so* files are.

 I did not see your last error in full.

 Le ven. 16 juin 2017 19:35, Daniel Seita  a 
 écrit :

> Ack, sorry, half of my post got deleted! Hopefully you can still see 
> it (i can find it by looking at the original post but it's in a really 
> ugly 
> format, sorry).
>
>
>
> On Friday, June 16, 2017 at 4:33:20 PM UTC-7, Daniel Seita wrote:
>
>> I was running into some more difficulties, so I gave up on getting 
>> this to work and tried to uninstall and then reinstall Theano. Just to 
>> be 
>> extra clear, here is my setup:
>>
>>- Ubuntu 16.04
>>- Cuda 8.0, stored in `usr/local/cuda-8.0`
>>- Titan X GPU with Pascal
>>
>> cuDNN is here:
>>
>> $ ls /usr/local/cuda-8.0/lib64/cudnn.h 
>> /usr/local/cuda-8.0/lib64/cudnn.h
>>
>> To verify that I can use my GPU I started this quick TensorFlow 

Re: [theano-users] Re: Cannot do a simple theano install (Python 2.7, Ubuntu 16.04, Theano 0.9, Cuda 8.0, TitanX GPU) due to pygpu errors

[Daniel Seita]

Hi nouiz and Pascal, thanks for the responses. I've been busy using the CPU 
version of Theano in the meantime, so sorry for the delay in responding.

nouiz:

I actually had my `cudnn.h` file in both a `lib64` directory and an 
`include` directory:

~$ ls -lh /usr/local/cuda-8.0/include/cudnn.h 
-r--r--r-- 1 root root 98K Oct 17  2016 /usr/local/cuda-8.0/include/cudnn.h
~$ ls -lh /usr/local/cuda-8.0/lib64/cudnn.h 
-r--r--r-- 1 root root 98K Oct 17  2016 /usr/local/cuda-8.0/lib64/cudnn.h

I must have copied them to both when I was installing it. Also, here is my 
error message in full, assuming that my `~/.theanorc` file is

~$ cat ~/.theanorc
[global]
device = cuda
floatX = float64

[cuda]
root = /usr/local/cuda-8.0
~$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) 
Type "copyright", "credits" or "license" for more information.

IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help  -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
  File 
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in 
use(config.device)
  File 
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
  File 
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
  File 
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py", 
line 80, in _make_handle
cudnn = _dnn_lib()
  File 
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py", 
line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])

This happens if I also set the device to be `cuda0` instead of `cuda`, and 
trying with `float32` instead of `float64`.




On Friday, June 30, 2017 at 5:43:35 PM UTC-7, Pascal Lamblin wrote:
>
> Something weird seems to be happening:
> - theano detects that cuDNN is available, in fact it called 
> _dnn_check_compile() and _dnn_check_version() successfully
> - however, calling _dnn_lib() failed, which means ctypes did not manage to 
> find the cudnn library.
>
> Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to 
> LD_LIBRARY_PATH)?
>
> On Monday, June 19, 2017 at 6:16:37 PM UTC-4, nouiz wrote:
>>
>> Your cudnn.h file should not be in the lib64 directory, but in an include 
>> directory. Tensorflow does none standard stuff related to import and cause 
>> problem in other setup, but it seem to tolerate your non standard setup. 
>> Theano does the standard setup.
>>
>> You can use the Theano flag dnn.include_path and dnn.library_path to tell 
>> Theano where your cudnn.h and cudnn.so* files are.
>>
>> I did not see your last error in full.
>>
>> Le ven. 16 juin 2017 19:35, Daniel Seita > > a écrit :
>>
>>> Ack, sorry, half of my post got deleted! Hopefully you can still see it 
>>> (i can find it by looking at the original post but it's in a really ugly 
>>> format, sorry).
>>>
>>>
>>>
>>> On Friday, June 16, 2017 at 4:33:20 PM UTC-7, Daniel Seita wrote:
>>>
 I was running into some more difficulties, so I gave up on getting this 
 to work and tried to uninstall and then reinstall Theano. Just to be extra 
 clear, here is my setup:

- Ubuntu 16.04
- Cuda 8.0, stored in `usr/local/cuda-8.0`
- Titan X GPU with Pascal

 cuDNN is here:

 $ ls /usr/local/cuda-8.0/lib64/cudnn.h 
 /usr/local/cuda-8.0/lib64/cudnn.h

 To verify that I can use my GPU I started this quick TensorFlow 
 computation:

 In [1]: import tensorflow as tf

 In [2]: tf.__version__
 Out[2]: '1.1.0'

 In [3]: tf.GPUOptions
 Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions

 In [4]: with tf.device('/gpu:0'):
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3
 ], name='a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2
 ], name='b')
...: c = tf.matmul(a,b)
...: 

 In [5]: with tf.Session() as sess:
...: print(sess.run(c))
...: 
 2017-06-16 16:10:54.402311: W tensorflow/core/platform/
 cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use 
 SSE4.1 instructions, but these are available on your machine and could 
 speed up CPU computations.
 2017-06-16 16:10:54.402328: W 
 tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library 
 wasn't compiled to use SSE4.2 instructions, but these are available 

Re: [theano-users] Re: Cannot do a simple theano install (Python 2.7, Ubuntu 16.04, Theano 0.9, Cuda 8.0, TitanX GPU) due to pygpu errors

[Pascal Lamblin]

Something weird seems to be happening:
- theano detects that cuDNN is available, in fact it called 
_dnn_check_compile() and _dnn_check_version() successfully
- however, calling _dnn_lib() failed, which means ctypes did not manage to 
find the cudnn library.

Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to 
LD_LIBRARY_PATH)?

On Monday, June 19, 2017 at 6:16:37 PM UTC-4, nouiz wrote:
>
> Your cudnn.h file should not be in the lib64 directory, but in an include 
> directory. Tensorflow does none standard stuff related to import and cause 
> problem in other setup, but it seem to tolerate your non standard setup. 
> Theano does the standard setup.
>
> You can use the Theano flag dnn.include_path and dnn.library_path to tell 
> Theano where your cudnn.h and cudnn.so* files are.
>
> I did not see your last error in full.
>
> Le ven. 16 juin 2017 19:35, Daniel Seita  a 
> écrit :
>
>> Ack, sorry, half of my post got deleted! Hopefully you can still see it 
>> (i can find it by looking at the original post but it's in a really ugly 
>> format, sorry).
>>
>>
>>
>> On Friday, June 16, 2017 at 4:33:20 PM UTC-7, Daniel Seita wrote:
>>
>>> I was running into some more difficulties, so I gave up on getting this 
>>> to work and tried to uninstall and then reinstall Theano. Just to be extra 
>>> clear, here is my setup:
>>>
>>>- Ubuntu 16.04
>>>- Cuda 8.0, stored in `usr/local/cuda-8.0`
>>>- Titan X GPU with Pascal
>>>
>>> cuDNN is here:
>>>
>>> $ ls /usr/local/cuda-8.0/lib64/cudnn.h 
>>> /usr/local/cuda-8.0/lib64/cudnn.h
>>>
>>> To verify that I can use my GPU I started this quick TensorFlow 
>>> computation:
>>>
>>> In [1]: import tensorflow as tf
>>>
>>> In [2]: tf.__version__
>>> Out[2]: '1.1.0'
>>>
>>> In [3]: tf.GPUOptions
>>> Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions
>>>
>>> In [4]: with tf.device('/gpu:0'):
>>>...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3
>>> ], name='a')
>>>...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2
>>> ], name='b')
>>>...: c = tf.matmul(a,b)
>>>...: 
>>>
>>> In [5]: with tf.Session() as sess:
>>>...: print(sess.run(c))
>>>...: 
>>> 2017-06-16 16:10:54.402311: W tensorflow/core/platform/cpu_feature_guard
>>> .cc:45] The TensorFlow library wasn't compiled to use SSE4.1 
>>> instructions, but these are available on your machine and could speed up 
>>> CPU computations.
>>> 2017-06-16 16:10:54.402328: W 
>>> tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library 
>>> wasn't compiled to use SSE4.2 instructions, but these are available on 
>>> your machine and could speed up CPU computations.
>>> 2017-06-16 16:10:54.402346: W tensorflow/core/platform/cpu_feature_guard
>>> .cc:45] The TensorFlow library wasn't compiled to use AVX instructions, 
>>> but these are available on your machine and could speed up CPU computations.
>>> 2017-06-16 16:10:54.402350: W 
>>> tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library 
>>> wasn't compiled to use AVX2 instructions, but these are available on 
>>> your machine and could speed up CPU computations.
>>> 2017-06-16 16:10:54.402356: W tensorflow/core/platform/cpu_feature_guard
>>> .cc:45] The TensorFlow library wasn't compiled to use FMA instructions, 
>>> but these are available on your machine and could speed up CPU computations.
>>> 2017-06-16 16:10:54.527167: I 
>>> tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA 
>>> node read from SysFS had negative value (-1), but there must be at least 
>>> one NUMA node, so returning NUMA node zero
>>> 2017-06-16 16:10:54.527553: I 
>>> tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with 
>>> properties: 
>>> name: TITAN X (Pascal)
>>> major: 6 minor: 1 memoryClockRate (GHz) 1.531
>>> pciBusID :01:00.0
>>> Total memory: 11.90GiB
>>> Free memory: 11.38GiB
>>> 2017-06-16 16:10:54.527565: I 
>>> tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0 
>>> 2017-06-16 16:10:54.527568: I 
>>> tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0:   Y 
>>> 2017-06-16 16:10:54.527590: I 
>>> tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow 
>>> device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id: 
>>> :01:00.0)
>>> [[ 22.  28.]
>>>  [ 49.  64.]]
>>>
>>>
>>> This looks like it indicates a successful GPU and/or cuDNN installation.
>>>
>>> Great, now let's install the *development version* of Theano. The 
>>> instructions I'm following step-by-step: 
>>> http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html
>>>
>>> The first step seems to be to install miniconda. I downloaded the bash 
>>> script for Python 2.7 and ran it:
>>>
>>> ~/Downloads$ bash Miniconda2-latest-Linux-x86_64.sh 
>>>
>>> Welcome to Miniconda2 4.3.21 (by Continuum Analytics, Inc.)
>>>
>>> In order to continue the installation process, please 

[theano-users] Re: Type mismatch TensorType(float64, row) TensorType(float64, matrix)

[Pascal Lamblin]

It's hard to know from that level, but somewhere there is a shared variable 
that is a row (1 x n matrix), and its update value is a full matrix (k x m).
This could be a parameter of the model, or a value updated by the optimizer 
(something like a velocity).
It is possible that the update value is actually always 1 x n, but Theano 
could not infer that (in which case, this is an issue in Theano and we 
should fix it), in that case you can use tensor.addbroadcast() on the 
update expression (it will fail if k != 1 at runtime).
It is also possible that k != 1, in which case it is a problem in the model 
itself.

On Friday, June 23, 2017 at 3:43:39 AM UTC-4, Vimal K wrote:
>
>
>
> i am trying to run this program  https://github.com/npow/ubottu for 
> making a model for Ubuntu Dialog Corpus
>
> It uses Theano and Lasagne as dependencies.
>
> when i install Theano and Lasagne from pip, and do a simple 
>
> import lasagne
>
>
> i get the error
>
> from theano.tensor.signal import downsampleImportError: cannot import name 
> 'downsample'
>
>
> then found a workaround here, 
> https://stackoverflow.com/questions/42998355/lasagne-vs-theano-possible-version-mismatch-windows
> suggestion was to install Theano and Lasagne from their github master 
> branches respectively
>
> pip install --upgrade https://github.com/Theano/Theano/archive/master.zip
> pip install --upgrade https://github.com/Lasagne/Lasagne/archive/master.zip
>
>
> This removed the above erorr, but i am getting the following error.
>
>
> TypeError: ('An update must have the same type as the original shared 
> variable (shared_var=hid_init, shared_var.type=TensorType(float64, row), 
> update_val=Elemwise{sub,no_inplace}.0, update_val.type=TensorType(float64, 
> matrix)).', 'If the difference is related to the broadcast pattern, you 
> can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function 
> to remove broadcastable dimensions.')
>
>
>
> it seems that there is a type mismatch somewhere. a variable of row type 
> is updated with a value of matrix type. Full traceback as follows :
>
> Traceback (most recent call last):
>   File "main.py", line 669, in 
> main()
>   File "main.py", line 661, in main
> model = Model(**args.__dict__)
>   File "main.py", line 375, in __init__
> self.update_params()
>   File "main.py", line 408, in update_params
> self.train_model = theano.function([], self.cost, updates=updates, 
> givens=givens, on_unused_input='warn')
>   File 
> "/home/vimal/anaconda2/lib/python2.7/site-packages/theano/compile/function.py"
> , line 326, in function
> output_keys=output_keys)
>   File 
> "/home/vimal/anaconda2/lib/python2.7/site-packages/theano/compile/pfunc.py"
> , line 449, in pfunc
> no_default_updates=no_default_updates)
>   File 
> "/home/vimal/anaconda2/lib/python2.7/site-packages/theano/compile/pfunc.py"
> , line 208, in rebuild_collect_shared
> raise TypeError(err_msg, err_sug)
> TypeError: ('An update must have the same type as the original shared 
> variable (shared_var=cell_init, shared_var.type=TensorType(float64, row), 
> update_val=Elemwise{sub,no_inplace}.0, update_val.type=TensorType(float64, 
> matrix)).', 'If the difference is related to the broadcast pattern, you 
> can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function 
> to remove broadcastable dimensions.')
>
>
>
>
>
>
> the versions are
> >>> import theano as th
> >>> print th.__version__
> 0.10.0dev1.dev-RELEASE
>
>
> >>> import lasagne as la
> >>> print la.__version__
> 0.2.dev1
>
>
>
>
>
>
> Can you help me to resolve this error? 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: How can I calculate the size of output of convolutional operation in theano?

[Pascal Lamblin]

On Monday, June 26, 2017 at 3:36:21 PM UTC-4, Sunjeet Jena wrote:
>
> Is there any way I can calculate the size of the output after the 
> Convolution Operation?
>

You can check theano.tensor.nnet.abstract_conv.get_conv_output_shape() 

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] And operator doesn't work with theano logical operators

[Pascal Lamblin]

On Thursday, June 29, 2017 at 10:17:26 AM UTC-4, Sym wrote:
>
> I am using theano 0.9.0 and this function does not exist.. I can't find it 
> in the documentation either !
>

That would be theano.tensor.and_ with a trailing underscore.
 

> And in the theano docs it says that :
> __{abs,neg,lt,le,gt,ge,invert,and,or,add,sub,mul,div,truediv,floordiv}__
> Those elemwise operation are supported via Python syntax.
>

This is true, but the Python syntax that will call `__and__` is actually `a 
&& b`, not `a and b`.

So you can use `a && b`, `and_(a, b)`, or `a * b` if you prefer.
 

>
> Le mercredi 28 juin 2017 21:15:37 UTC-4, nouiz a écrit :
>>
>> Don't use the Python "and" operation. Use theano.tensor.and(a,b) instead. 
>> I think it will fix your problem.
>>
>> Le mer. 28 juin 2017 10:26, Sym  a écrit :
>>
>>>
>>> I want to build a piecewise function with theano, for instance a 
>>> function that is nonzero only in the interval [2,3].
>>>
>>> Here is the minimal code reproducing the error : 
>>>
>>>
>>> import theano
>>> import theano.tensor as T
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>>
>>> r = T.scalar()
>>> gate = T.switch( T.ge(r,2.) and T.le(r,3.) , 1., 0.)
>>> f = theano.function([r],gate)
>>> x = np.arange(0.,4.,0.05,dtype='float32')
>>> y = [f(i) for i in x]
>>> plt.plot(x,y)
>>>
>>>  
>>>
>>> The result is the following : https://i.stack.imgur.com/XMQme.png 
>>>
>>> Which is clearly not correct : only one condition is satisfied here.
>>>
>>>
>>> If I replace T.switch by theano.ifelse.ifelse the result is the same...
>>>
>>> Is it a known bug, or am I missing something here?
>>>
>>>
>>> Thanks a lot !
>>>
>>> -- 
>>>
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "theano-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to theano-users...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Gradient Problem (always 0)

[Pascal Lamblin]

I'm assuming m and n were defined as T.vector(), and that the last line of 
the "def rnn(...)" functions is actually "return x_t, r_t", is that correct?

Do you have a non-zero gradient for Wrec?
Can you monitor the value of theano.grad(cost, Wrec).sum() is?
Normally, the sum of the gradient wrt Wrec should be equal to the gradient 
wrt dot(u, v). So if the gradient wrt Wrec is not zero everywhere, but its 
sum is zero, then it would explain that result.
If we manually backprop, then we can see that the gradient of the cost wrt 
u is equivalent to grad(cost, Wrec).sum() * v (and the gradient wrt v 
should be equivalent to grad(cost, Wrec).sum() * u). Can you monitor those 
values?


On Wednesday, June 28, 2017 at 7:12:06 PM UTC-4, Mohamed Akrout wrote:
>
> Hi all,
>
> I am running a neuroscience with an recurrent neural network model with 
> Theano:
>
>
>
> def rnn(u_t, x_tm1, r_tm1, Wrec):
>  x_t = ( (1 - alpha)*x_tm1 + alpha*(T.dot(r_tm1, Wrec ) + brec + 
> u_t[:,Nin:]) )
>  r_t = f_hidden(x_t)
>
>
> then I define the scan function to iterate at each time step iteration
>
> [x, r], _ = theano.scan(fn=rnn,
> outputs_info=[x0_, f_hidden(x0_)],
> sequences=u,
> non_sequences=[Wrec])
>
> Wrec and brec are learnt by stochastic gradient descent: g = T.grad(cost , 
> [Wrec, brec])
>
> where cost is the cost function: T.sum(f_loss(z, target[:,:,:Nout])) with 
> z = f_output(T.dot(r, Wout_.T) + bout )
>
> Until now, everything works good.
>
>
>
> Now I want to add two new vectors, let's call them u and v so that the 
> initial rnn function becomes:
>
>
> def rnn(u_t, x_tm1, r_tm1, Wrec, *u, v*):
>  x_t = ( (1 - alpha)*x_tm1 + alpha*(T.dot(r_tm1, Wrec + *T.dot(u, 
> v)* ) + brec + u_t[:,Nin:]) )
>  r_t = f_hidden(x_t)
>
> [x, r], _ = theano.scan(fn=rnn,
> outputs_info=[x0_, f_hidden(x0_)],
> sequences=u,
> non_sequences=[Wrec,* m, n*])
>
> m and n are the variables corresponding to u and v in the main function.
>
> and suddenly, the gradient T.grad(cost, m) and T.grad(cost, n) are zeros
>
> I am blocked since 2 weeks now on this problem. I verified that the values 
> are not integer by using dtype=theano.config.floatX every where in the 
> definition of the variables.
>
> As you can see the link between the cost and m (or n) is: the cost 
> function depends on  z, and z depends on r and r is one of the outputs of 
> the rnn function that uses m and n in the equation.
>
> Do you have any ideas why this does not work ?
>
> Any idea is welcome. I hope I can unblock this problem soon.
> Thank you!
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] theano0.9 and cuda-8

[Frédéric Bastien]

You should not mix cuda version...

Do you still use the old gpu back-end (device=gpu*) or the new back-end
(device=cuda*)?

Fred

On Fri, Jun 30, 2017 at 9:57 AM  wrote:

> I trying to understand some unexplained behavior of my code.
> To be sure that the problem is with my code and not with software
> incompatibility I would like to sure  about the correctness of my setup
> I have:
> theano version 0.9
>
> CUDA_ROOT =/usr/local/cuda-7.5
> LD_PATH=/usr/local/cuda-8/lib64:...
> PATH=/usr/local/cuda-8/bin: ...
>
> Essentially I am using some parts of cuda-8 and some of cuda-7.5.
>
> With CUDA_ROOT =/usr/local/cuda-8, I cannot compile the theano functions.
>
> Thanks
>
>
>
>
> ***
>
> This e-mail and any of its attachments may contain Interactions
> Corporation proprietary information, which is privileged, confidential, or
> subject to copyright belonging to the Interactions Corporation. This e-mail
> is intended solely for the use of the individual or entity to which it is
> addressed. If you are not the intended recipient of this e-mail, you are
> hereby notified that any dissemination, distribution, copying, or action
> taken in relation to the contents of and attachments to this e-mail is
> strictly prohibited and may be unlawful. If you have received this e-mail
> in error, please notify the sender immediately and permanently delete the
> original and any copy of this e-mail and any printout. Thank You.
>
>
> ***
>
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Deleting .theano each time I change server/machine/node

[Frédéric Bastien]

You can use this Theano flags to append to the default compiledir the
hostname. This way you will have one for each computer, but it still will
be in your home:

compiledir_formet="compiledir_%(short_platform)s-%(processor)s-%(python_version)s-%(python_bitwidth)s-%(hostname)s

But you can go simplet, change the base compiledir to be local to each
computer:

base_compiledir=/tmp/%(user)s/theano_base_compiledir

Fred

On Fri, Jun 30, 2017 at 8:54 AM André L  wrote:

> *Context of the issue *
>
> I work with Theano in my university server via SSH and with virtualenv.
> I dont have admin privileges.
> I can access several servers.
>
> Normally i work in server "D". But when i tried running the same
> experiment on server "R", it failed with illegal instruction. So i deleted
> the .theano folder and it worked.
> However, when if i want to run again on server "D", i must delete .theano
> folder again.
>
>
> *Problem:*
> How can i have theano working on two different machines without deleting
> .theano each time i change server? Also, what if i want to run a theano
> code in server "D" and also in server "R" at the same time?
>
>
> *According to Daniel Renshaw:*
>
>
> *Now I look at the documentation I see that the full directory determined
> by the compiledir_format flag includes details that should, I would have
> thought, ensured that your compiled bits were kept separate on the two
> different architectures. Not idea why it isn't working. However, if you
> set base_compiledir differently on each machine you'll ensure each gets its
> own compilation cache.*
>
>
> However, i cant have two .locals since im no admin and im always logging
> via SSH with the same user.
>
> What should I do ?
>
>
>
>
> References :
>
> 
>
> https://stackoverflow.com/questions/29338016/import-theano-gets-illegal-instruction
>
>
> https://groups.google.com/forum/#!searchin/theano-users/.local$20different$20computer|sort:relevance/theano-users/MxPMjrt-ZB8/930HYOa6BAAJ
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] theano0.9 and cuda-8

[ngupta]

I trying to understand some unexplained behavior of my code.
To be sure that the problem is with my code and not with software 
incompatibility I would like to sure  about the correctness of my setup
I have:
theano version 0.9

CUDA_ROOT =/usr/local/cuda-7.5
LD_PATH=/usr/local/cuda-8/lib64:...
PATH=/usr/local/cuda-8/bin: ...

Essentially I am using some parts of cuda-8 and some of cuda-7.5.

With CUDA_ROOT =/usr/local/cuda-8, I cannot compile the theano functions.

Thanks



-- 
 

***

This e-mail and any of its attachments may contain Interactions Corporation 
proprietary information, which is privileged, confidential, or subject to 
copyright belonging to the Interactions Corporation. This e-mail is 
intended solely for the use of the individual or entity to which it is 
addressed. If you are not the intended recipient of this e-mail, you are 
hereby notified that any dissemination, distribution, copying, or action 
taken in relation to the contents of and attachments to this e-mail is 
strictly prohibited and may be unlawful. If you have received this e-mail 
in error, please notify the sender immediately and permanently delete the 
original and any copy of this e-mail and any printout. Thank You.  

*** 
 

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Always Segmentation fault(core dumped) when use theano(NOT when import)

[Frédéric Bastien]

Install the dev version of Theano. It contains segmentation fault fixes.

If that don't work, tell us, but I think it should work.

Le ven. 30 juin 2017 06:00, noodles  a écrit :

> Hello,
>
> I encounter a strange problem when using theano. These days I bought
> a new computer and install theano on it, and I can even import it in
> python with no error, but everytime I create a function, it corrupted with 
> "Segmentation
> fault(core dumped)". Below is the detail:
> I have installed theano on another two old machine, and they works 
> well.
> This new machine is : CPU: intel 7700; GPU  2xGTX1080Ti, OS: ubuntu16.04.
> CUDA 8.0, cudnn 5.1 .I use miniconda2 to install theano( conda install
> theano), python 2.7, theano 0.9.0
>
>   when I import theano in python, the output is:
>
>> *nice@fat01:~$ python*
>> *Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016,
>> 23:09:15) *
>> *[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2*
>> *Type "help", "copyright", "credits" or "license" for more information.*
>> *Anaconda is brought to you by Continuum Analytics.*
>> *Please check out: http://continuum.io/thanks > >
>> and https://anaconda.org *
>> *>>> import theano*
>> *Using cuDNN version 5110 on context None*
>> *Mapped name None to device cuda1: GeForce GTX 1080 Ti (:02:00.0)*
>> *>>> *
>
>
> then I input the code from the exercise of
> http://deeplearning.net/software/theano/tutorial/using_gpu.html#gpuarray
>
> 
>
> *import numpy*
> *import theano*
> *import theano.tensor as T*
> *rng = numpy.random*
> *N = 400*
> *feats = 784*
> *D = (rng.randn(N, feats).astype(theano.config.floatX),*
> *rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))*
> *training_steps = 1*
> *# Declare Theano symbolic variables*
> *x = T.matrix("x")*
> *y = T.vector("y")*
> *w = theano.shared(rng.randn(feats).astype(theano.config.floatX),
> name="w")*
> *b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX),
> name="b")*
> *x.tag.test_value = D[0]*
> *y.tag.test_value = D[1]*
> *# Construct Theano expression graph*
> *p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one*
> *prediction = p_1 > 0.5 # The prediction that is done: 0 or 1*
> *xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy*
> *cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize*
> *gw,gb = T.grad(cost, [w,b])*
> *# Compile expressions to functions*
> *train = theano.function(*
> *inputs=[x,y],*
> *outputs=[prediction, xent],*
> *updates=[(w, w-0.01*gw), (b, b-0.01*gb)],*
> *name = "train")*
>
>
> ==
> It corrupted at this line.
> I have run numpy.test() and scipy.test() and they work well, but when I
> run theano.test(), it corrupted too. The full log is too long, so I just
> post
> the end of it:
>
> */home/nice/miniconda2/lib/python2.7/site-packages/
>> theano/compile/nanguardmode.py:168:
>> RuntimeWarning: All-NaN axis encountered*
>> *  return np.isinf(np.nanmax(arr)) or np.isinf(np.nanmin(arr))*
>> *.E/home/nice/
>> miniconda2/lib/python2.7/site-packages/theano/gof/vm.py:851:
>> UserWarning: CVM does not support memory profile, using Stack VM.*
>> *  'CVM does not support memory profile, using Stack VM.')*
>> *...SS.0.930614401665*
>> *0.930614401665*
>> *0.930614401665*
>> *0.930614401665*
>> *...
>> 
>> ...E/home/nice/miniconda2/
>> lib/python2.7/site-packages/theano/gof/vm.py:854:
>> UserWarning: LoopGC does not support partial evaluation, using Stack VM.*
>> *  'LoopGC does not support partial evaluation, '*
>> *.Segmentation fault (core dumped)*
>
>
>
> I hope someone can help me.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: Theano sort - new GPU implementation

[Frédéric Bastien]

There is a pr for topk, but it is not sorted. You can use it and keep the
sort on CPU for now. It will be faster like that I think.

Le jeu. 29 juin 2017 21:11, Victor Campmany  a écrit :

> We are working on both sorts for 1D arrays and sort of an axis of an
> nd-array. We are trying to release it as soon as possible.
>
> El jueves, 29 de junio de 2017, 21:02:08 (UTC-4), Adam Becker escribió:
>>
>> Has there been any progress? I'm in need of sorted TopK on GPU. I can go
>> with CPU sort but seems a bit slow.
>>
>> On Thursday, June 15, 2017 at 4:01:00 AM UTC+8, Victor Campmany wrote:
>>>
>>> Hi,
>>>
>>> We are planning to implement a new GPU accelerated sorting algorithm.
>>> We'd like to know which are the most frequent sorting cases that you guys
>>> use and the data sizes you are dealing with. For example, sorting a large
>>> 1d array, sorting a given axis of a tensor or minibatch, or any other type
>>> of sorting you come up with.
>>>
>> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Always Segmentation fault(core dumped) when use theano(NOT when import)

[noodles]

Hello,

I encounter a strange problem when using theano. These days I bought 
a new computer and install theano on it, and I can even import it in
python with no error, but everytime I create a function, it corrupted with 
"Segmentation 
fault(core dumped)". Below is the detail:
I have installed theano on another two old machine, and they works 
well. 
This new machine is : CPU: intel 7700; GPU  2xGTX1080Ti, OS: ubuntu16.04.  
CUDA 8.0, cudnn 5.1 .I use miniconda2 to install theano( conda install 
theano), python 2.7, theano 0.9.0

  when I import theano in python, the output is:

> *nice@fat01:~$ python*
> *Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016,
> 23:09:15) *
> *[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2*
> *Type "help", "copyright", "credits" or "license" for more information.*
> *Anaconda is brought to you by Continuum Analytics.*
> *Please check out: http://continuum.io/thanks 
> and https://anaconda.org *
> *>>> import theano*
> *Using cuDNN version 5110 on context None*
> *Mapped name None to device cuda1: GeForce GTX 1080 Ti (:02:00.0)*
> *>>> *

 
then I input the code from the exercise of 
http://deeplearning.net/software/theano/tutorial/using_gpu.html#gpuarray 


*import numpy*
*import theano*
*import theano.tensor as T*
*rng = numpy.random*
*N = 400*
*feats = 784*
*D = (rng.randn(N, feats).astype(theano.config.floatX),*
*rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))*
*training_steps = 1*
*# Declare Theano symbolic variables*
*x = T.matrix("x")*
*y = T.vector("y")*
*w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name="w")*
*b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name="b")*
*x.tag.test_value = D[0]*
*y.tag.test_value = D[1]*
*# Construct Theano expression graph*
*p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one*
*prediction = p_1 > 0.5 # The prediction that is done: 0 or 1*
*xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy*
*cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize*
*gw,gb = T.grad(cost, [w,b])*
*# Compile expressions to functions*
*train = theano.function(*
*inputs=[x,y],*
*outputs=[prediction, xent],*
*updates=[(w, w-0.01*gw), (b, b-0.01*gb)],*
*name = "train")*

== 
It corrupted at this line.
I have run numpy.test() and scipy.test() and they work well, but when I run 
theano.test(), 
it corrupted too. The full log is too long, so I just post
the end of it:

*/home/nice/miniconda2/lib/python2.7/site-packages/
> theano/compile/nanguardmode.py:168:
> RuntimeWarning: All-NaN axis encountered*
> *  return np.isinf(np.nanmax(arr)) or np.isinf(np.nanmin(arr))*
> *.E/home/nice/
> miniconda2/lib/python2.7/site-packages/theano/gof/vm.py:851:
> UserWarning: CVM does not support memory profile, using Stack VM.*
> *  'CVM does not support memory profile, using Stack VM.')*
> *...SS.0.930614401665*
> *0.930614401665*
> *0.930614401665*
> *0.930614401665*
> *...
> 
> ...E/home/nice/miniconda2/
> lib/python2.7/site-packages/theano/gof/vm.py:854:
> UserWarning: LoopGC does not support partial evaluation, using Stack VM.*
> *  'LoopGC does not support partial evaluation, '*
> *.Segmentation fault (core dumped)*



I hope someone can help me.  

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.