Re: [theano-users] Could not initialize pygpu error with CUDA 9.0 and latest Theano

[Frédéric Bastien]

creating 2 env seem the right thing to do. I do not think dnn.libary_path
override LD_LIBRARY_PATH.

On Thu, May 10, 2018 at 7:06 PM Michael Klachko 
wrote:

> What is the best way to do that? Should I use separate conda environments
> for theano and tensorflow, and create LD_LIBRARY_PATH in each? Does
> dnn.libary_path in theanorc override LD_LIBRARY_PATH?
>
>
>
> On Thursday, May 10, 2018 at 3:09:29 PM UTC-7, nouiz wrote:
>
>> You could have multiple cuda version installed to have TF working.
>>
>> Le jeu. 10 mai 2018 16:28, Michael Klachko  a
>> écrit :
>>
>>> After struggling with this error for a day, I decided to upgrade CUDA to
>>> 9.1 and CuDNN to 7.1. After that I got "your driver might be too old"
>>> error, which was resolved by updating the driver to 396.24. Also, in the
>>> process I found out I had older CuDNN files in /usr/lib/x86_64-linux-gnu/
>>> directory. Not sure how they got there, perhaps because sometimes I
>>> installed CuDNN using .deb package, and sometimes by manually copying the
>>> files. So it's probably not a good idea to mix .deb and .run cuda
>>> installation methods.
>>>
>>> Anyway, now theano works fine now, but unfortunately my Tensorflow is
>>> broken because it does not support cuda 9.1 yet... Will probably have to
>>> compile it from source.
>>>
>>>
>>>
>>> On Thursday, May 10, 2018 at 11:30:38 AM UTC-7, Arnaud Bergeron wrote:
>>>
 This is a new one.  It is also very weird since gemm doesn't involve
 cuLinkAddData.  This may be an error message from something else.

 First things first, since you are on cuda 9.0, I would recommend that
 you update your driver to 384.111 or 390.*.  If that doesn't help, then
 I'll need some help reproducing the problem since I don't get that in any
 of my environments.

>>> Le 8 mai 2018 à 18:15, Michael Klachko  a écrit :

 I have CUDA 9.0 and CuDNN 7.0.5 on my Ubuntu 16.04, and Tensorflow
 works fine. In order to install theano, I first installed miniconda, then
 ran "conda install theano pygpu" and it seemed to have installed fine.



 However, here's what I get:


 $ python
 Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
 [GCC 7.2.0] on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import theano
 Using cuDNN version 7005 on context None
 ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
 Traceback (most recent call last):
   File 
 "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
  line 227, in 
 use(config.device)
   File 
 "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
  line 214, in use
 init_dev(device, preallocate=preallocate)
   File 
 "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
  line 159, in init_dev
 pygpu.blas.gemm(0, tmp, tmp, 0, tmp, overwrite_c=True)
   File "pygpu/blas.pyx", line 149, in pygpu.blas.gemm
   File "pygpu/blas.pyx", line 47, in pygpu.blas.pygpu_blas_rgemm
 pygpu.gpuarray.GpuArrayException: (b'cuLinkAddData: CUDA_ERROR_UNKNOWN: 
 unknown error', 3)



 Here's the packages I have installed in this environment:


 $ conda list
 # packages in environment at /home/michael/miniconda2/envs/las:
 #
 # NameVersion   Build  Channel
 binutils_impl_linux-642.28.1   had2808c_3
 binutils_linux-64 7.2.026
 ca-certificates   2018.03.070
 certifi   2018.4.16py36_0
 gcc_impl_linux-64 7.2.0habb00fd_3
 gcc_linux-64  7.2.026
 gxx_impl_linux-64 7.2.0hdf63c60_3
 gxx_linux-64  7.2.026
 intel-openmp  2018.0.0  8
 libedit   3.1  heed3624_0
 libffi3.2.1hd88cf55_4
 libgcc-ng 7.2.0hdf63c60_3
 libgfortran-ng7.2.0hdf63c60_3
 libgpuarray   0.7.5h14c3975_0
 libstdcxx-ng  7.2.0hdf63c60_3
 mako  1.0.7py36h0727276_0
 markupsafe1.0  py36hd9260cd_1
 mkl   2018.0.2  1
 mkl-service   1.1.2py36h17a0993_4
 mkl_fft   1.0.1py36h3010b51_0
 mkl_random1.0.1py36h629b387_0
 ncurses   6.0  

Re: [theano-users] Could not initialize pygpu error with CUDA 9.0 and latest Theano

[Michael Klachko]

What is the best way to do that? Should I use separate conda environments 
for theano and tensorflow, and create LD_LIBRARY_PATH in each? Does 
dnn.libary_path in theanorc override LD_LIBRARY_PATH? 



On Thursday, May 10, 2018 at 3:09:29 PM UTC-7, nouiz wrote:
>
> You could have multiple cuda version installed to have TF working.
>
> Le jeu. 10 mai 2018 16:28, Michael Klachko  > a écrit :
>
>> After struggling with this error for a day, I decided to upgrade CUDA to 
>> 9.1 and CuDNN to 7.1. After that I got "your driver might be too old" 
>> error, which was resolved by updating the driver to 396.24. Also, in the 
>> process I found out I had older CuDNN files in /usr/lib/x86_64-linux-gnu/ 
>> directory. Not sure how they got there, perhaps because sometimes I 
>> installed CuDNN using .deb package, and sometimes by manually copying the 
>> files. So it's probably not a good idea to mix .deb and .run cuda 
>> installation methods. 
>>
>> Anyway, now theano works fine now, but unfortunately my Tensorflow is 
>> broken because it does not support cuda 9.1 yet... Will probably have to 
>> compile it from source.
>>
>>
>>
>> On Thursday, May 10, 2018 at 11:30:38 AM UTC-7, Arnaud Bergeron wrote:
>>
>>> This is a new one.  It is also very weird since gemm doesn't involve 
>>> cuLinkAddData.  This may be an error message from something else.
>>>
>>> First things first, since you are on cuda 9.0, I would recommend that 
>>> you update your driver to 384.111 or 390.*.  If that doesn't help, then 
>>> I'll need some help reproducing the problem since I don't get that in any 
>>> of my environments.
>>>
>> Le 8 mai 2018 à 18:15, Michael Klachko  a écrit :
>>>
>>> I have CUDA 9.0 and CuDNN 7.0.5 on my Ubuntu 16.04, and Tensorflow works 
>>> fine. In order to install theano, I first installed miniconda, then ran 
>>> "conda 
>>> install theano pygpu" and it seemed to have installed fine.
>>>
>>>
>>>
>>> However, here's what I get:
>>>
>>>
>>> $ python
>>> Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
>>> [GCC 7.2.0] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> >>> import theano
>>> Using cuDNN version 7005 on context None
>>> ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
>>> Traceback (most recent call last):
>>>   File 
>>> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>>>  line 227, in 
>>> use(config.device)
>>>   File 
>>> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>>>  line 214, in use
>>> init_dev(device, preallocate=preallocate)
>>>   File 
>>> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>>>  line 159, in init_dev
>>> pygpu.blas.gemm(0, tmp, tmp, 0, tmp, overwrite_c=True)
>>>   File "pygpu/blas.pyx", line 149, in pygpu.blas.gemm
>>>   File "pygpu/blas.pyx", line 47, in pygpu.blas.pygpu_blas_rgemm
>>> pygpu.gpuarray.GpuArrayException: (b'cuLinkAddData: CUDA_ERROR_UNKNOWN: 
>>> unknown error', 3)
>>>
>>>  
>>>
>>> Here's the packages I have installed in this environment:
>>>
>>>
>>> $ conda list
>>> # packages in environment at /home/michael/miniconda2/envs/las:
>>> #
>>> # NameVersion   Build  Channel
>>> binutils_impl_linux-642.28.1   had2808c_3
>>> binutils_linux-64 7.2.026
>>> ca-certificates   2018.03.070
>>> certifi   2018.4.16py36_0
>>> gcc_impl_linux-64 7.2.0habb00fd_3
>>> gcc_linux-64  7.2.026
>>> gxx_impl_linux-64 7.2.0hdf63c60_3
>>> gxx_linux-64  7.2.026
>>> intel-openmp  2018.0.0  8
>>> libedit   3.1  heed3624_0
>>> libffi3.2.1hd88cf55_4
>>> libgcc-ng 7.2.0hdf63c60_3
>>> libgfortran-ng7.2.0hdf63c60_3
>>> libgpuarray   0.7.5h14c3975_0
>>> libstdcxx-ng  7.2.0hdf63c60_3
>>> mako  1.0.7py36h0727276_0
>>> markupsafe1.0  py36hd9260cd_1
>>> mkl   2018.0.2  1
>>> mkl-service   1.1.2py36h17a0993_4
>>> mkl_fft   1.0.1py36h3010b51_0
>>> mkl_random1.0.1py36h629b387_0
>>> ncurses   6.0  h9df7e31_2
>>> nose  1.3.7py36hcdf7029_2
>>> numpy 1.14.2   py36hdbf6ddf_1
>>> openssl   1.0.2o   h20670df_0
>>> pip   10.0.1   py36_0
>>> pygpu   

Re: [theano-users] Could not initialize pygpu error with CUDA 9.0 and latest Theano

[Frédéric Bastien]

You could have multiple cuda version installed to have TF working.

Le jeu. 10 mai 2018 16:28, Michael Klachko  a
écrit :

> After struggling with this error for a day, I decided to upgrade CUDA to
> 9.1 and CuDNN to 7.1. After that I got "your driver might be too old"
> error, which was resolved by updating the driver to 396.24. Also, in the
> process I found out I had older CuDNN files in /usr/lib/x86_64-linux-gnu/
> directory. Not sure how they got there, perhaps because sometimes I
> installed CuDNN using .deb package, and sometimes by manually copying the
> files. So it's probably not a good idea to mix .deb and .run cuda
> installation methods.
>
> Anyway, now theano works fine now, but unfortunately my Tensorflow is
> broken because it does not support cuda 9.1 yet... Will probably have to
> compile it from source.
>
>
>
> On Thursday, May 10, 2018 at 11:30:38 AM UTC-7, Arnaud Bergeron wrote:
>
>> This is a new one.  It is also very weird since gemm doesn't involve
>> cuLinkAddData.  This may be an error message from something else.
>>
>> First things first, since you are on cuda 9.0, I would recommend that you
>> update your driver to 384.111 or 390.*.  If that doesn't help, then I'll
>> need some help reproducing the problem since I don't get that in any of my
>> environments.
>>
> Le 8 mai 2018 à 18:15, Michael Klachko  a écrit :
>>
>> I have CUDA 9.0 and CuDNN 7.0.5 on my Ubuntu 16.04, and Tensorflow works
>> fine. In order to install theano, I first installed miniconda, then ran 
>> "conda
>> install theano pygpu" and it seemed to have installed fine.
>>
>>
>>
>> However, here's what I get:
>>
>>
>> $ python
>> Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
>> [GCC 7.2.0] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import theano
>> Using cuDNN version 7005 on context None
>> ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
>> Traceback (most recent call last):
>>   File 
>> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>>  line 227, in 
>> use(config.device)
>>   File 
>> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>>  line 214, in use
>> init_dev(device, preallocate=preallocate)
>>   File 
>> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>>  line 159, in init_dev
>> pygpu.blas.gemm(0, tmp, tmp, 0, tmp, overwrite_c=True)
>>   File "pygpu/blas.pyx", line 149, in pygpu.blas.gemm
>>   File "pygpu/blas.pyx", line 47, in pygpu.blas.pygpu_blas_rgemm
>> pygpu.gpuarray.GpuArrayException: (b'cuLinkAddData: CUDA_ERROR_UNKNOWN: 
>> unknown error', 3)
>>
>>
>>
>> Here's the packages I have installed in this environment:
>>
>>
>> $ conda list
>> # packages in environment at /home/michael/miniconda2/envs/las:
>> #
>> # NameVersion   Build  Channel
>> binutils_impl_linux-642.28.1   had2808c_3
>> binutils_linux-64 7.2.026
>> ca-certificates   2018.03.070
>> certifi   2018.4.16py36_0
>> gcc_impl_linux-64 7.2.0habb00fd_3
>> gcc_linux-64  7.2.026
>> gxx_impl_linux-64 7.2.0hdf63c60_3
>> gxx_linux-64  7.2.026
>> intel-openmp  2018.0.0  8
>> libedit   3.1  heed3624_0
>> libffi3.2.1hd88cf55_4
>> libgcc-ng 7.2.0hdf63c60_3
>> libgfortran-ng7.2.0hdf63c60_3
>> libgpuarray   0.7.5h14c3975_0
>> libstdcxx-ng  7.2.0hdf63c60_3
>> mako  1.0.7py36h0727276_0
>> markupsafe1.0  py36hd9260cd_1
>> mkl   2018.0.2  1
>> mkl-service   1.1.2py36h17a0993_4
>> mkl_fft   1.0.1py36h3010b51_0
>> mkl_random1.0.1py36h629b387_0
>> ncurses   6.0  h9df7e31_2
>> nose  1.3.7py36hcdf7029_2
>> numpy 1.14.2   py36hdbf6ddf_1
>> openssl   1.0.2o   h20670df_0
>> pip   10.0.1   py36_0
>> pygpu 0.7.5py36h14c3975_0
>> python3.6.5hc3d631a_2
>> readline  7.0  ha6073c6_4
>> scipy 1.0.1py36hfc37229_0
>> setuptools39.1.0   py36_0
>> six   1.11.0   py36h372c433_1
>> sqlite

Re: [theano-users] Could not initialize pygpu error with CUDA 9.0 and latest Theano

[Arnaud Bergeron]

This is a new one.  It is also very weird since gemm doesn't involve 
cuLinkAddData.  This may be an error message from something else.

First things first, since you are on cuda 9.0, I would recommend that you 
update your driver to 384.111 or 390.*.  If that doesn't help, then I'll need 
some help reproducing the problem since I don't get that in any of my 
environments.

> Le 8 mai 2018 à 18:15, Michael Klachko  a écrit :
> 
> I have CUDA 9.0 and CuDNN 7.0.5 on my Ubuntu 16.04, and Tensorflow works 
> fine. In order to install theano, I first installed miniconda, then ran 
> "conda install theano pygpu" and it seemed to have installed fine.
> 
> 
> 
> 
> 
> However, here's what I get:
> 
> 
> $ python
> Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
> [GCC 7.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import theano
> Using cuDNN version 7005 on context None
> ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
> Traceback (most recent call last):
>   File 
> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>  line 227, in 
> use(config.device)
>   File 
> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>  line 214, in use
> init_dev(device, preallocate=preallocate)
>   File 
> "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py",
>  line 159, in init_dev
> pygpu.blas.gemm(0, tmp, tmp, 0, tmp, overwrite_c=True)
>   File "pygpu/blas.pyx", line 149, in pygpu.blas.gemm
>   File "pygpu/blas.pyx", line 47, in pygpu.blas.pygpu_blas_rgemm
> pygpu.gpuarray.GpuArrayException: (b'cuLinkAddData: CUDA_ERROR_UNKNOWN: 
> unknown error', 3)
> 
>  
> Here's the packages I have installed in this environment:
> 
> 
> 
> $ conda list
> # packages in environment at /home/michael/miniconda2/envs/las:
> #
> # NameVersion   Build  Channel
> binutils_impl_linux-642.28.1   had2808c_3
> binutils_linux-64 7.2.026
> ca-certificates   2018.03.070
> certifi   2018.4.16py36_0
> gcc_impl_linux-64 7.2.0habb00fd_3
> gcc_linux-64  7.2.026
> gxx_impl_linux-64 7.2.0hdf63c60_3
> gxx_linux-64  7.2.026
> intel-openmp  2018.0.0  8
> libedit   3.1  heed3624_0
> libffi3.2.1hd88cf55_4
> libgcc-ng 7.2.0hdf63c60_3
> libgfortran-ng7.2.0hdf63c60_3
> libgpuarray   0.7.5h14c3975_0
> libstdcxx-ng  7.2.0hdf63c60_3
> mako  1.0.7py36h0727276_0
> markupsafe1.0  py36hd9260cd_1
> mkl   2018.0.2  1
> mkl-service   1.1.2py36h17a0993_4
> mkl_fft   1.0.1py36h3010b51_0
> mkl_random1.0.1py36h629b387_0
> ncurses   6.0  h9df7e31_2
> nose  1.3.7py36hcdf7029_2
> numpy 1.14.2   py36hdbf6ddf_1
> openssl   1.0.2o   h20670df_0
> pip   10.0.1   py36_0
> pygpu 0.7.5py36h14c3975_0
> python3.6.5hc3d631a_2
> readline  7.0  ha6073c6_4
> scipy 1.0.1py36hfc37229_0
> setuptools39.1.0   py36_0
> six   1.11.0   py36h372c433_1
> sqlite3.23.1   he433501_0
> theano1.0.1py36h6bb024c_0
> tk8.6.7hc745277_3
> wheel 0.31.0   py36_0
> xz5.2.3h5e939de_4
> zlib  1.2.11   ha838bed_2
> 
>  
> Here's my .theanorc file:
> 
> 
> 
> [global]
> device = cuda0
> optimizer_including = cudnn
> floatX = float32
> 
> [dnn]
> include_path = /usr/local/cuda/include
> library_path = /usr/local/cuda/lib64
> 
> [lib]
> cnmem = 0.7
> 
> [nvcc]
> fastmath = True
> 
> [blas]
> # Only used for device = cpu
> ldflags = -lopenblas
> 
> [cuda]
> root = /usr/local/cuda/bin
> 
>  
> Nvidia driver: Driver Version: 384.81
> 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com 
>