Re: [theano-users] Compilation Error on Node

[Pascal Lamblin]

On 2018-05-24 06:47 AM, prakhar mishra wrote:

Keras on theano is giving this error. Help!


Which version of Theano are you using?
Also, this message does not specify the actual compilation error. Did 
you get this code from a file? Is there another part of the error 
message somewhere?


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] broadcast_batched_dot

[Pascal Lamblin]

Hi,

You can use batched_tensordot for that, but it assumes that the 
"batched" dimension is the first one, so you'd have to transpose A first 
so that the "5" is first, and then transpose the result back to get C.


So here, you'd do something like:
A_T = A.transpose(1, 0, 2, 3)  # shape = [5, 2, 7, 3]
C_T = T.batched_tensordot(A_T, B, axes=[3, 1])  # "axes" matches the "3" 
between A_T and B, shape = [5, 2, 7, 6]

C = C_T.transpose(1, 0, 2, 3)  # shape = [2, 5, 7, 6]

It seems to work:
>>> C.eval({A: np.zeros((2, 5, 7, 3)), B: np.ones((5, 3, 6))}).shape
(2, 5, 7, 6)


On 2018-05-22 08:56 AM, luke wrote:

Hi all,


I want to achieve a "broadcast batched dot" operation in theano, such 
that the two arguments A and B with shapes


A.shape = [2,5,7,3]
B.shape = [5,3,6]


produce an output C of shape tensor4 [2,5,7,6], with a np equivalent of:

     for i in range(A.shape[0]):
     for j in range(A.shape[1]):
     C[i,j,:,:] = np.dot( A[i,j,:,:], B[j,:,:] )


So, basically, the last two dimensions of A and B are multiplied 
together with dot, dimension 1 of A and 0 of B are batched, and 
dimension 0 of A is broadcasted onto B.
I've played around a bit with T.batched_tensordot, but could not achieve 
this.


The only way I could make this work involves a scan over dimension 0 of 
A, and a T.batched_dot over the remaining 3 dimensions. But this is of 
course dauntingly slow.



Any ideas?


br,
Luke





--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Compilation issue in Theano 1.0.1

[Pascal Lamblin]

The new error is:
> Failed to run "C:/Temp/Anaconda3/Library/mingw-w64/bin" (Access is 
denied.

>).

Which may be consistent with the OS being stricter on security on the 
laptop.


In any case, did I understand correctly that indicating a specific 
subdirectory inside AppData works for both versions?
If so, is that enough to let you run different jobs with different cache 
directories?


On 2018-05-08 10:30 AM, ivan.d.marroq...@gmail.com wrote:
I am using two machines: a laptop and a desktop for testing Theano 
installation. Both machines have windows 7 and it seems to me that they 
were configured identically (including antivirus) but I  will check with 
IT people.


I decided to do the following: remove anaconda and related files from 
both machines, and install anaconda 5.1 - to have a fresh start. Note 
that in the laptop, I installed Theano 0.9.0. While on the desktop, I 
installed  Theano 1.0.1


Then, I conducted these tests:

A) Both laptop and desktop: os.environ["THEANO_FLAGS"]= 
r"base_compiledir=C:\\Users\\IMarroquin\\AppData\\local\\theano\\Model_1"


-> It worked fine on both machines

B) Both laptop and desktop: os.environ["THEANO_FLAGS"]= 
r"base_compiledir=Model_1"


-> on the laptop: it worked fine. The folder "Model_1" got created in 
the directory where the Python script is located. See file 
"Laptop_message.txt" for a listing of the last lines before the script 
started to run.


-> on the desktop: it failed with the same issue, missing pyd file. See 
the compilation error file



Hope this helps!




On Monday, May 7, 2018 at 9:51:37 PM UTC-5, Pascal Lamblin wrote:

It may be another Windows protection mechanism that prevents the
creation of that .pyd file here, but allows it in the default
compiledir. I can understand why such restrictions could exist, since
it's a binary created by another program.

If running the command manually gives the same error, then it does not
seem to be specific to Theano, even though it is weird that version 0.9
would work.

Two things you could try in order to investigate:
- Use a subdirectory of the _default_ base_compiledir, with the full
path. For instance, something like
"C:\\Users\\IMarroquin\\AppData\\local\\theano\\Model_1"
- Get the full compilation command line generated by Theano 0.9 (the
one
that works), and post both, so we can compare.


On 2018-05-04 08:43 AM, ivan.d.m...@gmail.com  wrote:
 > Hi Pascal,
 >
 > I tried the full path with your suggestion. This time, windows
didn't
 > compliant about the length of the filename and the sub-folder
"Model_1"
 > got created. Unfortunately, I still got the same issue about the
missing
 > *.pyd file (I attached again the compilation error file)
 >
 > Then, I ran the command (the version in the attached file) on a
conda
 > window. I got the following output:
 >
 >

C:/Temp/Anaconda3/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:

 > cannot open output file
 > C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_c
 >

ontest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmpyuw3l9gk\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd:

 > No such file or directory
 > collect2.exe: error: ld returned 1 exit status
 >
 > Hope this information  helps,
 >
 > Ivan
 >
 >
 > On Thursday, May 3, 2018 at 5:32:20 PM UTC-5, Pascal Lamblin wrote:
 >
 >     On 2018-05-03 04:50 PM, ivan.d.m...@gmail.com 
wrote:
 >      > Thanks for the suggestion.
 >      >
 >      > I checked the disk space and have ~700Gbytes. Then, I
tested with
 >      > "base_compiledir" using the entire path.
 >      >
 >      > This time,  a sub-folder with this name was created:
 >      >
 >
UsersIMarroquinDocumentsMy_Python_ScriptsMLPSEG_contestIndependent_ScriptsModel_1


 >
 >
 >     backslashes may have not been escaped enough, maybe with
 >     os.environ["THEANO_FLAGS"] = r"basecompiledir=C:\\Users\\ ..."
 >
 >      > I am running the command: python  from a conda
window. I
 >      > attached the compilation error by Theano and the problem
occurs
 >     when it
 >      > tries to open the *.pyd file.
 >
 >     OK, the log file says the attempt command was:
 >       > "C:\Temp\Anaconda3\Library\mingw-w64\bin\g++.exe" -shared
-g -O3
 >     -fno-math-errno -Wno-unused-label -Wno-unused

Re: [theano-users] Compilation issue in Theano 1.0.1

[Pascal Lamblin]

It may be another Windows protection mechanism that prevents the 
creation of that .pyd file here, but allows it in the default 
compiledir. I can understand why such restrictions could exist, since 
it's a binary created by another program.


If running the command manually gives the same error, then it does not 
seem to be specific to Theano, even though it is weird that version 0.9 
would work.


Two things you could try in order to investigate:
- Use a subdirectory of the _default_ base_compiledir, with the full 
path. For instance, something like 
"C:\\Users\\IMarroquin\\AppData\\local\\theano\\Model_1"
- Get the full compilation command line generated by Theano 0.9 (the one 
that works), and post both, so we can compare.



On 2018-05-04 08:43 AM, ivan.d.marroq...@gmail.com wrote:

Hi Pascal,

I tried the full path with your suggestion. This time, windows didn't 
compliant about the length of the filename and the sub-folder "Model_1" 
got created. Unfortunately, I still got the same issue about the missing 
*.pyd file (I attached again the compilation error file)


Then, I ran the command (the version in the attached file) on a conda 
window. I got the following output:


C:/Temp/Anaconda3/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot open output file 
C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_c
ontest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmpyuw3l9gk\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd: 
No such file or directory

collect2.exe: error: ld returned 1 exit status

Hope this information  helps,

Ivan


On Thursday, May 3, 2018 at 5:32:20 PM UTC-5, Pascal Lamblin wrote:

On 2018-05-03 04:50 PM, ivan.d.m...@gmail.com  wrote:
 > Thanks for the suggestion.
 >
 > I checked the disk space and have ~700Gbytes. Then, I tested with
 > "base_compiledir" using the entire path.
 >
 > This time,  a sub-folder with this name was created:
 >

UsersIMarroquinDocumentsMy_Python_ScriptsMLPSEG_contestIndependent_ScriptsModel_1


backslashes may have not been escaped enough, maybe with
os.environ["THEANO_FLAGS"] = r"basecompiledir=C:\\Users\\ ..."

 > I am running the command: python  from a conda window. I
 > attached the compilation error by Theano and the problem occurs
when it
 > tries to open the *.pyd file.

OK, the log file says the attempt command was:
  > "C:\Temp\Anaconda3\Library\mingw-w64\bin\g++.exe" -shared -g -O3
-fno-math-errno -Wno-unused-label -Wno-unused-variable
-Wno-write-strings -march=haswell -mmmx -mno-3dnow -msse -msse2 -msse3
-mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul
-mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm
-mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c
-mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt
-mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1
-mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw
-mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit
-mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64
--param l2-cache-size=25600 -mtune=haswell
-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64
-I"C:\Temp\Anaconda3\lib\site-packages\numpy\core\include"
-I"C:\Temp\Anaconda3\include"
-I"C:\Temp\Anaconda3\lib\site-packages\theano\gof\c_code"
-L"C:\Temp\Anaconda3\libs" -L"C:\Temp\Anaconda3" -o

"C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd"


"C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\mod.cpp"

-lpython36

The name of the "tmp..." directory changed, but that is expected (it is
randomly generated each time).
The file name for the .pyd is the same as before, since it only depends
on the file's content.

Can you try running that command line from the conda window?

 >
 > Also, I tried with this command os.environ["THEANO_FLAGS"]=
 > 'base_compiledir=Model_1' and unfortunately, I ended with the same
 > problem (see attached file).
 >
 > It is interesting that it used to work on Theano 0.9.0
 >
 > I am hoping that you have other suggestions.
 >
 > Thanks 

Re: [theano-users] Compilation issue in Theano 1.0.1

[Pascal Lamblin]

Also, according to this [1], it may be your antivirus preventing the 
creation of the file.


[1] 
https://stackoverflow.com/questions/7655471/ld-exe-cannot-open-output-file-permission-denied


On 2018-05-03 06:32 PM, Pascal Lamblin wrote:

On 2018-05-03 04:50 PM, ivan.d.marroq...@gmail.com wrote:

Thanks for the suggestion.

I checked the disk space and have ~700Gbytes. Then, I tested with 
"base_compiledir" using the entire path.


This time,  a sub-folder with this name was created: 
UsersIMarroquinDocumentsMy_Python_ScriptsMLPSEG_contestIndependent_ScriptsModel_1 



backslashes may have not been escaped enough, maybe with
os.environ["THEANO_FLAGS"] = r"basecompiledir=C:\\Users\\ ..."

I am running the command: python  from a conda window. I 
attached the compilation error by Theano and the problem occurs when 
it tries to open the *.pyd file.


OK, the log file says the attempt command was:
 > "C:\Temp\Anaconda3\Library\mingw-w64\bin\g++.exe" -shared -g -O3 
-fno-math-errno -Wno-unused-label -Wno-unused-variable 
-Wno-write-strings -march=haswell -mmmx -mno-3dnow -msse -msse2 -msse3 
-mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul 
-mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm 
-mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c 
-mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt 
-mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 
-mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw 
-mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit 
-mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64 
--param l2-cache-size=25600 -mtune=haswell 
-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 
-I"C:\Temp\Anaconda3\lib\site-packages\numpy\core\include" 
-I"C:\Temp\Anaconda3\include" 
-I"C:\Temp\Anaconda3\lib\site-packages\theano\gof\c_code" 
-L"C:\Temp\Anaconda3\libs" -L"C:\Temp\Anaconda3" -o 
"C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd" 
"C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\mod.cpp" 
-lpython36


The name of the "tmp..." directory changed, but that is expected (it is 
randomly generated each time).
The file name for the .pyd is the same as before, since it only depends 
on the file's content.


Can you try running that command line from the conda window?



Also, I tried with this command os.environ["THEANO_FLAGS"]= 
'base_compiledir=Model_1' and unfortunately, I ended with the same 
problem (see attached file).


It is interesting that it used to work on Theano 0.9.0

I am hoping that you have other suggestions.

Thanks for all,
Ivan



On Thursday, May 3, 2018 at 3:01:29 PM UTC-5, Pascal Lamblin wrote:

    On 2018-05-03 02:00 PM, ivan.d.m...@gmail.com  wrote:>
    A) Added the
    option nocleanup=True to the .theanorc file

    Thanks. This makes sure the "mod.c" file you mention later is not
    deleted.

 > B) The backslash before "Model 1". In my previous attempts in 
Theano

 > 0.9.0, this was the way that I found to make it work.

    We have added quotation marks and such in order to better handle 
paths

    on Windows since 0.9, so I'm not surprised.

 > Following your suggestion, I edited the header of the script to
 > this: os.environ["THEANO_FLAGS"]= "base_compiledir=Model_1"
 >
 > and ran the script. This time, I noticed an extra sub-folder under
 >

"Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64". 



 > The sub-folder is named "tmpgqmnjspd", which only contains a
    mod.c file

    That makes sense.

 > The compilation ended with this error:
 >
 > Exception: ('The following error happened while compiling the 
node',
 > InplaceDimShuffle{x,0}(layer:tanh-1/bias), '\n', 'Compilation 
failed

 > (return status=1): C:/Temp/Anaconda3/Library/mingw-w64/bin/../
 >

lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 



 > cannot open output file
 >

C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1\ 



 >

\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\\tmpgqmnjspd\\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd: 



 > No suc

Re: [theano-users] Using theano.tensor.repeat with repeats.ndim == 1

[Pascal Lamblin]

Oh, right.

Then, I don't think it will be implemented.
And I don't think RepeatOp is optimized to use the GPU anyway.


Sorry about that

On 2018-05-03 06:01 PM, Kristjan Arumae wrote:
No, it only works with two vectors.  2 Matrixes is not supported in 
numpy either I don't think, since it is likely to mess up the output shape.


On Thursday, May 3, 2018 at 4:03:15 PM UTC-4, Pascal Lamblin wrote:

Does it work with two matrices?
If so, you can try to use dimshuffle to make v a "row" instead of a
"vector".

On 2018-05-03 03:24 PM, Kristjan Arumae wrote:
 > An example of what I am doing:
 >
 > Here m is an fmatrix, and v is an ivector
 >
 > out = T.repeat(m, v, axis=0)
 >
 > The forward pass works fine, but there is no gradient code
implemented.
 >
 > This works fine when both inputs are vectors but not as above.
 >
 > I am not familiar with theano enough to fill in the missing code in
 > grad() for class RepeatOp().  Does anyone have suggestions as to a
 > workaround?  I have not found anything even remotely helpful so far.
 > I've tried using tile with scan, but to no end.
 >
 > Thanks.
 >
 > --
 >
 > ---
 > You received this message because you are subscribed to the Google
 > Groups "theano-users" group.
 > To unsubscribe from this group and stop receiving emails from it,
send
 > an email to theano-users...@googlegroups.com 
 > <mailto:theano-users+unsubscr...@googlegroups.com >.
 > For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.

-- 
Pascal Lamblin


--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Compilation issue in Theano 1.0.1

[Pascal Lamblin]

On 2018-05-03 04:50 PM, ivan.d.marroq...@gmail.com wrote:

Thanks for the suggestion.

I checked the disk space and have ~700Gbytes. Then, I tested with 
"base_compiledir" using the entire path.


This time,  a sub-folder with this name was created: 
UsersIMarroquinDocumentsMy_Python_ScriptsMLPSEG_contestIndependent_ScriptsModel_1


backslashes may have not been escaped enough, maybe with
os.environ["THEANO_FLAGS"] = r"basecompiledir=C:\\Users\\ ..."

I am running the command: python  from a conda window. I 
attached the compilation error by Theano and the problem occurs when it 
tries to open the *.pyd file.


OK, the log file says the attempt command was:
> "C:\Temp\Anaconda3\Library\mingw-w64\bin\g++.exe" -shared -g -O3 
-fno-math-errno -Wno-unused-label -Wno-unused-variable 
-Wno-write-strings -march=haswell -mmmx -mno-3dnow -msse -msse2 -msse3 
-mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul 
-mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm 
-mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c 
-mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt 
-mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 
-mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw 
-mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit 
-mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64 
--param l2-cache-size=25600 -mtune=haswell 
-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 
-I"C:\Temp\Anaconda3\lib\site-packages\numpy\core\include" 
-I"C:\Temp\Anaconda3\include" 
-I"C:\Temp\Anaconda3\lib\site-packages\theano\gof\c_code" 
-L"C:\Temp\Anaconda3\libs" -L"C:\Temp\Anaconda3" -o 
"C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd" 
"C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\mod.cpp" 
-lpython36


The name of the "tmp..." directory changed, but that is expected (it is 
randomly generated each time).
The file name for the .pyd is the same as before, since it only depends 
on the file's content.


Can you try running that command line from the conda window?



Also, I tried with this command os.environ["THEANO_FLAGS"]= 
'base_compiledir=Model_1' and unfortunately, I ended with the same 
problem (see attached file).


It is interesting that it used to work on Theano 0.9.0

I am hoping that you have other suggestions.

Thanks for all,
Ivan



On Thursday, May 3, 2018 at 3:01:29 PM UTC-5, Pascal Lamblin wrote:

On 2018-05-03 02:00 PM, ivan.d.m...@gmail.com  wrote:>
A) Added the
option nocleanup=True to the .theanorc file

Thanks. This makes sure the "mod.c" file you mention later is not
deleted.

 > B) The backslash before "Model 1". In my previous attempts in Theano
 > 0.9.0, this was the way that I found to make it work.

We have added quotation marks and such in order to better handle paths
on Windows since 0.9, so I'm not surprised.

 > Following your suggestion, I edited the header of the script to
 > this: os.environ["THEANO_FLAGS"]= "base_compiledir=Model_1"
 >
 > and ran the script. This time, I noticed an extra sub-folder under
 >

"Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64".

 > The sub-folder is named "tmpgqmnjspd", which only contains a
mod.c file

That makes sense.

 > The compilation ended with this error:
 >
 > Exception: ('The following error happened while compiling the node',
 > InplaceDimShuffle{x,0}(layer:tanh-1/bias), '\n', 'Compilation failed
 > (return status=1): C:/Temp/Anaconda3/Library/mingw-w64/bin/../
 >
lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:

 > cannot open output file
 >

C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1\

 >

\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\\tmpgqmnjspd\\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd:

 > No such file or directory\r. collect2.exe: error: ld returned 1 exit
 > status\r. ', '[InplaceDimShuffle{x,0}(layer:tanh-1/bias)]')
 >
 > It seems that the source of the problem is a missing file with
extension
 > *.pyd

  From my understanding, 

Re: [theano-users] Using theano.tensor.repeat with repeats.ndim == 1

[Pascal Lamblin]

Does it work with two matrices?
If so, you can try to use dimshuffle to make v a "row" instead of a 
"vector".


On 2018-05-03 03:24 PM, Kristjan Arumae wrote:

An example of what I am doing:

Here m is an fmatrix, and v is an ivector

out = T.repeat(m, v, axis=0)

The forward pass works fine, but there is no gradient code implemented.

This works fine when both inputs are vectors but not as above.

I am not familiar with theano enough to fill in the missing code in 
grad() for class RepeatOp().  Does anyone have suggestions as to a 
workaround?  I have not found anything even remotely helpful so far.

I've tried using tile with scan, but to no end.

Thanks.

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Compilation issue in Theano 1.0.1

[Pascal Lamblin]

On 2018-05-03 02:00 PM, ivan.d.marroq...@gmail.com wrote:> A) Added the 
option nocleanup=True to the .theanorc file


Thanks. This makes sure the "mod.c" file you mention later is not deleted.

B) The backslash before "Model 1". In my previous attempts in Theano 
0.9.0, this was the way that I found to make it work.


We have added quotation marks and such in order to better handle paths 
on Windows since 0.9, so I'm not surprised.


Following your suggestion, I edited the header of the script to 
this: os.environ["THEANO_FLAGS"]= "base_compiledir=Model_1"


and ran the script. This time, I noticed an extra sub-folder under 
"Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64". 
The sub-folder is named "tmpgqmnjspd", which only contains a mod.c file


That makes sense.


The compilation ended with this error:

Exception: ('The following error happened while compiling the node', 
InplaceDimShuffle{x,0}(layer:tanh-1/bias), '\n', 'Compilation failed 
(return status=1): C:/Temp/Anaconda3/Library/mingw-w64/bin/../
lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot open output file 
C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1\
\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\\tmpgqmnjspd\\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd: 
No such file or directory\r. collect2.exe: error: ld returned 1 exit 
status\r. ', '[InplaceDimShuffle{x,0}(layer:tanh-1/bias)]')


It seems that the source of the problem is a missing file with extension 
*.pyd


From my understanding, the process is not able to open the *output* 
file (the .pyd) in order to *write* to it, and so the compilation cannot 
proceed.


I find that surprising, since it was able to create the mod.c file.
Maybe it's a quota problem, or not enough free disk space?

Can you specify a full path, for instance:

os.environ["THEANO_FLAGS"] = 
"base_compiledir=C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1"


Another thing to try would be to put a breakpoint or try to print the 
compilation command line that is executed, and try to execute it 
directly from a shell (and the same working directory), rather than 
inside Theano.






C) "compiler" instead of "compiledir". Sorry for the confusion, I made a 
mistake during the copy/paste


Many thanks,
Ivan

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Compilation issue in Theano 1.0.1

[Pascal Lamblin]

If you add "nocleanup=True" to the config options (for instance in the 
.theanorc), can you see more things in Model_1\compiledir...?


I see a backslash before "Model_1" in the config option, is that 
intended, or an artifact of the email? Is it necessary?

What happens if you put the whole path instead?

I see you mention once
compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64,
and then
compiler_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64

with "compiler" instead of "compiledir", is that just a copy/paste error?


On 2018-05-03 09:12 AM, ivan.d.marroq...@gmail.com wrote:

Hi,

I have a windows machine and the content of the .theanorc file is:

[global]
floatX = float32
device = cpu

[nvcc]
fastmath = True
flags = -LC:\Temp\Anaconda3\libs
compiler_bindir = "C:\Temp\Anaconda3\Library\mingw-w64\bin"


[blas]
ldflags = -LC:\Temp\Anaconda3\Library\bin -lmkl_rt

[cuda]
root = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0

[dnn]
include_path = C:\Program Files\NVIDIA GPU Computing 
Toolkit\CUDA\v8.0\include
library_path = C:\Program Files\NVIDIA GPU Computing 
Toolkit\CUDA\v8.0\lib\x64

enabled = True

[lib]
cudnn = 1.0


I used Theano 0.9.0 and was able to set up the configuration option 
"base_compilerdir" inside a script as follows:


import itertools

import numpy as np

from sklearn import metrics, preprocessing

import matplotlib.pyplot as plt

import time

import os


os.environ["THEANO_FLAGS"]= "base_compiledir=\Model_1"


from functools import partial

from hyperopt import hp, Trials, fmin, tpe, STATUS_OK

from neupy import algorithms, layers, init#, environment, plots

from neupy.exceptions import StopTraining

import random

import pandas as pd

#import shutil

import theano


print(theano.config.base_compiledir)

plt.style.use('ggplot')

theano.config.floatX= 'float32'


Theano was able to create a folder named "Model_1" in the path were the 
script was located. Then, Theano was able to compile and run the script 
as expected.



Now, I replaced the old Theano by its newest version 1.0.1. When I run 
the script, I got this message at the end:



Exception: ('The following error happened while compiling the node', 
InplaceDimShuffle{x,0}(layer:tanh-1/bias), '\n', 'Compilation failed 
(return status=1): 
C:/Temp/Anaconda3/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot open output file 
C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1\\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\\tmphss2exfn\\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd: 
No such file or directory\r. collect2.exe: error: ld returned 1 exit 
status\r. ', '[InplaceDimShuffle{x,0}(layer:tanh-1/bias)]')



I noticed the following:

a) the folder "Model_1" got created

b) the sub-folder 
"compiler_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64" 
got created. Inside this sub-folder, I only have another folder named 
"lazylinker.ext" and a file name "__init__"




Then, I comment out these lines from the header in the script:

import os


os.environ["THEANO_FLAGS"]= "base_compiledir=\Model_1"


and run the script. Theano uses the default path: 
C:\Users\IMarroquin\AppData\Local\Theano. Additionally, Theano creates a 
whole series of "tmp" sub-folders right under the folder 
"compiler_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64"





What other configuration variable should I have in my Python script? Any 
other suggestions?



Many thanks,

Ivan






--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] from DownsampleFactorMax to Pool in Theano 1.0.1

[Pascal Lamblin]

On 2018-05-03 04:12 AM, Antonio95100 wrote:

/op = T.signal.downsample.DownsampleFactorMax((ds[1], ds[2]), 
ignore_border)/

/output = op(input_4D)/

with these:

/op = T.signal.pool.Pool(ignore_border)(input, ws=(ds[1],ds[2]))/
/output = op(input_4D)/


The line-to-line equivalent would be:

op = T.signal.pool.Pool(ignore_border)
output = op(input_4D, ws=(ds[1], ds[2]))



and I receive the following error message:


/File "/home//VoxNet/voxnet/voxnet/max_pool_3d.py", line 78, in max_pool_3d/
/    output = op(input_4D)/
/TypeError: 'TensorVariable' object is not callable/
/
/
What is wrong in my update?
Thank you in advance for any suggestion.
Cheers.

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] No matching function for call to batch_gemm

[Pascal Lamblin]

What's the version of g++ you are using?


On 2018-04-21 02:47 PM, Стас Утикеев wrote:
I'm trying to launch some Python projects which I need for my bachelor 
thesis and they use theano.


The problem is, that one of the cell compiles these functions and fails 
with following compiler error: https://pastebin.com/TunvSwWU


|
train_fn 
=theano.function([X_tensor1,X_tensor2,Mask_matrix1,Mask_matrix2, Y_vector],cost,updates=lasagne_adagrad(cost,params_to_optimize,learning_rate=lr),on_unused_input='ignore')
predict_fn 
=theano.function([X_tensor1,X_tensor2,Mask_matrix1,Mask_matrix2],output,on_unused_input='ignore')

|

The strangest thing is that in other notebook I have this code and it 
compiles just fine:


|
train_fn 
=theano.function([X_tensor1,X_tensor2,Mask_matrix1,Mask_matrix2, Y_vector],cost,updates=lasagne_adagrad(cost,[W,U,h0,b,softmax_W,softmax_b],learning_rate=lr),on_unused_input='ignore')
predict_fn 
=theano.function([X_tensor1,X_tensor2,Mask_matrix1,Mask_matrix2],output,on_unused_input='ignore')

|

I'm not really experienced theano user and right now just try to run 
this code before digging into it. What can be the problem?


--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Need help with running simultaneously three neural network scripts

[Pascal Lamblin]

Hi,

This is just an informative warning, because several scripts are trying 
to compile and cache binaries at the same time.
The second time the script is run, the cache should be full and this 
should not happen any more.

In any case, there should be no consequence on the computation or its speed.

If you specify a different cache for each process, you will get rid of 
the messages, but each script will compile its own versions of the 
binaries, so work will be duplicated and it is not clear that the 
computation will actually start sooner.


On 2018-04-11 04:49 PM, ivan.d.marroq...@gmail.com wrote:

Hi all,

I have a windows machine with a kepler card. On this machine, I 
installed Conda 4.3.30 and theano 0.9.0 (along with its required packages).


I used neupy package as a front end to theano for implementing the 
neural network to train.


I would like to run simultaneously three scripts - each of them trains 
the same neural network architecture on three different data sets. My 
first approach was to open three conda terminals to run the scripts.


So, I let the first script to work for a short moment and then, I ran 
the second script. While I am waiting to run the third script, I noticed 
that the second script stopped and there was the following message on 
the conda terminal used to run the first script:


  WARNING (theano.gof.compilelock): Overriding existing lock by dead 
process '13168' (I am process '11140')


The first script just continued working.

any suggestion on how to solve this? or, on how to run several scripts 
simultaneously?


many thanks,

Ivan

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] cannot compile with cuDNN error when importing theano

[Pascal Lamblin]

And you _still_ have the following in your error message?
> cannot find C:/Program: No such file or directory

That is surprising, I don't see where it would get "C:/Program" from.

On 2018-04-11 08:56 AM, Qinpeng Wang wrote:

Hi Pascal,

I have got rid of that line a while ago following your advice, here is 
my new theanoc file after Arnaud's advice:


[global]
floatX = float32
device = cuda

[dnn]
enabled = True

Thanks!


On Tuesday, April 10, 2018 at 4:58:33 PM UTC-5, Pascal Lamblin wrote:

You _also_ need to remove the incorrect "blas.ldflags" line from
.theanorc.

On 2018-04-10 05:19 PM, Qinpeng Wang wrote:
 > Hi, Arnaud,
 >
 > Thanks for your reply! I reinstalled CUDA, copied cudnn files in the
 > cuda directory, and removed the two parameters. However, I got
the same
 > errors from my first post:
 >
 > |ERROR (theano.gpuarray): Could not initialize pygpu, support
disabled
 > Traceback (most recent call last): File "C:\Program
 >
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py",
 > line 227, in  use(config.device) File "C:\Program
 >
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py",
 > line 214, in use init_dev(device, preallocate=preallocate) File
 > "C:\Program
 >
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py",
 > line 121, in init_dev dnn.dnn_available.msg) RuntimeError: You
enabled
 > cuDNN, but we aren't able to use it: cannot compile with cuDNN.
We got
 > this error: b'C:/Program
 >

Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:

 > cannot find C:/Program: No such file or directory C:/Program
 >

Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:

 > cannot find
 >

Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/lib/../lib/default-manifest.o:

 > No such file or directory collect2.exe: error: ld returned 1 exit
status'|
 >
 >
 > Any other suggestion you have?
 >
 > Thanks!
 >
 >
 > On Tuesday, April 10, 2018 at 12:45:59 PM UTC-5, Arnaud Bergeron
wrote:
 >
 >
 >
 >>     Le 3 avr. 2018 à 12:01, Qinpeng Wang <wqp.g...@gmail.com
 >>     > a écrit :
 >>
 >>     Hi Pascal,
 >>
 >>     I have CUDA installed a while ago when I was still using the
old
 >>     GPU backend at C:\Program Files\NVIDIA GPU Computing
Toolkit, then
 >>     I copied the CUDA folder to C root.
 >
 >     This most likely broke DLL registrations.  Please don't do that.
 >
 >     If you reinstall cuda in its original location, put the cudnn
files
 >     in the cuda directory and remove these two parameters in your
 >     theanorc file:
 >
 >     |include_path=C:/CUDA/v7.5/include
library_path=C:/CUDA/v7.5/lib/x64|
 >
 >     It should work again.
 >
 >     ---
 >
 >     I you really want to try with the current setup, you can try
to set this
 >
 >     |base_path=C:/CUDA/v7.5 |
 >
 >     Instead of the two above, but I'm pretty sure that will still
break
 >     on the cuda DLLs.
 >
 >
 >
 >>     Then I installed cuDNN by copying unzipped files into
 >>     corresponding folders inside CUDA folder. I'm using CUDA 7.5
and
 >>     cuDNN 6.0 (Windows7). See my theanoc file in my original
post to
 >>     make sure I have pointed theano to the right location.
 >>     Yes this error is not related to Idflags.
 >>
 >>     Thanks!
 >>
 >>     On Tuesday, April 3, 2018 at 9:59:38 AM UTC-5, Pascal
Lamblin wrote:
 >>
 >>         This is a completely different error message.
 >>         This one has to do with cuDNN, not BLAS or MKL.
 >>
 >>         Where have you installed cuDNN?
 >>         Does _that_ error still happen when you do not ldflags?
 >>
 >>         On 2018-04-03 10:46 AM, Qinpeng Wang wrote:
 >>         > Hi Pascal,
 >>         >
 >>         > Thanks for your reply!
 >>         > I have "mkl-service" package installed via conda (I have
 >>         versions of
 >>         > installed libraries in my original post), but somehow
theano
 >>         is unable
 >&

Re: [theano-users] cannot compile with cuDNN error when importing theano

[Pascal Lamblin]

You _also_ need to remove the incorrect "blas.ldflags" line from .theanorc.

On 2018-04-10 05:19 PM, Qinpeng Wang wrote:

Hi, Arnaud,

Thanks for your reply! I reinstalled CUDA, copied cudnn files in the 
cuda directory, and removed the two parameters. However, I got the same 
errors from my first post:


|ERROR (theano.gpuarray): Could not initialize pygpu, support disabled 
Traceback (most recent call last): File "C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 227, in  use(config.device) File "C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 214, in use init_dev(device, preallocate=preallocate) File 
"C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 121, in init_dev dnn.dnn_available.msg) RuntimeError: You enabled 
cuDNN, but we aren't able to use it: cannot compile with cuDNN. We got 
this error: b'C:/Program 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot find C:/Program: No such file or directory C:/Program 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot find 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/lib/../lib/default-manifest.o: 
No such file or directory collect2.exe: error: ld returned 1 exit status'|



Any other suggestion you have?

Thanks!


On Tuesday, April 10, 2018 at 12:45:59 PM UTC-5, Arnaud Bergeron wrote:




Le 3 avr. 2018 à 12:01, Qinpeng Wang <wqp.g...@gmail.com
> a écrit :

Hi Pascal,

I have CUDA installed a while ago when I was still using the old
GPU backend at C:\Program Files\NVIDIA GPU Computing Toolkit, then
I copied the CUDA folder to C root.


This most likely broke DLL registrations.  Please don't do that.

If you reinstall cuda in its original location, put the cudnn files
in the cuda directory and remove these two parameters in your
theanorc file:

|include_path=C:/CUDA/v7.5/include library_path=C:/CUDA/v7.5/lib/x64|

It should work again.

---

I you really want to try with the current setup, you can try to set this

|base_path=C:/CUDA/v7.5 |

Instead of the two above, but I'm pretty sure that will still break
on the cuda DLLs.




Then I installed cuDNN by copying unzipped files into
corresponding folders inside CUDA folder. I'm using CUDA 7.5 and
cuDNN 6.0 (Windows7). See my theanoc file in my original post to
make sure I have pointed theano to the right location.
Yes this error is not related to Idflags.

Thanks!

On Tuesday, April 3, 2018 at 9:59:38 AM UTC-5, Pascal Lamblin wrote:

This is a completely different error message.
This one has to do with cuDNN, not BLAS or MKL.

Where have you installed cuDNN?
Does _that_ error still happen when you do not ldflags?

On 2018-04-03 10:46 AM, Qinpeng Wang wrote:
> Hi Pascal,
>
> Thanks for your reply!
> I have "mkl-service" package installed via conda (I have
versions of
> installed libraries in my original post), but somehow theano
is unable
> to locate the BLAS library automatically, hence the warning
message.
> Since the "gcc-v" output seems reasonable, could you please
help me
> figure out how to get rid of the error message in my
original post? I
> attach it here again for your convenience.
>
> When I run:
> |python -c "import theano"|
>
> I get this message:
>
> |ERROR (theano.gpuarray): Could not initialize pygpu,
support disabled
> Traceback (most recent call last): File "C:\Program
>
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py",
> line 227, in  use(config.device) File "C:\Program
>
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py",
> line 214, in use init_dev(device, preallocate=preallocate) File
> "C:\Program
>
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py",
> line 121, in init_dev dnn.dnn_available.msg) RuntimeError:
You enabled
> cuDNN, but we aren't able to use it: cannot compile with
cuDNN. We got
> this error: b'C:/Program
>

Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
> cannot find C:/Program: No such file or directory C:/Program
>

File

Re: [theano-users] Got following error when using performed the following code

[Pascal Lamblin]

T.shape is expecting a tensor, not a layer object.

On 2018-04-07 03:47 PM, Ashwini Rahangdale wrote:

for h in H_layers:
         network = lasagne.layers.DenseLayer(network, h, 
nonlinearity=leaky_rectify, W=lasagne.init.GlorotNormal())#network is 
incoming network, h is num of units
     network = lasagne.layers.DenseLayer(network, len(y_train), 
nonlinearity=sigmoid, W=lasagne.init.GlorotNormal())

     params_original = lasagne.layers.get_all_param_values(network)
     params = lasagne.layers.get_all_params(network, trainable=True)
     layers = lasagne.layers.get_all_layers(network)
     get_params = lasagne.layers.get_all_param_values(network)

#define loss function
     prediction = lasagne.layers.get_output(network)
     dim =  lasagne.layers.get_output_shape(network)
     loss = lasagne.objectives.categorical_crossentropy(prediction, 
target_var)


Hello,
In the above code, I want the predicted value at the output layer. Thus, 
I am trying to get it using T.sahpe(network).eval and T.eval(prediction)


But using these command I am getting following errors and unable to 
locate it. Please help to configure this error.



  T.shape(network).eval()
Traceback (most recent call last):
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\type.py", 
line 269, in dtype_specs

     }[self.dtype]
KeyError: 'object'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\basic.py", 
line 246, in constant

     ttype = TensorType(dtype=x_.dtype, broadcastable=bcastable)
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\type.py", 
line 51, in __init__

     self.dtype_specs()  # error checking is done there
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\type.py", 
line 272, in dtype_specs

     % (self.__class__.__name__, self.dtype))
TypeError: Unsupported dtype for TensorType: object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\basic.py", 
line 194, in as_tensor_variable

     return constant(x, name=name, ndim=ndim)
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\basic.py", 
line 266, in constant

     raise TypeError("Could not convert %s to TensorType" % x, type(x))
TypeError: ('Could not convert at 0x0263DD6F7828> to TensorType', 'lasagne.layers.dense.DenseLayer'>)


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File "<pyshell#37>", line 1, in 
     T.shape(network).eval()
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\op.py", 
line 615, in __call__

     node = self.make_node(*inputs, **kwargs)
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\compile\ops.py", 
line 252, in make_node

     x = theano.tensor.as_tensor_variable(x)
   File 
"C:\Users\Ashwini\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\basic.py", 
line 200, in as_tensor_variable

     raise AsTensorError("Cannot convert %s to TensorType" % str_x, type(x))
theano.tensor.var.AsTensorError: ('Cannot convert 
 to 
TensorType', )



Please reply as soon as possibel.

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] cannot compile with cuDNN error when importing theano

[Pascal Lamblin]

This is a completely different error message.
This one has to do with cuDNN, not BLAS or MKL.

Where have you installed cuDNN?
Does _that_ error still happen when you do not ldflags?

On 2018-04-03 10:46 AM, Qinpeng Wang wrote:

Hi Pascal,

Thanks for your reply!
I have "mkl-service" package installed via conda (I have versions of 
installed libraries in my original post), but somehow theano is unable 
to locate the BLAS library automatically, hence the warning message.
Since the "gcc-v" output seems reasonable, could you please help me 
figure out how to get rid of the error message in my original post? I 
attach it here again for your convenience.


When I run:
|python -c "import theano"|

I get this message:

|ERROR (theano.gpuarray): Could not initialize pygpu, support disabled 
Traceback (most recent call last): File "C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 227, in  use(config.device) File "C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 214, in use init_dev(device, preallocate=preallocate) File 
"C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 121, in init_dev dnn.dnn_available.msg) RuntimeError: You enabled 
cuDNN, but we aren't able to use it: cannot compile with cuDNN. We got 
this error: b'C:/Program 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot find C:/Program: No such file or directory C:/Program 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot find 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/lib/../lib/default-manifest.o: 
No such file or directory collect2.exe: error: ld returned 1 exit status' |



Thanks!


On Tuesday, April 3, 2018 at 9:28:37 AM UTC-5, Pascal Lamblin wrote:

On 2018-03-30 11:16 PM, Qinpeng Wang wrote:
 > Hi Pascal,
 >
 > Thanks for your reply!! Indeed I wasn't really sure how to set the
 > ldflags parameter, but if I don't have this line, I get this warning
 > message:
 >
 > WARNING (theano.tensor.blas): Using NumPy C-API based
implementation for
 > BLAS functions.

This is just a warning, the worse issue would be a slight slow down in
some CPU operations.

 > After inspecting theano/tensor/blas_headers.py
 >

<https://github.com/Theano/Theano/blob/82a3ec977b7c8630d92871e70d62e5a2c52cea62/theano/tensor/blas_headers.py

<https://github.com/Theano/Theano/blob/82a3ec977b7c8630d92871e70d62e5a2c52cea62/theano/tensor/blas_headers.py>>,

 > I found the following:
 >
 > |if not config.blas.ldflags: logger.warning('Using NumPy C-API based
 > implementation for BLAS functions.')|
 >
 > So I realize I may have to have ldflags in the theaorc file to
get rid
 > of this warning message. *How to set ldflags correctly?*

These flags are for specifying where an optimized BLAS library is, when
it is not found automatically.
If you do not have an optimized BLAS library installed, then it should
stay empty.

Did you install the "mkl-service" package in conda?
This is usually the easiest way to install an optimized BLAS that will
be automatically found.

 > I was asking around what the correct output of "gcc -v" would
look like,
 > but have not got any reply yet, here it is:

This seems reasonable.

 >
 > |Using built-in specs. COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=C:/Program\
 >

Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/lto-wrapper.exe

 > Target: x86_64-w64-mingw32 Configured with: ../gcc-5.3.0/configure
 > --prefix=/mingw64 --with-local-prefix=/mingw64/local
 > --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32
 > --target=x86_64-w64-mingw32
 > --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include
 > --libexecdir=/mingw64/lib
 > --with-gxx-include-dir=/mingw64/include/c++/5.3.0 --enable-bootstrap
 > --with-arch=x86-64 --with-tune=generic
 > --enable-languages=c,lto,c++,objc,obj-c++,fortran,ada
--enable-shared
 > --enable-static --enable-libatomic --enable-threads=posix
 > --enable-graphite --enable-fully-dynamic-string
 > --enable-libstdcxx-time=yes --disable-libstdcxx-pch
 > --disable-libstdcxx-debug --enable-version-specific-runtime-libs
 > --disable-isl-version-check --enable-lto --enable-libgomp
 > --disable-multilib --enable-checking=release --disable-rpath
 > --disable-win32-registry --disable-nls --disable-werror

Re: [theano-users] cannot compile with cuDNN error when importing theano

[Pascal Lamblin]

On 2018-03-30 11:16 PM, Qinpeng Wang wrote:

Hi Pascal,

Thanks for your reply!! Indeed I wasn't really sure how to set the 
ldflags parameter, but if I don't have this line, I get this warning 
message:


WARNING (theano.tensor.blas): Using NumPy C-API based implementation for 
BLAS functions.


This is just a warning, the worse issue would be a slight slow down in 
some CPU operations.


After inspecting theano/tensor/blas_headers.py 
<https://github.com/Theano/Theano/blob/82a3ec977b7c8630d92871e70d62e5a2c52cea62/theano/tensor/blas_headers.py>, 
I found the following:


|if not config.blas.ldflags: logger.warning('Using NumPy C-API based 
implementation for BLAS functions.')|


So I realize I may have to have ldflags in the theaorc file to get rid 
of this warning message. *How to set ldflags correctly?*


These flags are for specifying where an optimized BLAS library is, when 
it is not found automatically.
If you do not have an optimized BLAS library installed, then it should 
stay empty.


Did you install the "mkl-service" package in conda?
This is usually the easiest way to install an optimized BLAS that will 
be automatically found.


I was asking around what the correct output of "gcc -v" would look like, 
but have not got any reply yet, here it is:


This seems reasonable.



|Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=C:/Program\ 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/lto-wrapper.exe 
Target: x86_64-w64-mingw32 Configured with: ../gcc-5.3.0/configure 
--prefix=/mingw64 --with-local-prefix=/mingw64/local 
--build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 
--target=x86_64-w64-mingw32 
--with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include 
--libexecdir=/mingw64/lib 
--with-gxx-include-dir=/mingw64/include/c++/5.3.0 --enable-bootstrap 
--with-arch=x86-64 --with-tune=generic 
--enable-languages=c,lto,c++,objc,obj-c++,fortran,ada --enable-shared 
--enable-static --enable-libatomic --enable-threads=posix 
--enable-graphite --enable-fully-dynamic-string 
--enable-libstdcxx-time=yes --disable-libstdcxx-pch 
--disable-libstdcxx-debug --enable-version-specific-runtime-libs 
--disable-isl-version-check --enable-lto --enable-libgomp 
--disable-multilib --enable-checking=release --disable-rpath 
--disable-win32-registry --disable-nls --disable-werror 
--disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 
--with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 
--with-pkgversion='Rev5, Built by MSYS2 project' 
--with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as 
--with-gnu-ld Thread model: posix gcc version 5.3.0 (Rev5, Built by 
MSYS2 project)|



Lastly, I have restarted the computer, the error message stayed the same.

Thank you so much!

On Friday, March 30, 2018 at 9:38:11 PM UTC-5, Pascal Lamblin wrote:

Your "ldflags" option in the "[blas]" section is not correct, it should
at least start with -I or -l, and there should be quotation marks
around
the path since there is a space in "Program Files".

What happens if you do not include this line in the .theanorc?

Are you able to run `gcc -v` from inside the conda env, and what is the
output?

Also, it may seem stupid, but have you restarted your computer, or at
least the terminal, since you installed m2w64-toolchain?

On 2018-03-30 01:27 PM, Qinpeng Wang wrote:
 > Hi, all,
 >
 >
 > I'm trying to setup theano 1.0.1 with CUDA 7.5 and cuDNN 6.0
(Windows7).
 > I created a new virtual environment with conda and installed
required
 > libraries like this:
 >
 > |conda install numpy scipy mkl-service libpython m2w64-toolchain
conda
 > install theano pygpu |
 >
 > Here are versions of major relavant libraries:
 >
 > |theano 1.0.1 python 3.5.5 numpy 1.14.2 scipy 1.0.0
m2w64-toolchain 5.3.0
 > mkl-service 1.1.2 pygpu 0.7.5 libgpuarray 0.7.5 libpython 2.1 mkl
2018.0.2 |
 >
 > The way I installed cuDNN is to copy files into corresponding
folders
 > inside CUDA folder. Originally I installed CUDA at C:\Program
 > Files\NVIDIA GPU Computing Toolkit, then I copied the CUDA folder
to C root.
 >
 > When I run:
 > |python -c "import theano"|
 >
 > I get this message:
 >
 > |ERROR (theano.gpuarray): Could not initialize pygpu, support
disabled
 > Traceback (most recent call last): File "C:\Program
 >
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py",
 > line 227, in  use(config.device) File "C:\Program
 >
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py",
 > line 214, in use init_dev(device, preallocate=preallocate) File
 > "C:\Program
 

Re: [theano-users] cannot compile with cuDNN error when importing theano

[Pascal Lamblin]

Your "ldflags" option in the "[blas]" section is not correct, it should 
at least start with -I or -l, and there should be quotation marks around 
the path since there is a space in "Program Files".


What happens if you do not include this line in the .theanorc?

Are you able to run `gcc -v` from inside the conda env, and what is the 
output?


Also, it may seem stupid, but have you restarted your computer, or at 
least the terminal, since you installed m2w64-toolchain?


On 2018-03-30 01:27 PM, Qinpeng Wang wrote:

Hi, all,


I'm trying to setup theano 1.0.1 with CUDA 7.5 and cuDNN 6.0 (Windows7).
I created a new virtual environment with conda and installed required 
libraries like this:


|conda install numpy scipy mkl-service libpython m2w64-toolchain conda 
install theano pygpu |


Here are versions of major relavant libraries:

|theano 1.0.1 python 3.5.5 numpy 1.14.2 scipy 1.0.0 m2w64-toolchain 5.3.0 
mkl-service 1.1.2 pygpu 0.7.5 libgpuarray 0.7.5 libpython 2.1 mkl 2018.0.2 |


The way I installed cuDNN is to copy files into corresponding folders 
inside CUDA folder. Originally I installed CUDA at C:\Program 
Files\NVIDIA GPU Computing Toolkit, then I copied the CUDA folder to C root.


When I run:
|python -c "import theano"|

I get this message:

|ERROR (theano.gpuarray): Could not initialize pygpu, support disabled 
Traceback (most recent call last): File "C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 227, in  use(config.device) File "C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 214, in use init_dev(device, preallocate=preallocate) File 
"C:\Program 
Files\Anaconda3\envs\te\lib\site-packages\theano\gpuarray\__init__.py", 
line 121, in init_dev dnn.dnn_available.msg) RuntimeError: You enabled 
cuDNN, but we aren't able to use it: cannot compile with cuDNN. We got 
this error: b'C:/Program 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot find C:/Program: No such file or directory C:/Program 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
cannot find 
Files/Anaconda3/envs/te/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/lib/../lib/default-manifest.o: 
No such file or directory collect2.exe: error: ld returned 1 exit status' |


My theanoc file:

|[global] floatX = float32 device = cuda [dnn] enabled = True 
include_path=C:/CUDA/v7.5/include library_path=C:/CUDA/v7.5/lib/x64 
[blas] ldflags = C:/Program Files/Anaconda3/envs/t/Library/bin mkl_rt |


Can somebody help please?


Thanks!

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] TypeError: 'KeysView' object does not support indexing

[Pascal Lamblin]

It does not seem to have anything to do with Theano, but in Python3, 
`dataset.keys()` would return a KeysView object, instead of a list like 
in Python2.

That may be the reason.

On 2018-03-22 04:32 PM, Vinay Rajpoot wrote:
i am working on project and occuring this traceback can anybody suggest 
me what changes should i make in below section of code

     for index in indices:
     key = dataset_keys[index]
     data_x = dataset[key]['features'][...].astype(_DTYPE)
     L_distance_mat = cdist(data_x, data_x, 'euclidean')
     L_dissim_mat = 1 - np.dot(data_x, data_x.T)

Traceback (most recent call last):
   File "vsum_train.py", line 155, in 
     train_dataset_path=args.dataset)
   File "vsum_train.py", line 83, in train
     key = dataset_keys[index]
TypeError: 'KeysView' object does not support indexing

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Regarding theano error

[Pascal Lamblin]

That particular convolution might not be supported by cuDNN.
I would advise you to update cuDNN to a more recent version if possible, 
it will probably help.


On 2018-03-14 04:44 AM, SNEHA RAJU BUNDELE wrote:



Dear Sir/Mam,

I am working on the UNET model with theano as backend with cuda8 and 
cudnn5.1 configuration. It throws following error. please advice me fore 
solving this.



I am working on UNET model.I am getting below error. Can someone please 
advise further.


RuntimeError: error getting worksize: CUDNN_STATUS_BAD_PARAM
Apply node that caused the error: GpuDnnConv{algo='small', inplace=True, 
num_groups=1}(GpuContiguous.0, GpuContiguous.0, 
GpuAllocEmpty{dtype='float32', context_name=None}.0, 
GpuDnnConvDesc{border_mode='half', subsample=(1, 1), dilation=(1, 1), 
conv_mode='conv', precision='float32', num_groups=1}.0, Constant{1.0}, 
Constant{0.0})

Toposort index: 402
Inputs types: [GpuArrayType(float32, 4D), GpuArrayType(float32, 4D), 
GpuArrayType(float32, 4D), 0x7f9e7941f5d0>, Scalar(float32), Scalar(float32)]
Inputs shapes: [(2, 1, 512, 512), (32, 1, 3, 3), (2, 32, 512, 512), 'No 
shapes', (), ()]
Inputs strides: [(1048576, 1048576, 2048, 4), (36, 36, 12, 4), 
(33554432, 1048576, 2048, 4), 'No strides', (), ()]
Inputs values: ['not shown', 'not shown', 'not shown', NULL at 0x7f9e5708fe10>, 1.0, 0.0]
Outputs clients: [[GpuElemwise{add,no_inplace}(GpuDnnConv{algo='small', 
inplace=True, num_groups=1}.0, InplaceGpuDimShuffle{x,0,x,x}.0)]]


Backtrace when the node is created(use Theano flag traceback.limit=N to 
make it longer):
File 
"/home/coe/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", 
line 705, in runfile

execfile(filename, namespace)
File 
"/home/coe/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", 
line 94, in execfile

builtins.execfile(filename, *where)
File "/home/coe/UNET_CODE FROM 
GITHUB/Luna2016-Lung-Nodule-Detection-master/UNET/Code/LUNA_unet.py", 
line 208, in

model = train(False)
File "/home/coe/UNET_CODE FROM 
GITHUB/Luna2016-Lung-Nodule-Detection-master/UNET/Code/LUNA_unet.py", 
line 188, in train

model = get_unet_small(options)
File "/home/coe/UNET_CODE FROM 
GITHUB/Luna2016-Lung-Nodule-Detection-master/UNET/Code/LUNA_unet.py", 
line 72, in get_unet_small
conv1 = Convolution2D(32, options.filter_width, options.stride, 
activation='elu',border_mode='same')(inputs)
File 
"/home/coe/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", 
line 617, in *call*

output = self.call(inputs, **kwargs)
File 
"/home/coe/anaconda2/lib/python2.7/site-packages/keras/layers/convolutional.py", 
line 168, in call

dilation_rate=self.dilation_rate)
File 
"/home/coe/anaconda2/lib/python2.7/site-packages/keras/backend/theano_backend.py", 
line 1916, in conv2d

filter_dilation=dilation_rate)

HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint 
and storage map footprint of this apply node.


Thanks in advance..!!

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Pascal Lamblin]

OK, thanks.

self.truncate_gradient should not be a Python float, it should be an 
integer.
This is probably why the dtype of grad_steps is float64, instead of 
int64 (or another integer dtype).


Do you have any idea why self.truncate_gradient would not be "-1" (the 
default value)? Did you set it?


On 2018-03-06 02:07 PM, Siddhartha Saxena wrote:
The dtype of  grad_steps and s_ is float64 while self.truncate_gradient 
is a python float.

Sorry I didn't answer it properly previously.

Thanks

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Pascal Lamblin]

OK, but what is the `dtype` (data type) of those variables?

On 2018-03-06 01:48 PM, Siddhartha Saxena wrote:
grad_steps itself is of  with 
value "Elemwise{minimum,no_inplace}.0". So here a tensor that is s_ ( of 
type Subtensor{::int64}.0}) is being sliced by a variable. Again how it 
is reaching there is what i am unable to understand.


Thanks

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Pascal Lamblin]

  File 
"/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py", 
line 2250, in L_op

     outer_inp_seqs = [s_[:grad_steps] for s_ in outer_inp_seqs]
   File "/usr/local/lib/python2.7/dist-packages/theano/tensor/var.py", 
line 519, in __getitem__

     theano.tensor.subtensor.Subtensor.convert(arg)
   File 
"/usr/local/lib/python2.7/dist-packages/theano/tensor/subtensor.py", 
line 378, in convert

     slice_b = Subtensor.convert(b, False)
   File 
"/usr/local/lib/python2.7/dist-packages/theano/tensor/subtensor.py", 
line 348, in convert

     raise TypeError("Expected an integer")
TypeError: Expected an integer




--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Getting an error while being new to theano

[Pascal Lamblin]

On 2018-02-26 08:58 AM, Loïc wrote:

Hello,

Sorry for the delay but I got sick and stuck at home for a few days.

Indeed I am using v0.6.0 of Theano merely because it is the version that 
the author used. Plus they added that if someone wanted to update Theano 
he will have to find a new way to downsample.


In that case, you will probably need to use an old version of numpy as 
well, since the error seems to be raised from numpy.


In fact, I have tested with Theano 1.0.1 and I do get an error earlier 
in the program as it seem that in version 1.0.1 the downsample method 
from theano.tensor.signal module does not exist anymore. Are you aware 
of a workaround ? Maybe a new way to downsample has replaced this 
obsolete one ?


Yes, the module has been renamed to "pool": 
http://deeplearning.net/software/theano/library/tensor/signal/pool.html




Anyway thank you for your previous answer, I will let you know if I can 
get theano version 1.0.1 to downsample as the 0.6.0 used to and if it 
also resolves the initial issue.



--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Getting an error while being new to theano

[Pascal Lamblin]

You seem to be using Theano 0.6.0.
Only version 1.0.1 is still supported.

Does the error still occur with 1.0.1?

On 2018-02-21 11:26 AM, Loïc wrote:

Hi,

I have downloaded an open source code which uses theano 
(https://www.repository.cam.ac.uk/handle/1810/263961)


When I try running the learning step I get an error (I have set theano 
verbosity to high).



python train_bnn.py -i lytroPatches_30x30.pkl.gz

|

Traceback(most recent call last):
File"train_bnn.py",line 525,in
      evaluate_dcnn(dataset=dataset_file,nkerns=kernels)
File"train_bnn.py",line 440,inevaluate_dcnn
print("res = {}".format(validate_model(0)))
File"/home/PRODUITS/COTS/python2.7.9/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/compile/function_module.py",line 
579,in__call__

      outputs =self.fn()
File"/home/PRODUITS/COTS/python2.7.9/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/compile/debugmode.py",line 
2030,indeco

returnf()
File"/home/PRODUITS/COTS/python2.7.9/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/compile/debugmode.py",line 
1790,inf

      thunk_py()
File"/home/PRODUITS/COTS/python2.7.9/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/compile/debugmode.py",line 
1623,in

      n=node:p(n,[x[0]forx ini],o))
File"/home/PRODUITS/COTS/python2.7.9/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/tensor/basic.py",line 
1418,inperform

      max[0]=theano._asarray(numpy.max(x,axis),
File"/home/PRODUITS/COTS/python2.7.9/lib/python2.7/site-packages/numpy/core/fromnumeric.py",line 
2320,inamax

out=out,**kwargs)
File"/home/PRODUITS/COTS/python2.7.9/lib/python2.7/site-packages/numpy/core/_methods.py",line 
26,in_amax

returnumr_maximum(a,axis,None,out,keepdims)

TypeError:only integer scalar arrays can be converted to a scalar index
Applynode that caused the 
error:MaxAndArgmax(Softmax.0,TensorConstant{(1,)of 1})

Inputsshapes:[(100,2),(1,)]
Inputsstrides:[(16,8),(8,)]
Inputstypes:[TensorType(float64,matrix),TensorType(int64,(True,))]
Debugprintof the apply node:
MaxAndArgmax.0[@A]<TensorType(float64,vector)>'max'
MaxAndArgmax.1[@A]<TensorType(int64,vector)>'argmax'
|



Now I am really new to Theano and do not even have dnn experience which 
makes it hard to understand the messages I get from Theano and to debug 
this code...
As far as I know this could even be not related to Theano (though I hope 
I am not wasting your time).


What I think I have understood is that the train_model theano.function 
goes well. But the validate_model and test_model theano.function do not.
I think this is due to the errors() method of the LogisticRegression 
layer (the negative_log_likelihood is used what I guess is called the 
cost function for the train_model, while the errors one is the function 
that is supposed to test how well the dnn has learnt so far).



|
classLogisticRegression(object):
''' Logistic regression layer
      '''

# initialization
def__init__(self,input,n_in,n_out):

# initialize the weight matrix to zeros
self.W =theano.shared(value=np.zeros((n_in,n_out),
                                                dtype=theano.config.floatX),
                                 name='W',borrow=True)

# initialize the bias term
self.b =theano.shared(value=np.zeros((n_out,),
                                                dtype=theano.config.floatX),
                                 name='b',borrow=True)

# define symbolic expressions for probability and prediction
self.p_y_given_x =T.nnet.softmax(T.dot(input,self.W)+self.b)

self.y_pred =T.argmax(self.p_y_given_x,axis=1)

self.params=[self.W,self.b]

defnegative_log_likelihood(self,y):
# the loss function
return-T.mean(T.log(self.p_y_given_x)[T.arange(y.shape[0]),y])

deferrors(self,y):

# check if y has same dimension of y_pred
ify.ndim !=self.y_pred.ndim:
raiseTypeError(
'y should have the same shape as self.y_pred',
('y',y.type,'y_pred',self.y_pred.type)
)
# check if y is of the current data type
ify.dtype.startswith('int'):
# compute error
returnT.mean(T.neq(self.y_pred,y))
else:
print("NotImplementedError for LogisticRegression::errors()")
raiseNotImplementedError()
|



So here I come to you hopping that maybe you will have hints or clues to 
help me find out where this error comes from.


Could it be related to the shapes of the tensors ? How can I access them 
? I have truble playing with tensors so far...


I really thank you for the time you will spend reading this and 
answering it.



--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message bec

Re: [theano-users] Using NumPy C-API based implementation for BLAS functions.

[Pascal Lamblin]

Hi,

Usually, Theano tries to link directly with a BLAS library (MKL, 
OpenBlas...) if it is able to detect one, and use it for dot products on 
CPU.
If it does not, it uses the fallback of using the C API of numpy 
instead, which can be slower and result in more memory copies.


On 2017-11-23 05:00 AM, Mathias Müller wrote:


Hi,

With the newest Theano, I get the following warning:

|
WARNING (theano.tensor.blas):UsingNumPyC-API based implementation 
forBLAS functions.

|

What does this message mean? Does that mean there is an alternative to 
Numpy C-API based BLAS functions?


Thanks a lot for your help.
Mathias

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Using Theano and cuDNN with clang-3.8 in a linux environment

[Pascal Lamblin]

Theano 0.8 is not supported any more. Can you use 1.0?

On 2017-11-22 05:44 AM, Antonio Abello wrote:

Hello!

I'm using Theano 0.8.0 on a machine that I have no control over and has 
an older version of Debian installed (Debian Squeeze - 6.3.0).
It does not have a low enough gcc version to run cuda-7.5 (<= 4.9) nor 
it has a high enough version to run cuda 8
Whenever I tried to import theano it said that clang-3.8 was available 
and it should be used instead. So I included this on my .theanorc and it 
worked:


[global]
device=gpu
floatX = float32
[nvcc]
flags = -ccbin=clang-3.8
[dnn]
include_path="/home/$USER/cuda/include"
library_path="/home/$USER/cuda/lib64"

But I noticed it didn't use cuDNN, so it was significantly slower. When 
I enforced cuDNN use with optimizer_including=cudnn I got:


AssertionError: cuDNN optimization was enabled, but Theano was not able 
to use it. We got this error:

Theano can not compile with cuDNN. We got this error:
ERROR: No supported gcc/g++ host compiler found, but clang-3.8 is 
available. Use 'nvcc -ccbin clang-3.8' to use that instead.\n"


Which is incidentaly the exact same error I got in the beggining.
I believe gcc/g++ is used as the default option whenever theano is on a 
Linux environment. Is there a way to force Theano to use clang-3.8 to 
compile cuDNN instead? Is there something else I'm missing here? 
Unfortunately upgrading Theano is not an option since the NN lib im 
using (Lasagne) has not updated to Theano 1.0


Thanks,
Antonio

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] How to save this error :"TypeError: bool output not supported" ?

[Pascal Lamblin]

 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/blocks/blocks/bricks/base.py", 
line 312, in apply

     outputs = self.application_function(brick, *args, **kwargs)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/blocks/blocks/bricks/recurrent.py", 
line 133, in recurrent_apply

     return application_function(brick, *args, **kwargs)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/blocks/blocks/bricks/attention.py", 
line 703, in do_apply

     self.preprocessed_attended_name: preprocessed_attended}))
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/blocks/blocks/bricks/base.py", 
line 377, in __call__

     return self.application.apply(self, *args, **kwargs)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/blocks/blocks/bricks/base.py", 
line 312, in apply

     outputs = self.application_function(brick, *args, **kwargs)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/blocks/blocks/bricks/attention.py", 
line 616, in take_glimpses

     **dict_union(states, glimpses_needed))
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/blocks/blocks/bricks/base.py", 
line 377, in __call__

     return self.application.apply(self, *args, **kwargs)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/blocks/blocks/bricks/base.py", 
line 312, in apply

     outputs = self.application_function(brick, *args, **kwargs)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/lvsr/bricks/attention.py", 
line 141, in take_glimpses

     ali_median_pos = ali_to_05[:,1:] - ali_to_05[:,:-1]
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/Theano/theano/tensor/var.py", 
line 147, in __sub__

     return theano.tensor.basic.sub(self, other)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/Theano/theano/gof/op.py", 
line 615, in __call__

     node = self.make_node(*inputs, **kwargs)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/Theano/theano/tensor/elemwise.py", 
line 578, in make_node

     DimShuffle, *inputs)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/Theano/theano/tensor/elemwise.py", 
line 520, in get_output_info

     for i in inputs])
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/Theano/theano/scalar/basic.py", 
line 1041, in make_node

     for input in inputs])]
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/Theano/theano/scalar/basic.py", 
line 1049, in output_types

     variables = self.output_types_preference(*types)
   File 
"/nobackup/f1/asr/wulong/tensorflowEnv/attention-lvcsr-master/libs/Theano/theano/scalar/basic.py", 
line 842, in upcast_out_nobool

     raise TypeError("bool output not supported")
TypeError: bool output not supported

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Separate 2d convolution over different channels

[Pascal Lamblin]

I think the grouped convolutions, that have been merged in the 
development version this summer, should do what you need.

The speed may not be optimal, but it should be faster than using scan.

On 2017-10-19 12:48 PM, Peter O'Connor wrote:
Hi.  Did anyone figure this out?  I really need this op.  In my case I 
want to compute the per-channel cross correlation of two (512, 20, 20) 
feature maps.


Currently, I can't see any way to do this without using scan.  My 
current solution is:


class ChannelwiseCrossCorr(object):

 def __init__(self, border_mode='full', subsample=(1, 1)):
 self.border_mode = border_mode
 self.subsample = subsample

 def __call__(self, (x1, x2)):
 """
:param(x1, x2): are each (n_samples, n_channels, size_y, size_x) images
:return: A (n_samples, n_channels, size_y*2-1, size_x*2-1) image 
representing the channelwise cross-correlation between

each pair of images.
"""
from theano.tensor.signal.convimport conv2das sconv2d
 x1_flat = x1.reshape((x1.shape[0]*x1.shape[1], x2.shape[2], 
x2.shape[3]))
 x2_flat = x2.reshape((x2.shape[0]*x2.shape[1], x2.shape[2], 
x2.shape[3]))[:, ::-1, ::-1]
 map_flat, _ = theano.scan(partial(sconv2d, 
border_mode=self.border_mode, subsample=self.subsample), sequences=[x1_flat, 
x2_flat])
 conv_maps = map_flat.reshape((x1.shape[0], x1.shape[1], 
map_flat.shape[1], map_flat.shape[2]))
 return conv_maps


But it runs much slower than I expected, presumably due to the scan op.

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] GPU Exception : GpuArrayException: ('nvrtcCompileProgram: NVRTC_ERROR_INVALID_OPTION', 3)

[Pascal Lamblin]

Which version of CUDA are you using?
It may be an old one since cuDNN 5.0 seems to be used. That version may 
be incompatible with the latest libgpuarray.

Can you update?

On 2017-10-15 12:36 AM, Vaibhav Khanduja wrote:

I am getting exception while executing gpu_test.py program.


I am using bleeding edge theano and 0.7.2 version for pygpu and libgpuarray


Using cuDNN version 5005 on context None

ERROR (theano.gpuarray): Could not initialize pygpu, support disabled

Traceback (most recent call last):

File 
"/usr/local/lib/python2.7/dist-packages/theano/gpuarray/__init__.py", 
line 220, in 


use(config.device)

File 
"/usr/local/lib/python2.7/dist-packages/theano/gpuarray/__init__.py", 
line 207, in use


init_dev(device, preallocate=preallocate)

File 
"/usr/local/lib/python2.7/dist-packages/theano/gpuarray/__init__.py", 
line 152, in init_dev


pygpu.blas.gemm(0, tmp, tmp, 0, tmp, overwrite_c=True)

File "pygpu/blas.pyx", line 149, in pygpu.blas.gemm

File "pygpu/blas.pyx", line 47, in pygpu.blas.pygpu_blas_rgemm

GpuArrayException: ('nvrtcCompileProgram: NVRTC_ERROR_INVALID_OPTION', 3)

[Elemwise{exp,no_inplace}(<TensorType(float32, vector)>)]

Looping 1000 times took 1.544191 seconds

Result is [ 1.231780291.618793371.52278066 ...,2.207718132.29967761

1.62323284]

Used the cpu


--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Strange theano / C++ problem

[Pascal Lamblin]

C:\Anaconda3\lib\site-packages\numpy\core\include/numpy/npy_3kcompat.h:288:41:
error: ‘lseek’ was not declared in this scope
if (npy_lseek(fd, orig_pos, SEEK_SET) == -1) {
^

--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano gradient of subtensor

[Pascal Lamblin]

Maybe the following can help you.

http://deeplearning.net/software/theano/tutorial/faq_tutorial.html#how-to-update-a-subset-of-weights

Also, if you take a subtensor of the gradient itself, some optimizations 
can apply that would avoid the computation of the full gradient.


For instance, with your example, the "subtensor" and "* 2" operations 
are swapped:


>>> grad0 = full_grad[0]
>>> g0 = theano.function([X, Y], grad0)

>>> theano.printing.debugprint(g0)
Elemwise{mul,no_inplace} [id A] ''   1
 |TensorConstant{(1,) of 2.0} [id B]
 |Subtensor{int64} [id C] ''   0
   |<TensorType(float64, matrix)> [id D]
   |Constant{0} [id E]


On 2017-09-27 05:25 PM, Daniel Hernandez wrote:

Hi,

I was wondering if someone here had an answer to this unsolved question 
over in stack overflow:


https://stackoverflow.com/questions/37545325/theano-gradient-of-subtensor

Basically, how do you compute gradients w.r.t. a subtensor?

The question arises in the context of large tensors, say Y and X, where 
it is known that each entry in Y depends only on a small subset of the 
entries of X. Taking T.grad(Y, X) is computationally expensive since it 
will compute every possible gradient so one would like to be able to 
compute, e.g. T.grad(Y, X[i]) . Here is some basic code illustrating the 
problem.


X = T.matrix()
Y = T.sum(X**2)

full_grad = T.grad(Y, X) # This works

X0 = X[0]
test = T.grad(Y, X0) # This pukes a Disconnected Input error

Silencing the Disconnected Input can be done in grad, but of course, 
that doesn't solve anything, evaluating the gradients only results in a 
bunch of 0s. So, is there a way of taking these gradients with respect 
to a subtensor?



--

---
You received this message because you are subscribed to the Google 
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to theano-users+unsubscr...@googlegroups.com 
<mailto:theano-users+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Pascal Lamblin

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] MILA and the future of Theano

[Pascal Lamblin]

Dear users and developers,

After almost ten years of development, we have the regret to announce 
that we will put an end to our Theano development after the 1.0 release, 
which is due in the next few weeks. We will continue minimal maintenance 
to keep it working for one year, but we will stop actively implementing 
new features. Theano will continue to be available afterwards, as per 
our engagement towards open source software, but MILA does not commit to 
spend time on maintenance or support after that time frame.


The software ecosystem supporting deep learning research has been 
evolving quickly, and has now reached a healthy state: open-source 
software is the norm; a variety of frameworks are available, satisfying 
needs spanning from exploring novel ideas to deploying them into 
production; and strong industrial players are backing different software 
stacks in a stimulating competition.


We are proud that most of the innovations Theano introduced across the 
years have now been adopted and perfected by other frameworks. Being 
able to express models as mathematical expressions, rewriting 
computation graphs for better performance and memory usage, transparent 
execution on GPU, higher-order automatic differentiation, for instance, 
have all become mainstream ideas.


In that context, we came to the conclusion that supporting Theano is no 
longer the best way we can enable the emergence and application of novel 
research ideas. Even with the increasing support of external 
contributions from industry and academia, maintaining an older code base 
and keeping up with competitors has come in the way of innovation.


MILA is still committed to supporting researchers and enabling the 
implementation and exploration of innovative (and sometimes wild) 
research ideas, and we will keep working towards this goal through other 
means, and making significant open source contributions to other projects.


Thanks to all of you who for helping develop Theano, and making it 
better by contributing bug reports, profiles, use cases, documentation, 
and support.


-- Yoshua Bengio,
Head of MILA

--

--- 
You received this message because you are subscribed to the Google Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: num_outputs equivalent for nnet.conv2d?

[Pascal Lamblin]

It would be the first item in "filter_shape".
It is optional, though, and can be read at run-time from the shape of 
"filters".

On Thursday, July 27, 2017 at 11:40:44 PM UTC-4, Ethan wrote:
>
> I'm porting some Tensorflow code to Theano, and the convolution layer 
> involves a parameter called num_outputs 
> . I 
> was wondering which parameter in theano's conv2d 
> class
>  
> controlled this. Thanks very much!
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: How to update theano-0.9.0 to theano-0.10.0?

[Pascal Lamblin]

The conda package you installed as a user takes precedence over the new one 
that you installed as root (with sudo).
I would advise that you uninstall the one as root (sudo pip uninstall 
theano), and then run the "pip install" without sudo.

On Wednesday, July 26, 2017 at 11:31:56 PM UTC-4, 김병수 wrote:
>
> Hello, my name is Kim, one of the heavy theano users.
>
> At first, my OS is ubuntu 16.04-2 LTS
>
> I recently installed theano through the command : 'conda install theano 
> pygpu', which install the theano-0.9.0.
>
> And updated the theano to bleeding edge version (theano-0.10.0) through 
> the commands : 
>
> 'sudo pip install --no-deps git+
> https://github.com/Theano/Theano.git#egg=Theano'
>
> 'sudo pip install --upgrade --no-deps git+
> https://github.com/Theano/Theano.git#egg=Theano'
>
> And it shows following messages, 
>
> 'Installing collected packages: Theano
>   Found existing installation: Theano 0.10.0.dev1
> Uninstalling Theano-0.10.0.dev1:
>   Successfully uninstalled Theano-0.10.0.dev1
>   Running setup.py install for Theano ... done
> Successfully installed Theano-0.10.0.dev1'
>
> But when I check the theano version, it shows that the theano version is 
> still theano-0.9.0.
>
> I suspect that the update is not reflected to the conda environment.
>
> How can I upgrade to the theano-0.10.0?
>
> Thanks, in advance. 
>
>
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Error in a simple scan code

[Pascal Lamblin]

Since the step function only returns one single variable, the first value 
returned by scan() is also one single variable. So it should be:
h, _ = theano.scan(step , sequences=[x])

If instead you use 
[h], _ = theano.scan(step , sequences=[x])
then it is trying to expand that variable into a list of one element, which 
it is not able to do since it does not know its length.

On Wednesday, July 26, 2017 at 8:56:03 AM UTC-4, Sunjeet Jena wrote:
>
>
> Any idea why this code is not working?
>
>
> import theano
> from theano import tensor
> from theano import shared
> from theano import function
> import numpy as np
>
> x=tensor.ivector('x')
>
> def step(u_t):
>
>
> h=u_t*1 
>
> return h
>
> [h], _ = theano.scan(step , sequences=[x])
>
> f=function([x],[h])
>
> b=[1,2,3,4,5,6]
> print(f(b))
>
> *Error is: *
>
> Traceback (most recent call last):
>   File "Test1.py", line 31, in 
> [h], _ = theano.scan(step , sequences=x)
>   File "/usr/local/lib/python2.7/dist-packages/theano/tensor/var.py", line 
> 596, in __iter__
> for i in xrange(theano.tensor.basic.get_vector_length(self)):
>   File "/usr/local/lib/python2.7/dist-packages/theano/tensor/basic.py", 
> line 4685, in get_vector_length
> raise ValueError("length not known: %s" % msg)
> ValueError: length not known: for{cpu,scan_fn} [id A] ''   
>  |Subtensor{int64} [id B] ''   
>  | |Shape [id C] ''   
>  | | |Subtensor{int64::} [id D] 'x[0:]'   
>  | |   |x [id E]
>  | |   |Constant{0} [id F]
>  | |Constant{0} [id G]
>  |Subtensor{:int64:} [id H] ''   
>  | |Subtensor{int64::} [id D] 'x[0:]'   
>  | |ScalarFromTensor [id I] ''   
>  |   |Subtensor{int64} [id B] ''   
>  |Subtensor{int64} [id B] ''   
>
> Inner graphs of the scan ops:
>
> for{cpu,scan_fn} [id A] ''   
>  >Elemwise{mul,no_inplace} [id J] ''   
>  > |x[t] [id K] -> [id H]
>  > |TensorConstant{1} [id L]
>
>
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: How to import the data of set of images into data-frame.

[Pascal Lamblin]

This does not seem to have anything to do with Theano.

On Sunday, July 23, 2017 at 10:32:08 AM UTC-4, Ali Alani wrote:
>
> I try to import the data of 3000 images to the data-frame each row 
> corresponds to one image. the problem with following code is only the last 
> image data store in the data-frame as shown below   
> 
>   
>
>
> from PIL import Image
> import glob
> image_list = []
> for filename in glob.glob('Images path\*.bmp'):
>img = PIL.Image.open(filename).convert("L")
>np1 = numpy.array(img)
>np_convert=numpy.invert(np1)
>flat_arr = np_convert.ravel()
>vector = np.matrix(flat_arr)
>image_list.append(vector)
>df_2 = pd.DataFrame(vector)
>
>
>
> 
>
>
> Please, any help to import all data into data-frame will be highly 
> appreciated
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: How to clip tensor value only in forward-pass, not in backward-pass?

[Pascal Lamblin]

You could implement a new op similar to clip, except the gradient would be 
1 everywhere.
An option to do that would be to use `OpFromGraph` and override the 
gradient.

On Saturday, July 22, 2017 at 7:42:06 PM UTC-4, Sihyeon Seong wrote:
>
>
> Hi, I found out T.clip(x, min, max) makes gradient zero when x is not in 
> [min, max]. 
>
> It would be useful if there is a function only clips value in the forward 
> pass, and calculate df/dx(x=max) at the backward-pass.
>
> Is the function I explained exist in Theano? Or, can somebody teach me how 
> to implement it? Thanks.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Scan checkpointing - what exactly theano stores?

[Pascal Lamblin]

I'm not sure I have all the details, but I think in principle, everything 
that is an output of the scan will be stored (if needed) for the backward 
pass.
For scan checkpointing, we basically hide explicit outputs inside another 
level of scan, which forces the recomputation during the backprop.
In order to do it the other way around, adding explicit outputs should make 
them available without recomputation.

On Friday, July 21, 2017 at 8:43:11 PM UTC-4, Alexander Botev wrote:
>
> So the scan checkpointing seems very ineteresting from the prespective 
> that it can be used for things like learning-to-learn.
> However, my question is can we tell Theano which part of each N-th 
> iteration it to store and which not? For instance in the learning-to-learn 
> framework where we unroll SGD
> the optimal would be to store only the "updated" parameters which get pass 
> to the next time step, rather than the whole computation. Is it possible to 
> achieve something like that? 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Implementing a GPU op

[Pascal Lamblin]

Indeed, there was an issue when using a later version of libgpuarray with 
the released version 0.9 of Theano.
This should be fixed when libgpuarray 0.6.9 is available.
See https://github.com/Theano/Theano/issues/6141

On Sunday, July 16, 2017 at 7:15:51 AM UTC-4, Christopher Bourez wrote:
>
> Moreover if you install Theano from scratch :
>
> conda install theano pygpu
>
> Fetching package metadata .
> Solving package specifications: .
>
> Package plan for installation in environment /root/miniconda2:
>
> The following NEW packages will be INSTALLED:
>
> libgpuarray: 0.6.8-0 
> pygpu:   0.6.8-py27_0
> theano:  0.9.0-py27_0
>
>
> you need to run the following correction after :
>
> conda install libgpuarray=0.6.4-0 pygpu=0.6.4
>
>
> And now that works on the server I was initially working on. 
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] NotImplementedError: In() instances and tuple inputs trigger the old semantics...

[Pascal Lamblin]

Indeed, this is the reason. "inputs" has to be a flat list of Theano 
Variables.

On Wednesday, July 19, 2017 at 10:36:47 AM UTC-4, Francesco Ciompi wrote:
>
> Hi,
>
> I found this message while looking for a solution to this same error 
> message.
> I am using theano 0.9 and python 3.5.
>
> The error I get is:
>
> "In() instances and tuple inputs trigger the old "
> NotImplementedError: In() instances and tuple inputs trigger the old 
> semantics, which disallow using updates and givens
>
> when trying to compile:
>
> self.train_fn = theano.function([input_var, target_var], [loss, out], 
> updates=updates)
>
> where input_var is actually a Python list of 3 tensor variables of the 
> same shape.
>
> Could this be the reason for this error? Are lists supported as type of 
> input variables?
>
> Thanks,
> Francesco
>
>
>
> On Friday, February 24, 2017 at 9:43:17 PM UTC+1, nouiz wrote:
>>
>> What is the running code? Did you update theano as the last message 
>> suggest? What is your Theano version?
>>
>> Make sure to use Theano 0.9beta1 or 0.9rc1. If the code come from the 
>> Deep learning tutorial, update it. Old combination of DLT and Theano could 
>> give this.
>>
>> Current Theano version don't raise this error anymore.
>>
>> Fred
>>
>> On Mon, Jun 9, 2014 at 5:10 PM Tariq Daouda  wrote:
>>
>>> Hello,
>>>
>>> Does anyone know what this error means?
>>>
>>> Traceback (most recent call last):
>>>   File "mlp.py", line 115, in 
>>> *nn.train([train_set_x[i]], [train_set_y[i]])*
>>>   File "mlp.py", line 75, in train
>>> *return theano.function(inputs = [x, y], outputs = self.cost, 
>>> updates = self.updates)*
>>>   File 
>>> "/u/daoudat/usr/lib/python2.6/site-packages/theano/compile/function.py", 
>>> line 205, in function
>>> "In() instances and tuple inputs trigger the old "
>>> *NotImplementedError: In() instances and tuple inputs trigger the old 
>>> semantics, which disallow using updates and givens*
>>>
>>> Concerning the inputs, train_set_x[i] is a numpy array and 
>>> train_set_y[i] is an integer.
>>>
>>> Cheers,
>>>
>>> Tariq Daouda
>>>
>>> -- 
>>>
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "theano-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to theano-users...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: WARNING (theano.gof.cmodule): OPTIMIZATION WARNING: Theano was not able to find the default g++ parameters. This is needed to tune the compilation to your specific CPU. This can slo

[Pascal Lamblin]

Which version of Theano are you using?
I think this error could happen before the 0.9 release.
I would encourage you to update to the latest development version.

On Tuesday, July 18, 2017 at 7:31:56 AM UTC-4, mai Khai wrote:
>
> Dear group,
> I ran into this problem when i ran a program using theano:
> "WARNING (theano.gof.cmodule): OPTIMIZATION WARNING: Theano was not able 
> to find the default g++ parameters. This is needed to tune the compilation 
> to your specific CPU. This can slow down the execution of Theano functions. 
> Please submit the following lines to Theano's mailing list so that we can 
> fix this problem:"
> Here is the content to submit: 
>
>  ['# 1 ""\n', '# 1 ""\n', '# 1 ""\n', '# 1 
> "/usr/include/stdc-predef.h" 1 3 4\n', '# 1 "" 2\n', '# 1 
> ""\n', 'Using built-in specs.\n', 'COLLECT_GCC=/usr/bin/g++\n', 
> 'Target: aarch64-linux-gnu\n', "Configured with: ../src/configure -v 
> --with-pkgversion='Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4' 
> --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs 
> --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr 
> --program-suffix=-5 --enable-shared --enable-linker-build-id 
> --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix 
> --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu 
> --enable-libstdcxx-debug --enable-libstdcxx-time=yes 
> --with-default-libstdcxx-abi=new --enable-gnu-unique-object 
> --disable-libquadmath --enable-plugin --with-system-zlib 
> --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo 
> --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-arm64/jre --enable-java-home 
> --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-arm64 
> --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-arm64 
> --with-arch-directory=aarch64 
> --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-multiarch 
> --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release 
> --build=aarch64-linux-gnu --host=aarch64-linux-gnu 
> --target=aarch64-linux-gnu\n", 'Thread model: posix\n', 'gcc version 5.4.0 
> 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4) \n', 
> "COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mlittle-endian' 
> '-mabi=lp64'\n", ' /usr/lib/gcc/aarch64-linux-gnu/5/cc1 -E -quiet -v 
> -imultiarch aarch64-linux-gnu - -mlittle-endian -mabi=lp64 
> -fstack-protector-strong -Wformat -Wformat-security\n', 'ignoring 
> nonexistent directory "/usr/local/include/aarch64-linux-gnu"\n', 'ignoring 
> nonexistent directory 
> "/usr/lib/gcc/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/include"\n', 
> '#include "..." search starts here:\n', '#include <...> search starts 
> here:\n', ' /usr/lib/gcc/aarch64-linux-gnu/5/include\n', ' 
> /usr/local/include\n', ' /usr/lib/gcc/aarch64-linux-gnu/5/include-fixed\n', 
> ' /usr/include/aarch64-linux-gnu\n', ' /usr/include\n', 'End of search 
> list.\n', 
> 'COMPILER_PATH=/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/:/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/\n',
>  
> 'LIBRARY_PATH=/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/:/usr/lib/gcc/aarch64-linux-gnu/5/../../../../lib/:/lib/aarch64-linux-gnu/:/lib/../lib/:/usr/lib/aarch64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/aarch64-linux-gnu/5/../../../:/lib/:/usr/lib/\n',
>  
> "COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mlittle-endian' 
> '-mabi=lp64'\n"]
> error at line: 4592
>
> Best regard,
> Khai 
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] dot product of sparse and dense matrix

[Pascal Lamblin]

You just have to declare a symbolic input variable representing the sparse 
matrix, use it in the computation graph, compile a theano.function, and 
then pass the value when calling it.
See the examples at 
http://deeplearning.net/software/theano/library/sparse/index.html#csr-matrix 
in particular.

On Monday, July 17, 2017 at 11:55:50 AM UTC-4, Lalit Pradhan wrote:
>
> Hi, 
>
> I have:
> A scipy sparse matrix 'csr' with shape (262144, 262144)  and 6522916 non 
> zero rows
> and 
> A theano tensor variable, a flattened (512*512) = 262144 output of a 
> feature map.
>
> I want to do a dot product in theano between them. How can i convert the 
> scipy sparse matrix into a theano sparse matrix? How do I go about this 
> task?
>
> Thanks,
> Lalit
>
>
>
> On Tuesday, March 5, 2013 at 2:21:43 AM UTC+5:30, Frédéric Bastien wrote:
>>
>> I split the PR, the new PR is: https://github.com/Theano/Theano/pull/1264 
>>
>> I updated your name. 
>>
>> thanks 
>>
>> Fred 
>>
>> On Mon, Mar 4, 2013 at 3:08 PM, bizso09 <biz...@gmail.com> wrote: 
>> > thank you for the credit. If we are going formal, you can include my 
>> real 
>> > name which is Zsolt Bitvai, no less peculiar 
>> > 
>> > 
>> > On Mon, Mar 4, 2013 at 6:43 PM, Frédéric Bastien <no...@nouiz.org> 
>> wrote: 
>> >> 
>> >> The fix are in PR: https://github.com/Theano/Theano/pull/1263 
>> >> 
>> >> thanks for the report. 
>> >> 
>> >> Fred 
>> >> 
>> >> On Mon, Mar 4, 2013 at 1:40 PM, Frédéric Bastien <no...@nouiz.org> 
>> wrote: 
>> >> > I have a fix in preparation. 
>> >> > 
>> >> > The problem is sparse.Dot.make_node. It was always making a matrix 
>> >> > output even if the inputs aren't matrices. 
>> >> > 
>> >> > Fred 
>> >> > 
>> >> > On Mon, Mar 4, 2013 at 1:35 PM, Pascal Lamblin 
>> >> > <lamb...@iro.umontreal.ca> wrote: 
>> >> >> On Mon, Mar 04, 2013, bizso09 wrote: 
>> >> >>> theano.printing.debugprint(r) 
>> >> >>> 
>> >> >>> SparseDot [@A] '' 
>> >> >>>  |SparseVariable{csc,float64} [@B] 
>> >> >>>  |Subtensor{::, 3} [@C] '' 
>> >> >>>|W [@D] 
>> >> >> 
>> >> >> I'm guessing the problem is that SparseDot.infer_shape returns 
>> >> >> something 
>> >> >> of length 1, because one of the inputs is a vector, but it should 
>> be of 
>> >> >> length 2, because the output is a sparse matrix, as there are no 
>> sparse 
>> >> >> vectors. 
>> >> >> 
>> >> >>> I'm running '0.6.0rc3' When I try to upgrade I get an error 
>> though. 
>> >> >>> 
>> >> >>> sudo pip install --upgrade theano 
>> >> >>> Command python setup.py egg_info failed with error code 1 in 
>> >> >>> /home/user/build/scipy 
>> >> >> 
>> >> >> 0.6.0rc3 is the last release candidate at the moment, so it's OK. 
>> >> >> In the future, you should run "sudo pip install --upgrade --no-deps 
>> >> >> theano". 
>> >> >> The "--no-deps" bit ensures you upgrade only Theano, not scipy and 
>> >> >> numpy. 
>> >> >> 
>> >> >> -- 
>> >> >> Pascal 
>> >> >> 
>> >> >> -- 
>> >> >> 
>> >> >> --- 
>> >> >> You received this message because you are subscribed to the Google 
>> >> >> Groups "theano-users" group. 
>> >> >> To unsubscribe from this group and stop receiving emails from it, 
>> send 
>> >> >> an email to theano-users...@googlegroups.com. 
>> >> >> For more options, visit https://groups.google.com/groups/opt_out. 
>> >> >> 
>> >> >> 
>> >> 
>> >> -- 
>> >> 
>> >> --- 
>> >> You received this message because you are subscribed to a topic in the 
>> >> Google Groups "theano-users" group. 
>> >> To unsubscribe from this topic, visit 
>> >> 
>> https://groups.google.com/d/topic/theano-users/_rEHTO06utQ/unsubscribe?hl=en.
>>  
>>
>> >> To unsubscribe from this group and all its topics, send an email to 
>> >> theano-users...@googlegroups.com. 
>> >> 
>> >> For more options, visit https://groups.google.com/groups/opt_out. 
>> >> 
>> >> 
>> > 
>> > -- 
>> > 
>> > --- 
>> > You received this message because you are subscribed to the Google 
>> Groups 
>> > "theano-users" group. 
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> an 
>> > email to theano-users...@googlegroups.com. 
>> > For more options, visit https://groups.google.com/groups/opt_out. 
>> > 
>> > 
>>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Non-machine learning applications for Theano?

[Pascal Lamblin]

I would say PyMC3 https://github.com/pymc-devs/pymc3, which uses Theano for 
probabilistic programming.

On Thursday, July 13, 2017 at 9:06:20 PM UTC-4, Prashant Raina wrote:
>
> Does anyone have any ideas for using Theano for other things besides 
> machine learning?
> Do such projects already exist?
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Implementing a GPU op

[Pascal Lamblin]

On Wednesday, July 12, 2017 at 4:13:43 AM UTC-4, Christopher Bourez wrote: 
>
>  I don't know what you mean by "not modifying" the source for GpuEye:
> - In this example, I'm importing a not modifyed GpuEye  op from Theano 
> basic ops
> - If I'm using theano.tensor.eye, then it does not use the GpuEye
>

OK, I assumed that you had started from the implementation of GpuEye to 
implement a new GPU Op.
Your original example seems to work for me, though, so it may have to do 
with your setup:

In [3]: import theano
   ...: from theano.gpuarray.basic_ops import GpuEye
   ...: 
   ...: x = theano.tensor.iscalar('x')
   ...: y = theano.tensor.iscalar('y')
   ...: z = GpuEye(dtype='float32', context_name=None)(x,y, theano.tensor.
constant(0))
   ...: 
   ...: theano.printing.debugprint(z)
   ...: print("Compiling")
   ...: f = theano.function( [x,y], z)
   ...: theano.printing.debugprint(f)
   ...: print("Results")
   ...: print(f(3, 3))
   ...: 
GpuEye{dtype='float32', context_name=None} [id A] ''   
 |x [id B]
 |y [id C]
 |TensorConstant{0} [id D]
Compiling
GpuEye{dtype='float32', context_name=None} [id A] ''   0
 |x [id B]
 |y [id C]
 |TensorConstant{0} [id D]
Results
[[ 1.  0.  0.]
 [ 0.  1.  0.]
 [ 0.  0.  1.]]

Also, are you sure this test
>
> https://github.com/Theano/Theano/blob/2625464534147fd70da60a3a3ddcb63ed8e5a416/theano/gpuarray/tests/test_basic_ops.py#L401
> works well ? 
>

Yes, it gets tested in our daily buildbot and on several pull requests per 
week, by our continuous integration systems. I also just launched it 
manually:
$ theano-nose theano/gpuarray/tests/test_basic_ops.py:test_gpueye
Can not use cuDNN on context None: Disabled by dnn.enabled flag
Mapped name None to device cuda: GeForce GTX 580 (:02:00.0)
.
--
Ran 45 tests in 21.645s

OK


I've also tried to create an example with theano.gpuarray.nnet.GpuSoftmax but 
> after compilation it got replaced another implementation*GpuDnnSoftmax : *


Yes, there is an optimization that does that if cuDNN is available. You 
should be able to disable it with `optimizer_excluding=local_softmax_dnn`.

A second thing that is not clear to me in the documentation of Theano is 
> how you specify a C implementation and GPU implementation of the same own 
> op. Thank you
>

You do not specify C and GPU implementations for the same Op, what we have 
in general is two different Ops, one that has CPU inputs and outputs, and 
computes on CPU, and another one with GPU inputs and outputs, that computes 
on GPU.
This is necessary because the Variables in Theano are strongly typed, and 
the device is part of the type.
There are optimizations that replace CPU Ops by GPU ones, inserting 
transfer Ops (GpuFromHost, HostFromGpu) if necessary.
GPU Ops, like CPU ones, can have C (using CUDA) or Python implementations 
(or both). 

What surprises me is to get seg faults in the theano function, while I 
> would have expected them to occur during evaluation on values...
>

It is strange indeed. It may be possible that some GPU operations are 
executed on GPU during the compilation phase, for constant folding 
(constant propagation) for instance.
Does it happen as well with the latest master from GitHub?
 

>  
>
On Wednesday, July 12, 2017 at 10:05:30 AM UTC+2, Christopher Bourez wrote:
>>
>>
>>
>> On Wednesday, July 12, 2017 at 9:58:34 AM UTC+2, Christopher Bourez wrote:
>>>
>>>
>>>
>>> *Elemwise{mul,no_inplace} [id A] ''|HostFromGpu(gpuarray) [id B] '' 
>>>| |GpuSoftmax [id C] ''|   |GpuFromHost [id D] ''| |x 
>>> [id E] |InplaceDimShuffle{x,x} [id F] ''  |TensorConstant{2} [id 
>>> G]CompilingHostFromGpu(gpuarray) [id A] ''   5 |GpuElemwise{Mul}[(0, 
>>> 1)] [id B] ''   4   |GpuArrayConstant{[[ 2.]]} [id C]  
>>>  |InplaceGpuDimShuffle{0,1} [id D] ''   3
>>>  |GpuDnnSoftmax{mode='channel', algo='accurate'} [id E] ''   2  
>>>  |GpuContiguous [id F] ''   1 |InplaceGpuDimShuffle{0,1,x,x} [id G] 
>>> ''   0   |<GpuArrayType(float32, (False, False))> [id H]*I'm 
>>> looking of a good example with a GPU Kernel.
>>>
>>> On Wednesday, July 12, 2017 at 9:56:08 AM UTC+2, Christopher Bourez 
>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wednesday, July 12, 2017 at 2:48:44 AM UTC+2, Pascal Lamblin wrote:
>>>>>
>>>>> Does it work if you do not modify the source for GpuEye at all?
>>>>> If it does, then maybe sharing your new source would get you more help.
>>>>>
>>>>> On Tuesday, July 11, 2017 at 12:12:03 PM UTC-4, Chr

[theano-users] Re: Implementing a GPU op

[Pascal Lamblin]

Does it work if you do not modify the source for GpuEye at all?
If it does, then maybe sharing your new source would get you more help.

On Tuesday, July 11, 2017 at 12:12:03 PM UTC-4, Christopher Bourez wrote:
>
> Hi, 
>
> I'm trying to implement a simple GPU op but it always gives me a 
> Segmentation fault during compilation, without other message.
>
> For example :
> import theano
> from theano.gpuarray.basic_ops import GpuEye
>
> x = theano.tensor.iscalar('x')
> y = theano.tensor.iscalar('y')
> z = GpuEye(dtype='float32', context_name=None)(x,y, 
> theano.tensor.constant(0))
>
> theano.printing.debugprint(z)
> print("Compiling")
> f = theano.function( [x,y], z)
> theano.printing.debugprint(f)
> print("Results")
> print(f(3, 3))
>
> I've also tried with the softmax gpu function. Is there something I'm 
> missing ?
>
> I copied the file, created a complete new op, and the segmentation fault 
> appears when I'm defining a Kernel in gpu_kernels() method of the op.
>
> Thank you a lot for your help
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Unused input error with chained OpFromGraph ops

[Pascal Lamblin]

That error actually comes from the function compilation that happens inside 
op2.
One option is to add "inline=True" to the call to OpFromGraph returning 
op2, in that case the gradient gets overriden, but you will have the 
inlined subgraph in the global one, and OpFromGraph will not build an inner 
function.
The other one would be to add "on_unused_input='ignore'" to the call to 
OpFromGraph:

op2 = theano.OpFromGraph(
inputs=[z_var, m_var, r_var],
outputs=[z_var],
grad_overrides=grad_op2,
on_unused_input='ignore')

With it, the program worked, but I still got an error in g.eval(...):
TypeError: Cannot convert Type DisconnectedType (of Variable 
) into Type TensorType(float64, scalar). You can try to 
manually convert  into a TensorType(float64, scalar).
I'm not sure where that comes from, please let us know if you do not figure 
it out.

On Tuesday, July 11, 2017 at 5:32:50 AM UTC-4, nicolas.grange...@gmail.com 
wrote:
>
> Hi,
>
> I am trying to split an computation over two ops in order to avoid 
> spurious computations when computing the gradient.
> My current attempt uses a first op which returns the desired result for 
> the forward part and extra intermediate results. The second op just 
> forwards the desired result, but its grad is overriden to compute the 
> gradient based on the intermediate results.
>
> In this configuration, Theano complains about unused inputs in the forward 
> computation because the intermediate results are not used for the forward 
> method of the second op.
>
> Is this an expected behaviour or a bug?
>
> 
>
> import numpy as np
> import theano.tensor as T
> import theano
>
>
> def make_ops():
> x_var = T.vector()
> m_var = T.bvector()
>
> r = m_var.sum().astype('floatX')
> z = x_var * m_var / r
>
>
> def grad_op1(inputs, output_gradients):
> return [
> output_gradients[0],  # computation delegated to op2
> theano.gradient.DisconnectedType()()
> ]
>
>
> op1 = theano.OpFromGraph(
> inputs=[x_var, m_var],
> outputs=[z, r],
> grad_overrides=grad_op1)
>
>
> z_var = T.vector()
> r_var = T.scalar()
>
> def grad_op2(inputs, output_gradients):
> _, m_, r_ = inputs
> return [
> m_ * r_,
> theano.gradient.DisconnectedType()(),
> theano.gradient.DisconnectedType()()
> ]
>
> op2 = theano.OpFromGraph(
> inputs=[z_var, m_var, r_var],
> outputs=[z_var],
> grad_overrides=grad_op2)
>
> return op1, op2
>
>
> op1, op2 = make_ops()
> x_var = T.vector()
> m_var = T.bvector()
> z_, r = op1(x_var, m_var)
> z = op2(z_, m_var, r)
>
> print(z_.eval({x_var: np.array([1., .3, .0, .2], dtype=np.float32),
>m_var: np.array([1, 0, 1, 1], dtype=np.int8)}))
>
> f = theano.function([x_var, m_var], [z], on_unused_input='ignore')  # 
> raises anyway
>
> print(f(np.array([1., .3, .0, .2], dtype=np.float32),
>   np.array([1, 0, 1, 1], dtype=np.int8)))
>
> # g = theano.grad(T.sum(z), wrt=x_var)
> # print(g.eval({x_var: np.array([1., .3, .0, .2], dtype=np.float32),
> #   m_var: np.array([1, 0, 1, 1], dtype=np.int8)}))
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: Significant increase in GPU memory consumption with new GPU backend

[Pascal Lamblin]

On Monday, July 10, 2017 at 2:42:39 AM UTC-4, Fabian Stemmer wrote:
>
> Thanks, by setting gpuarray.preallocate=-1 I now get similar behavior for 
> the new backend as for the old.
>
> Do I understand correctly, that leaving preallocate at default behavior 
> (new backend) will not result in higher memory consumption, but merely 
> doesn't free memory once allocated, so what I see in nvidia-smi is 
> max-memory consumption up to this point?
>

Not really, it can actually result in higher memory consumption due to the 
way new memory blocks are allocated. For instance, in the worse case, if a 
tensor of 1 MB gets allocated and deallocated, then a 2 MB tensor, a new 2 
MB block will be added to the pool, however it will not be mergeable with 
the first one, and if it gets freed, a 3 MB tensor cannot be "split" 
between the first blocks. Due to that fragmentation effect, allocating / 
deallocating 1 MB, then 2 MB, 3 MB, etc., will end up using 1 + 2 + 3 + ... 
MB total on the GPU. 
 

> A related question: When I run with profile=True,profile_memory=True - 
> shouldn't the max GPU memory stat in the profiling correspond to what I see 
> in nvidia-smi when I run with preallocate on default?
>

Again, not really, due to that fragmentation effect.
 

> Currently, I see ~400MB GPU memory usage in profiling and that's what I 
> see with preallocate=-1 too (although I can't guarantuee there aren't 
> higher spikes that I don't see with nvidia-smi). When I leave preallocate 
> at default, I see GPU memory usage ~2GB (but the profiling still reports 
> only 400MB).
>

Preallocating 400 or 500 MB may avoid fragmentation and bring the total 
consumption peak closer to what is actually allocated to arrays.
 

>
> Thanks
> Fabian
>
> On Thursday, June 22, 2017 at 3:45:07 PM UTC+2, nouiz wrote:
>>
>> The equivalent to the old back-end setting for memory is: 
>> gpuarray.preallocate=-1.
>>
>> The new back-end by default will cache all call to cudaMalloc() to speed 
>> up computation. This flag will disable this cache. THis is the same default 
>> as the old back-end.
>>
>> On Thu, Jun 22, 2017 at 9:41 AM Fabian Stemmer  
>> wrote:
>>
>>> When I did use preallocation I used lib.cnmem=1 for theano 0.8.2 and 
>>> gpuarray.preallocate=1 for theano 0.9.0 and 0.10.dev.
>>> For most experiments (including those in the log files) I did not use 
>>> preallocation, because the only way I could see the difference in memory 
>>> usage was through nvidia-smi, which only shows the static pre-allocation 
>>> when it is used.
>>> I believe the problem does not disappear with pre-allocation, since I 
>>> see my training crash for much smaller models with the new backend even 
>>> then. However, I cannot measure the effect of switching backends on GPU 
>>> memory when I use preallocation.
>>>
>>>
>>> On Thursday, June 22, 2017 at 3:23:15 PM UTC+2, nouiz wrote:
>>>
 Do you use the Theano flag: gpuarray.preallocate=1? When you tried the 
 preallocation, how did you use it?

 Is is mostly equivalent to lib.cnmem. But our default is different and 
 by default give more speed up, but sometimes can cause memory 
 fragmentation. the flag above fix the new fragmentation that can happen by 
 default.

 On Thu, Jun 22, 2017 at 5:33 AM Fabian Stemmer  
 wrote:

>>> One addition:
> The theano 0.9.0 setup used libgpuarray v0.6.2.
> The theano 0.10.dev setup used libgpuarray v0.6.5 - I just updated to 
> v0.6.7 and tested again, but I still get ~2GB memory usage.
>
>
> On Thursday, June 22, 2017 at 8:38:26 AM UTC+2, Fabian Stemmer wrote:
>>
>> Hi,
>>
>> I recently tried to switch my CNN implementation to the new theano 
>> GPU backend. To do so, I switched from "device=gpu" to "device=cuda" 
>> with 
>> theano9 and libgpuarray installed. My theano code then works with the 
>> new 
>> backend without any further changes.
>>
>> However, when I do this, I see my GPU memory consumption increase 
>> drastically. When I use theano memory profiling both GPU backends show 
>> the 
>> same memory consumption, but when I use nvidia-smi to monitor memory 
>> usage 
>> while the job is running, the old backend hovers somewhere around 400MB, 
>> while the new backend uses 2GB for the same model size and data. When I 
>> try 
>> to train larger models, the new GPU backend fails with memory errors for 
>> much smaller models than the old backend. This is also true when I 
>> activate 
>> memory pre-allocation.
>>
>> I tried to remove parts of my model or exclude certain theano 
>> optimizations (e.g. exclude conv_dnn to force theano to use a different 
>> convolution algorithm) but nothing I changed in the model structure had 
>> an 
>> impact on the discrepancy I see in memory usage.
>>
>> I use CUDA 8.0 and cuDNN 5105 for these experiments. 

Re: [theano-users] theano0.9 and cuda-8

[Pascal Lamblin]

How did you determine it is using the CPU?

On Monday, July 3, 2017 at 10:20:40 AM UTC-4, ngu...@interactions.com wrote:
>
> Changing it to cuda* results is CPU usage and not GPU
>
> On Friday, June 30, 2017 at 4:20:22 PM UTC-4, nouiz wrote:
>>
>> You should not mix cuda version...
>>
>> Do you still use the old gpu back-end (device=gpu*) or the new back-end 
>> (device=cuda*)?
>>
>> Fred
>>
>> On Fri, Jun 30, 2017 at 9:57 AM  wrote:
>>
>>> I trying to understand some unexplained behavior of my code.
>>> To be sure that the problem is with my code and not with software 
>>> incompatibility I would like to sure  about the correctness of my setup
>>> I have:
>>> theano version 0.9
>>>
>>> CUDA_ROOT =/usr/local/cuda-7.5
>>> LD_PATH=/usr/local/cuda-8/lib64:...
>>> PATH=/usr/local/cuda-8/bin: ...
>>>
>>> Essentially I am using some parts of cuda-8 and some of cuda-7.5.
>>>
>>> With CUDA_ROOT =/usr/local/cuda-8, I cannot compile the theano functions.
>>>
>>> Thanks
>>>
>>>
>>>
>>>
>>> ***
>>>
>>> This e-mail and any of its attachments may contain Interactions 
>>> Corporation proprietary information, which is privileged, confidential, or 
>>> subject to copyright belonging to the Interactions Corporation. This e-mail 
>>> is intended solely for the use of the individual or entity to which it is 
>>> addressed. If you are not the intended recipient of this e-mail, you are 
>>> hereby notified that any dissemination, distribution, copying, or action 
>>> taken in relation to the contents of and attachments to this e-mail is 
>>> strictly prohibited and may be unlawful. If you have received this e-mail 
>>> in error, please notify the sender immediately and permanently delete the 
>>> original and any copy of this e-mail and any printout. Thank You.  
>>>
>>>
>>> ***
>>>   
>>>
>>>
>>> -- 
>>>
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "theano-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to theano-users...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Getting "pygpu.gpuarray.GpuArrayException: Out of memory" for a small application

[Pascal Lamblin]

What happens if you set gpuarray.preallocate to something much smaller, or 
even to -1?

Also, I see the script uses multiprocessing. Weird things happen if new 
Python processes are spawned after the GPU has been initialized. This is a 
limitation of how cuda handles GPU contexts I believe.
The solution would be not to use `device=cuda`, but `device=cpu`, and call 
`theano.gpuarray.use('cuda')` manually in the subprocess, or after all 
processes have been launched.

On Sunday, July 2, 2017 at 3:59:31 PM UTC-4, Daniel Seita wrote:
>
> I am attempting to run some reinforcement learning code on the GPU. (The 
> code is https://github.com/openai/imitation if it matters, running 
> `scripts/run_rl_mj.py`.)
>
> I converted the code to run on float32 by changing the way the data is 
> supplied via numpy. Unfortunately, with the new GPU backend, I am gettting 
> an out of memory error, despite having 12GB of memory on my Titan X Pascal 
> GPU. Here are my settings:
>
> $ cat ~/.theanorc 
> [global] 
> device = cuda 
> floatX = float32 
>
> [gpuarray] 
> preallocate = 1 
>
> [cuda] 
> root = /usr/local/cuda-8.0
>
>
> Theano seems to be importing correctly:
>
> $ ipython
> Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) 
>  
> Type "copyright", "credits" or "license" for more information. 
> IPython 5.3.0 -- An enhanced Interactive Python. 
> ? -> Introduction and overview of IPython's features. 
> %quickref -> Quick reference. 
> help  -> Python's own help system. 
> object?   -> Details about 'object', use 'object??' for extra details. 
>
> In [1]: import theano 
> Using cuDNN version 5105 on context None 
> Preallocating 11576/12186 Mb (0.95) on cuda 
> Mapped name None to device cuda: TITAN X (Pascal) (:01:00.0) 
>
> In [2]: 
>
>
>
> Unfortunately, running `python scripts/run_rl_mj.py --env_name 
> CartPole-v0 --log trpo_logs/CartPole-v0` on the very low-dimensional 
> CartPole setting (state space is just four numbers, actions are just one 
> number) gives me (after a bit of a setup):
>
>
> Traceback (most recent call last):
>
>   File "scripts/run_rl_mj.py", line 116, in 
>
> main()
>
>   File "scripts/run_rl_mj.py", line 109, in main
>
> iter_info = opt.step()
>
>   File "/home/daniel/imitation_noise/policyopt/rl.py", line 280, in step
>
> cfg=self.sim_cfg)
>
>   File "/home/daniel/imitation_noise/policyopt/__init__.py", line 411, in 
> sim_mp
>
> traj = job.get()
>
>   File "/home/daniel/anaconda2/lib/python2.7/multiprocessing/pool.py", 
> line 567, in get
>
> raise self._value
>
> pygpu.gpuarray.GpuArrayException: Out of memory
>
> Apply node that caused the error: GpuFromHost(obsfeat_B_Df)
>
> Toposort index: 4
>
> Inputs types: [TensorType(float32, matrix)]
>
> Inputs shapes: [(1, 4)]
>
> Inputs strides: [(16, 4)]
>
> Inputs values: [array([[ 0.04058,  0.00428,  0.03311, -0.02898]], 
> dtype=float32)]
>
> Outputs clients: [[GpuElemwise{Composite{((i0 - i1) / 
> i2)}}[](GpuFromHost.0, 
> /GibbsPolicy/obsnorm/Standardizer/mean_1_D, GpuElemwise{Composite{(i0 + 
> sqrt((i1 * (Composite{(i0 - sqr(i1))}(i2, i3) + Abs(Composite{(i0 - 
> sqr(i1))}(i2, i3))}}[].0)]]
>
>
> HINT: Re-running with most Theano optimization disabled could give you a 
> back-trace of when this node was created. This can be done with by setting 
> the Theano flag 'optimizer=fast_compile'. If that does not work, Theano 
> optimizations can be disabled with 'optimizer=None'.
>
> HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and 
> storage map footprint of this apply node.
>
> Closing remaining open files:trpo_logs/CartPole-v0...done
>
>
> What I'm confused about is that
>
>- This happens right at the beginning of the reinforcement learning, 
>so it's not as if the algorithm has been running a long time and then ran 
>out of memory.
>- The input shapes are quite small, (1,4) and (16,4). In addition, the 
>output is supposed to do normalization and several other element-wise 
>operations. None of this suggests high memory usage.
>
> I tried `optimizer = fast_compile` and re-ran this, but the error message 
> was actually less informative (it contains a subset of the above error 
> message). Running with `exception_verbosity = high` results in a different 
> error message:
>
>
> Max traj len: 200
>
> Traceback (most recent call last):
>
>   File "scripts/run_rl_mj.py", line 116, in 
>
> main()
>
>   File "scripts/run_rl_mj.py", line 109, in main
>
> iter_info = opt.step()
>
>   File "/home/daniel/imitation_noise/policyopt/rl.py", line 280, in step
>
> cfg=self.sim_cfg)
>
>   File "/home/daniel/imitation_noise/policyopt/__init__.py", line 411, in 
> sim_mp
>
> traj = job.get()
>
>   File "/home/daniel/anaconda2/lib/python2.7/multiprocessing/pool.py", 
> line 567, in get
>
> raise self._value
>
> pygpu.gpuarray.GpuArrayException: initialization error
>
> Closing remaining open 

[theano-users] Re: cudnn detected in cuda backend but not in gpuarray backend

[Pascal Lamblin]

The test actually ran on GPU, as evidenced by it printing "GpuElemwise".
The issue is you are using a really old version of "gputest", which does 
not correctly detect the new back-end. Please use the latest version at 
http://deeplearning.net/software/theano/tutorial/using_gpu.html#testing-the-gpu

On Sunday, July 2, 2017 at 8:58:41 AM UTC-4, Akshay Chaturvedi wrote:
>
> I was able to solve the issue by setting CPLUS_INCLUDE_PATH. Now the 
> output looks like this
> Using cuDNN version 5110 on context None
> Mapped name None to device cuda0: GeForce GTX 960 (:01:00.0)
> [GpuElemwise{exp,no_inplace}(), 
> HostFromGpu(gpuarray)(GpuElemwise{exp,no_inplace}.0)]
> Looping 1000 times took 0.196515 seconds
> Result is [ 1.23178029 1.61879349 1.52278066 ..., 2.20771813 2.29967761
> 1.62323296]
> Used the cpu
>
> The file is unable to detect that the program ran on gpu but it's a 
> seperate issue. I am attaching the file gputest.py. 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Always Segmentation fault(core dumped) when use theano(NOT when import)

[Pascal Lamblin]

For pylearn2, you can change the imports to use `six` directly, rather than 
`theano.six`.

On Sunday, July 2, 2017 at 1:53:35 AM UTC-4, noodles wrote:
>
> It works. When I install theano 1.0, it don't report segmentation fault. 
> But the problem is that I install theano to use lasagne and pylearn2, which 
> has't support theano1.0 (it report"ImportError: No module named 
> six.moves”）, do you know how to resolve this?
>
> Thank you very much.
>
> 在 2017年6月30日星期五 UTC+8下午8:36:49，nouiz写道：
>>
>> Install the dev version of Theano. It contains segmentation fault fixes.
>>
>> If that don't work, tell us, but I think it should work.
>>
>> Le ven. 30 juin 2017 06:00, noodles  a écrit :
>>
>>> Hello,
>>>
>>> I encounter a strange problem when using theano. These days I 
>>> bought 
>>> a new computer and install theano on it, and I can even import it in
>>> python with no error, but everytime I create a function, it corrupted 
>>> with "Segmentation fault(core dumped)". Below is the detail:
>>> I have installed theano on another two old machine, and they 
>>> works well. This new machine is : CPU: intel 7700; GPU  2xGTX1080Ti, 
>>> OS: ubuntu16.04.  CUDA 8.0, cudnn 5.1 .I use miniconda2 to install 
>>> theano( conda install theano), python 2.7, theano 0.9.0
>>>
>>>   when I import theano in python, the output is:
>>>
 *nice@fat01:~$ python*
 *Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016,
 23:09:15) *
 *[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2*
 *Type "help", "copyright", "credits" or "license" for more information.*
 *Anaconda is brought to you by Continuum Analytics.*
 *Please check out: http://continuum.io/thanks <
 http://continuum.io/thanks>
 and https://anaconda.org *
 *>>> import theano*
 *Using cuDNN version 5110 on context None*
 *Mapped name None to device cuda1: GeForce GTX 1080 Ti (:02:00.0)*
 *>>> *
>>>
>>>  
>>> then I input the code from the exercise of 
>>> http://deeplearning.net/software/theano/tutorial/using_gpu.html#gpuarray
>>>  
>>>
>>> 
>>>
>>> *import numpy*
>>> *import theano*
>>> *import theano.tensor as T*
>>> *rng = numpy.random*
>>> *N = 400*
>>> *feats = 784*
>>> *D = (rng.randn(N, feats).astype(theano.config.floatX),*
>>> *rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))*
>>> *training_steps = 1*
>>> *# Declare Theano symbolic variables*
>>> *x = T.matrix("x")*
>>> *y = T.vector("y")*
>>> *w = theano.shared(rng.randn(feats).astype(theano.config.floatX), 
>>> name="w")*
>>> *b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), 
>>> name="b")*
>>> *x.tag.test_value = D[0]*
>>> *y.tag.test_value = D[1]*
>>> *# Construct Theano expression graph*
>>> *p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one*
>>> *prediction = p_1 > 0.5 # The prediction that is done: 0 or 1*
>>> *xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy*
>>> *cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize*
>>> *gw,gb = T.grad(cost, [w,b])*
>>> *# Compile expressions to functions*
>>> *train = theano.function(*
>>> *inputs=[x,y],*
>>> *outputs=[prediction, xent],*
>>> *updates=[(w, w-0.01*gw), (b, b-0.01*gb)],*
>>> *name = "train")*
>>>
>>>
>>> ==
>>>  
>>> It corrupted at this line.
>>> I have run numpy.test() and scipy.test() and they work well, but when I 
>>> run theano.test(), it corrupted too. The full log is too long, so I 
>>> just post
>>> the end of it:
>>>
>>> */home/nice/miniconda2/lib/python2.7/site-packages/
 theano/compile/nanguardmode.py:168:
 RuntimeWarning: All-NaN axis encountered*
 *  return np.isinf(np.nanmax(arr)) or np.isinf(np.nanmin(arr))*
 *.E/home/nice/
 miniconda2/lib/python2.7/site-packages/theano/gof/vm.py:851:
 UserWarning: CVM does not support memory profile, using Stack VM.*
 *  'CVM does not support memory profile, using Stack VM.')*
 *...SS.0.930614401665*
 *0.930614401665*
 *0.930614401665*
 *0.930614401665*
 *...
 
 ...E/home/nice/miniconda2/
 lib/python2.7/site-packages/theano/gof/vm.py:854:
 UserWarning: LoopGC does not support partial evaluation, using Stack 
 VM.*
 *  'LoopGC does not support partial evaluation, '*
 *.Segmentation fault (core dumped)*
>>>
>>>
>>>
>>> I hope someone can help me.  
>>>
>>> -- 
>>>
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "theano-users" group.
>>> To unsubscribe from this group and stop receiving 

[theano-users] Re: Python stops working when importing theano.

[Pascal Lamblin]

This usually happens when mixing libraries compiled with different 
compilers on Windows.
Do you have another g++ than the one installed by the conda package "
m2w64-toolchain" that is mentioned in the installation instructions?

On Thursday, June 1, 2017 at 4:17:40 AM UTC-4, Rohit Dulam wrote:
>
> When i try to import theano, a window pops up saying python.exe has 
> stopped. What is the problem? The contents of my .theanorc file are given 
> below.
>
> [global]
> floatX = float32
> device = cuda0
> mode=FAST_RUN
>
> [cuda]
> root = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin
>
>
> [nvcc]
> fastmath = True
>
> [dnn]
>
> include_path = C:\Program Files\NVIDIA GPU Computing 
> Toolkit\CUDA\v8.0\include
> library_path = C:\Program Files\NVIDIA GPU Computing 
> Toolkit\CUDA\v8.0\lib\x64
>
> [gpuarray]
>
> preallocate = 0.2
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] How to run a single failing test, and what might the error I have be about?

[Pascal Lamblin]

It looks like an issue in the test itself (maybe that only shows itself 
with newer versions of numpy or something).
This can be safely ignored.

On Tuesday, May 30, 2017 at 11:36:24 AM UTC-4, Yimeng Zhang wrote:
>
> It's still happening. I installed Theano 0.9, ran the test for a while, 
> and got the following. 
>
> ==
> ERROR: test_adv_sub1_fast 
> (theano.sandbox.cuda.tests.test_basic_ops.T_subtensor)
> --
> Traceback (most recent call last):
>   File 
> "/home/yimengzh/miniconda2/envs/theano-tf-3/lib/python3.6/site-packages/theano/sandbox/cuda/tests/test_basic_ops.py",
>  
> line 1069, in test_adv_sub1_fast
> data = rand(*shape)
>   File "mtrand.pyx", line 1347, in mtrand.RandomState.rand 
> (numpy/random/mtrand/mtrand.c:19701)
>   File "mtrand.pyx", line 856, in mtrand.RandomState.random_sample 
> (numpy/random/mtrand/mtrand.c:15527)
>   File "mtrand.pyx", line 167, in mtrand.cont0_array 
> (numpy/random/mtrand/mtrand.c:6127)
> TypeError: 'float' object cannot be interpreted as an integer
>
> --
> Ran 20367 tests in 12141.904s
>
> FAILED (SKIP=600, errors=1)
>
> On Tuesday, April 11, 2017 at 5:39:24 PM UTC-4, nouiz wrote:
>>
>> Tests should be run with device=cpu. They enable the GPU individually 
>> when needed.
>>
>> Make sure to Theano theano 0.9, I think this type of errors got fixed 
>> before that release.
>>
>> I would also suggest to the Theanno new gpu back-end:
>>
>>
>> https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
>>
>> Fred
>>
>> On Mon, Mar 20, 2017 at 2:04 AM Marcelo Siero  
>> wrote:
>>
>>> I get this error on a new installation.  It has a GPU, but I have not 
>>> modified my configuration.
>>> It is on a CENTOS 7 machine, and I followed the bleeding edge 
>>> instructions.
>>> When I run:
>>>THEANO_FLAGS=''; python -c "import theano; theano.test()"
>>>
>>> After a long time, I got this error:
>>>Traceback (most recent call last):
>>>   File 
>>> "/home/siero/.local/lib/python2.7/site-packages/theano/sandbox/cuda/tests/test_basic_ops.py",
>>>  
>>> line 1069, in test_adv_sub1_fast
>>> data = rand(*shape)
>>>   File "mtrand.pyx", line 1347, in mtrand.RandomState.rand 
>>> (numpy/random/mtrand/mtrand.c:19701)
>>>   File "mtrand.pyx", line 856, in mtrand.RandomState.random_sample 
>>> (numpy/random/mtrand/mtrand.c:15527)
>>>   File "mtrand.pyx", line 167, in mtrand.cont0_array 
>>> (numpy/random/mtrand/mtrand.c:6127)
>>> TypeError: 'float' object cannot be interpreted as an index
>>>
>>> It then said:
>>>Ran 26010 tests in 15484.861s
>>>
>>> But no summary as to how many other errors.
>>> I would like to run this particular test only.
>>>
>>> I have 2 GPU's and will be enabling them.  But for now, I would like to 
>>> run just the one unit
>>> test that is failing.  I saw no instructions in the FAQs as to how to do 
>>> that.
>>>
>>> -Marcelo
>>>
>>> -- 
>>>
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "theano-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to theano-users...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Lop for matrix Jacobian product?

[Pascal Lamblin]

It is always possible to use scan for that, in fact that is what 
theano.gradient.jacobian does, by looping over each row of the Jacobian.
See 
https://github.com/Theano/Theano/blob/4a8fed96fb8d5faaff4441d19c8aca33350e3db5/theano/gradient.py#L1769

Some optimizations may push parts of the computation out of the loop, to 
avoid recomputing the same thing, or just vectorize them, but unfortunately 
this would still be a performance hit compared to a more optimized 
expression of the same computation.

On Friday, June 16, 2017 at 12:35:45 PM UTC-4, Juan Camilo Gamboa Higuera 
wrote:
>
> Hi,
>
> Is it possible to do this; e.g. the way autograd does it?
>
>
> https://github.com/HIPS/autograd/blob/a055b4282e4e9ac91843463aea04bc7cdee8716f/autograd/convenience_wrappers.py#L30
> )
>
>
> Thanks!
>
> --Jjuan Camilo
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Theano profiling question

[Pascal Lamblin]

Hi,
 = Py shows that the Python interface is used for that Op. However, 
in Scan's case, that Python implementation uses Cython, so most of the 
performance of C should be there.
Python code can also execute the computation on the GPU. In that case, you 
can see in the Op list that the scan Ops print as "forall_inplace,gpu,...", 
which confirm they are actually running on the GPU.

On Monday, June 5, 2017 at 12:01:06 PM UTC-4, Haining Yu wrote:
>
> Hello,
>
> I have a question about how to read theano profiling results. I am running 
> a RNN model, where the gradient is calculated via theano.scan. When I 
> profile the code, I can see that the bulk of time is spent on theano.scan. 
> Moreover I notice that the type of call is Py rather than C. 
>
> My questions are: 
> 1. does  = Py imply that the code is running as python code? 
> 2. does this also imply the code is executed on CPU (instead of GPU)?
>
> Thanks,
> Haining
>
> Below is results from profiling:
>
> Class
> ---
> <% time> <#call> <#apply> 
> 
>   99.3%99.3% 624.260s   1.90e-01s Py3284   2   
> theano.scan_module.scan_op.Scan
>0.2%99.5%   1.237s   1.57e-05s C78816  48   
> theano.sandbox.cuda.basic_ops.GpuAlloc
>0.2%99.7%   1.028s   2.02e-05s C50902  31   
> theano.sandbox.cuda.basic_ops.GpuElemwise
>0.1%99.7%   0.394s   1.64e-06s C   239732 146   
> ...
>
> Ops
> ---
> <% time> <#call> <#apply>  name>
>   87.3%87.3% 548.530s   3.34e-01s Py16421   
> forall_inplace,gpu,grad_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn_of_scan_fn}
>   12.0%99.3%  75.730s   4.61e-02s Py16421   
> forall_inplace,gpu,scan_fn_fn_fn_fn}
>0.2%99.5%   1.237s   1.57e-05s C 78816   48   
>
> ...
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: Cannot do a simple theano install (Python 2.7, Ubuntu 16.04, Theano 0.9, Cuda 8.0, TitanX GPU) due to pygpu errors

[Pascal Lamblin]

On Friday, June 30, 2017 at 9:50:58 PM UTC-4, Daniel Seita wrote:
>
> your plan worked!
>

Nice to hear that :)
 

> Pascal and nouiz, just one last thing, this GPU backend is the first time 
> that Theano can run float64 with the GPU, right? I'm hoping to take 
> existing code which uses float64 only and running it on the GPU. (That's 
> why I've been using the CPU recently, to use float64.)
>

Yes, indeed.
Note that float64 performance can be much slower than float32, depending on 
the GPU. For instance, on a Titan X (Pascal), it would be 32x slower than 
float32, K40 and K80 should be only 3x slower, and P100 / GP100 2x slower.
 

>
>
>
> On Friday, June 30, 2017 at 6:48:08 PM UTC-7, Daniel Seita wrote:
>>
>> Hi nouiz and Pascal, thanks for the responses. I've been busy using the 
>> CPU version of Theano in the meantime, so sorry for the delay in responding.
>>
>> nouiz:
>>
>> I actually had my `cudnn.h` file in both a `lib64` directory and an 
>> `include` directory:
>>
>> ~$ ls -lh /usr/local/cuda-8.0/include/cudnn.h 
>> -r--r--r-- 1 root root 98K Oct 17  2016 /usr/local/cuda-8.0/include/cudnn
>> .h
>> ~$ ls -lh /usr/local/cuda-8.0/lib64/cudnn.h 
>> -r--r--r-- 1 root root 98K Oct 17  2016 /usr/local/cuda-8.0/lib64/cudnn.h
>>
>> I must have copied them to both when I was installing it. Also, here is 
>> my error message in full, assuming that my `~/.theanorc` file is
>>
>> ~$ cat ~/.theanorc
>> [global]
>> device = cuda
>> floatX = float64
>>
>> [cuda]
>> root = /usr/local/cuda-8.0
>> ~$ ipython
>> Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) 
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 5.3.0 -- An enhanced Interactive Python.
>> ? -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help  -> Python's own help system.
>> object?   -> Details about 'object', use 'object??' for extra details.
>>
>> In [1]: import theano
>> ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
>> Traceback (most recent call last):
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
>> , line 164, in 
>> use(config.device)
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
>> , line 151, in use
>> init_dev(device)
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
>> , line 68, in init_dev
>> context.cudnn_handle = dnn._make_handle(context)
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
>> , line 80, in _make_handle
>> cudnn = _dnn_lib()
>>   File 
>> "/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
>> , line 67, in _dnn_lib
>> raise RuntimeError('Could not find cudnn library (looked for v5[.1])'
>> )
>> RuntimeError: Could not find cudnn library (looked for v5[.1])
>>
>> This happens if I also set the device to be `cuda0` instead of `cuda`, 
>> and trying with `float32` instead of `float64`.
>>
>>
>>
>>
>> On Friday, June 30, 2017 at 5:43:35 PM UTC-7, Pascal Lamblin wrote:
>>>
>>> Something weird seems to be happening:
>>> - theano detects that cuDNN is available, in fact it called 
>>> _dnn_check_compile() and _dnn_check_version() successfully
>>> - however, calling _dnn_lib() failed, which means ctypes did not manage 
>>> to find the cudnn library.
>>>
>>> Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to 
>>> LD_LIBRARY_PATH)?
>>>
>>> On Monday, June 19, 2017 at 6:16:37 PM UTC-4, nouiz wrote:
>>>>
>>>> Your cudnn.h file should not be in the lib64 directory, but in an 
>>>> include directory. Tensorflow does none standard stuff related to import 
>>>> and cause problem in other setup, but it seem to tolerate your non 
>>>> standard 
>>>> setup. Theano does the standard setup.
>>>>
>>>> You can use the Theano flag dnn.include_path and dnn.library_path to 
>>>> tell Theano where your cudnn.h and cudnn.so* files are.
>>>>
>>>> I did not see your last error in full.
>>>>
>>>> Le ven. 16 juin 2017 19:35, Daniel Seita <takesh...@gmail.com> a 
>>>> écrit :
>>&

Re: [theano-users] Re: Cannot do a simple theano install (Python 2.7, Ubuntu 16.04, Theano 0.9, Cuda 8.0, TitanX GPU) due to pygpu errors

[Pascal Lamblin]

Something weird seems to be happening:
- theano detects that cuDNN is available, in fact it called 
_dnn_check_compile() and _dnn_check_version() successfully
- however, calling _dnn_lib() failed, which means ctypes did not manage to 
find the cudnn library.

Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to 
LD_LIBRARY_PATH)?

On Monday, June 19, 2017 at 6:16:37 PM UTC-4, nouiz wrote:
>
> Your cudnn.h file should not be in the lib64 directory, but in an include 
> directory. Tensorflow does none standard stuff related to import and cause 
> problem in other setup, but it seem to tolerate your non standard setup. 
> Theano does the standard setup.
>
> You can use the Theano flag dnn.include_path and dnn.library_path to tell 
> Theano where your cudnn.h and cudnn.so* files are.
>
> I did not see your last error in full.
>
> Le ven. 16 juin 2017 19:35, Daniel Seita  a 
> écrit :
>
>> Ack, sorry, half of my post got deleted! Hopefully you can still see it 
>> (i can find it by looking at the original post but it's in a really ugly 
>> format, sorry).
>>
>>
>>
>> On Friday, June 16, 2017 at 4:33:20 PM UTC-7, Daniel Seita wrote:
>>
>>> I was running into some more difficulties, so I gave up on getting this 
>>> to work and tried to uninstall and then reinstall Theano. Just to be extra 
>>> clear, here is my setup:
>>>
>>>- Ubuntu 16.04
>>>- Cuda 8.0, stored in `usr/local/cuda-8.0`
>>>- Titan X GPU with Pascal
>>>
>>> cuDNN is here:
>>>
>>> $ ls /usr/local/cuda-8.0/lib64/cudnn.h 
>>> /usr/local/cuda-8.0/lib64/cudnn.h
>>>
>>> To verify that I can use my GPU I started this quick TensorFlow 
>>> computation:
>>>
>>> In [1]: import tensorflow as tf
>>>
>>> In [2]: tf.__version__
>>> Out[2]: '1.1.0'
>>>
>>> In [3]: tf.GPUOptions
>>> Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions
>>>
>>> In [4]: with tf.device('/gpu:0'):
>>>...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3
>>> ], name='a')
>>>...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2
>>> ], name='b')
>>>...: c = tf.matmul(a,b)
>>>...: 
>>>
>>> In [5]: with tf.Session() as sess:
>>>...: print(sess.run(c))
>>>...: 
>>> 2017-06-16 16:10:54.402311: W tensorflow/core/platform/cpu_feature_guard
>>> .cc:45] The TensorFlow library wasn't compiled to use SSE4.1 
>>> instructions, but these are available on your machine and could speed up 
>>> CPU computations.
>>> 2017-06-16 16:10:54.402328: W 
>>> tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library 
>>> wasn't compiled to use SSE4.2 instructions, but these are available on 
>>> your machine and could speed up CPU computations.
>>> 2017-06-16 16:10:54.402346: W tensorflow/core/platform/cpu_feature_guard
>>> .cc:45] The TensorFlow library wasn't compiled to use AVX instructions, 
>>> but these are available on your machine and could speed up CPU computations.
>>> 2017-06-16 16:10:54.402350: W 
>>> tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library 
>>> wasn't compiled to use AVX2 instructions, but these are available on 
>>> your machine and could speed up CPU computations.
>>> 2017-06-16 16:10:54.402356: W tensorflow/core/platform/cpu_feature_guard
>>> .cc:45] The TensorFlow library wasn't compiled to use FMA instructions, 
>>> but these are available on your machine and could speed up CPU computations.
>>> 2017-06-16 16:10:54.527167: I 
>>> tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA 
>>> node read from SysFS had negative value (-1), but there must be at least 
>>> one NUMA node, so returning NUMA node zero
>>> 2017-06-16 16:10:54.527553: I 
>>> tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with 
>>> properties: 
>>> name: TITAN X (Pascal)
>>> major: 6 minor: 1 memoryClockRate (GHz) 1.531
>>> pciBusID :01:00.0
>>> Total memory: 11.90GiB
>>> Free memory: 11.38GiB
>>> 2017-06-16 16:10:54.527565: I 
>>> tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0 
>>> 2017-06-16 16:10:54.527568: I 
>>> tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0:   Y 
>>> 2017-06-16 16:10:54.527590: I 
>>> tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow 
>>> device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id: 
>>> :01:00.0)
>>> [[ 22.  28.]
>>>  [ 49.  64.]]
>>>
>>>
>>> This looks like it indicates a successful GPU and/or cuDNN installation.
>>>
>>> Great, now let's install the *development version* of Theano. The 
>>> instructions I'm following step-by-step: 
>>> http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html
>>>
>>> The first step seems to be to install miniconda. I downloaded the bash 
>>> script for Python 2.7 and ran it:
>>>
>>> ~/Downloads$ bash Miniconda2-latest-Linux-x86_64.sh 
>>>
>>> Welcome to Miniconda2 4.3.21 (by Continuum Analytics, Inc.)
>>>
>>> In order to continue the installation process, please 

[theano-users] Re: Type mismatch TensorType(float64, row) TensorType(float64, matrix)

[Pascal Lamblin]

It's hard to know from that level, but somewhere there is a shared variable 
that is a row (1 x n matrix), and its update value is a full matrix (k x m).
This could be a parameter of the model, or a value updated by the optimizer 
(something like a velocity).
It is possible that the update value is actually always 1 x n, but Theano 
could not infer that (in which case, this is an issue in Theano and we 
should fix it), in that case you can use tensor.addbroadcast() on the 
update expression (it will fail if k != 1 at runtime).
It is also possible that k != 1, in which case it is a problem in the model 
itself.

On Friday, June 23, 2017 at 3:43:39 AM UTC-4, Vimal K wrote:
>
>
>
> i am trying to run this program  https://github.com/npow/ubottu for 
> making a model for Ubuntu Dialog Corpus
>
> It uses Theano and Lasagne as dependencies.
>
> when i install Theano and Lasagne from pip, and do a simple 
>
> import lasagne
>
>
> i get the error
>
> from theano.tensor.signal import downsampleImportError: cannot import name 
> 'downsample'
>
>
> then found a workaround here, 
> https://stackoverflow.com/questions/42998355/lasagne-vs-theano-possible-version-mismatch-windows
> suggestion was to install Theano and Lasagne from their github master 
> branches respectively
>
> pip install --upgrade https://github.com/Theano/Theano/archive/master.zip
> pip install --upgrade https://github.com/Lasagne/Lasagne/archive/master.zip
>
>
> This removed the above erorr, but i am getting the following error.
>
>
> TypeError: ('An update must have the same type as the original shared 
> variable (shared_var=hid_init, shared_var.type=TensorType(float64, row), 
> update_val=Elemwise{sub,no_inplace}.0, update_val.type=TensorType(float64, 
> matrix)).', 'If the difference is related to the broadcast pattern, you 
> can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function 
> to remove broadcastable dimensions.')
>
>
>
> it seems that there is a type mismatch somewhere. a variable of row type 
> is updated with a value of matrix type. Full traceback as follows :
>
> Traceback (most recent call last):
>   File "main.py", line 669, in 
> main()
>   File "main.py", line 661, in main
> model = Model(**args.__dict__)
>   File "main.py", line 375, in __init__
> self.update_params()
>   File "main.py", line 408, in update_params
> self.train_model = theano.function([], self.cost, updates=updates, 
> givens=givens, on_unused_input='warn')
>   File 
> "/home/vimal/anaconda2/lib/python2.7/site-packages/theano/compile/function.py"
> , line 326, in function
> output_keys=output_keys)
>   File 
> "/home/vimal/anaconda2/lib/python2.7/site-packages/theano/compile/pfunc.py"
> , line 449, in pfunc
> no_default_updates=no_default_updates)
>   File 
> "/home/vimal/anaconda2/lib/python2.7/site-packages/theano/compile/pfunc.py"
> , line 208, in rebuild_collect_shared
> raise TypeError(err_msg, err_sug)
> TypeError: ('An update must have the same type as the original shared 
> variable (shared_var=cell_init, shared_var.type=TensorType(float64, row), 
> update_val=Elemwise{sub,no_inplace}.0, update_val.type=TensorType(float64, 
> matrix)).', 'If the difference is related to the broadcast pattern, you 
> can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function 
> to remove broadcastable dimensions.')
>
>
>
>
>
>
> the versions are
> >>> import theano as th
> >>> print th.__version__
> 0.10.0dev1.dev-RELEASE
>
>
> >>> import lasagne as la
> >>> print la.__version__
> 0.2.dev1
>
>
>
>
>
>
> Can you help me to resolve this error? 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: How can I calculate the size of output of convolutional operation in theano?

[Pascal Lamblin]

On Monday, June 26, 2017 at 3:36:21 PM UTC-4, Sunjeet Jena wrote:
>
> Is there any way I can calculate the size of the output after the 
> Convolution Operation?
>

You can check theano.tensor.nnet.abstract_conv.get_conv_output_shape() 

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] And operator doesn't work with theano logical operators

[Pascal Lamblin]

On Thursday, June 29, 2017 at 10:17:26 AM UTC-4, Sym wrote:
>
> I am using theano 0.9.0 and this function does not exist.. I can't find it 
> in the documentation either !
>

That would be theano.tensor.and_ with a trailing underscore.
 

> And in the theano docs it says that :
> __{abs,neg,lt,le,gt,ge,invert,and,or,add,sub,mul,div,truediv,floordiv}__
> Those elemwise operation are supported via Python syntax.
>

This is true, but the Python syntax that will call `__and__` is actually `a 
&& b`, not `a and b`.

So you can use `a && b`, `and_(a, b)`, or `a * b` if you prefer.
 

>
> Le mercredi 28 juin 2017 21:15:37 UTC-4, nouiz a écrit :
>>
>> Don't use the Python "and" operation. Use theano.tensor.and(a,b) instead. 
>> I think it will fix your problem.
>>
>> Le mer. 28 juin 2017 10:26, Sym  a écrit :
>>
>>>
>>> I want to build a piecewise function with theano, for instance a 
>>> function that is nonzero only in the interval [2,3].
>>>
>>> Here is the minimal code reproducing the error : 
>>>
>>>
>>> import theano
>>> import theano.tensor as T
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>>
>>> r = T.scalar()
>>> gate = T.switch( T.ge(r,2.) and T.le(r,3.) , 1., 0.)
>>> f = theano.function([r],gate)
>>> x = np.arange(0.,4.,0.05,dtype='float32')
>>> y = [f(i) for i in x]
>>> plt.plot(x,y)
>>>
>>>  
>>>
>>> The result is the following : https://i.stack.imgur.com/XMQme.png 
>>>
>>> Which is clearly not correct : only one condition is satisfied here.
>>>
>>>
>>> If I replace T.switch by theano.ifelse.ifelse the result is the same...
>>>
>>> Is it a known bug, or am I missing something here?
>>>
>>>
>>> Thanks a lot !
>>>
>>> -- 
>>>
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "theano-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to theano-users...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Gradient Problem (always 0)

[Pascal Lamblin]

I'm assuming m and n were defined as T.vector(), and that the last line of 
the "def rnn(...)" functions is actually "return x_t, r_t", is that correct?

Do you have a non-zero gradient for Wrec?
Can you monitor the value of theano.grad(cost, Wrec).sum() is?
Normally, the sum of the gradient wrt Wrec should be equal to the gradient 
wrt dot(u, v). So if the gradient wrt Wrec is not zero everywhere, but its 
sum is zero, then it would explain that result.
If we manually backprop, then we can see that the gradient of the cost wrt 
u is equivalent to grad(cost, Wrec).sum() * v (and the gradient wrt v 
should be equivalent to grad(cost, Wrec).sum() * u). Can you monitor those 
values?


On Wednesday, June 28, 2017 at 7:12:06 PM UTC-4, Mohamed Akrout wrote:
>
> Hi all,
>
> I am running a neuroscience with an recurrent neural network model with 
> Theano:
>
>
>
> def rnn(u_t, x_tm1, r_tm1, Wrec):
>  x_t = ( (1 - alpha)*x_tm1 + alpha*(T.dot(r_tm1, Wrec ) + brec + 
> u_t[:,Nin:]) )
>  r_t = f_hidden(x_t)
>
>
> then I define the scan function to iterate at each time step iteration
>
> [x, r], _ = theano.scan(fn=rnn,
> outputs_info=[x0_, f_hidden(x0_)],
> sequences=u,
> non_sequences=[Wrec])
>
> Wrec and brec are learnt by stochastic gradient descent: g = T.grad(cost , 
> [Wrec, brec])
>
> where cost is the cost function: T.sum(f_loss(z, target[:,:,:Nout])) with 
> z = f_output(T.dot(r, Wout_.T) + bout )
>
> Until now, everything works good.
>
>
>
> Now I want to add two new vectors, let's call them u and v so that the 
> initial rnn function becomes:
>
>
> def rnn(u_t, x_tm1, r_tm1, Wrec, *u, v*):
>  x_t = ( (1 - alpha)*x_tm1 + alpha*(T.dot(r_tm1, Wrec + *T.dot(u, 
> v)* ) + brec + u_t[:,Nin:]) )
>  r_t = f_hidden(x_t)
>
> [x, r], _ = theano.scan(fn=rnn,
> outputs_info=[x0_, f_hidden(x0_)],
> sequences=u,
> non_sequences=[Wrec,* m, n*])
>
> m and n are the variables corresponding to u and v in the main function.
>
> and suddenly, the gradient T.grad(cost, m) and T.grad(cost, n) are zeros
>
> I am blocked since 2 weeks now on this problem. I verified that the values 
> are not integer by using dtype=theano.config.floatX every where in the 
> definition of the variables.
>
> As you can see the link between the cost and m (or n) is: the cost 
> function depends on  z, and z depends on r and r is one of the outputs of 
> the rnn function that uses m and n in the equation.
>
> Do you have any ideas why this does not work ?
>
> Any idea is welcome. I hope I can unblock this problem soon.
> Thank you!
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: CUDNN_STATUS_EXECUTION_FAILED when using dnn.conv.algo_bwd_filter=deterministic

[Pascal Lamblin]

Thanks for the update.
I managed to reproduce the issue with cuDNN v6 as well, with a simple 
script (below).
- with 'deterministic' it fails with CUDNN_STATUS_EXECUTION_FAILED
- with 'fft_tiling' it fails with CUDNN_STATUS_NOT_SUPPORTED
- with 'fft', surprisingly, it works. 'fft' is supposed to be deterministic 
as well, so you could also use that one.

Thanks for the report, we'll forward that to Nvidia.

```
import theano
import numpy as np


x = theano.shared(np.ones((1, 1, 541211, 10), 'f'))
y = theano.shared(np.ones((1, 50, 541211, 1), 'f'))
z = theano.tensor.nnet.abstract_conv.conv2d_grad_wrt_weights(x, y, 
filter_shape=(50, 1, 3, 10), border_mode=(1, 0), filter_flip=False)
f = theano.function([], z)
f()
```

On Tuesday, June 20, 2017 at 5:28:34 AM UTC-4, Fabian Stemmer wrote:
>
> I tried using cudnn v6, but still got the same error.
>
> I also added 'fft_tiling' to SUPPORTED_DNN_CONV_ALGO_RUNTIME in 
> cofigdefaults.py, to be able to test it, but still got the cuDNN error (see 
> below).
>
> I then added 'optimizer_excluding=conv_dnn' to my THEANO_FLAGS, which gave 
> me GpuCorrMM nodes in the computational graph. This runs without errors.
>
> GpuCorrMM gives me deterministic results, so I can use it as an 
> alternative to the deterministic cuDNN algorithm.
>
> Thanks for your help.
>
> On Tuesday, June 20, 2017 at 12:15:55 AM UTC+2, nouiz wrote:
>>
>> Try cudnn v6. The GPU that have problem are more recent. Maybe it was not 
>> implemented case in v5.
>>
>> Le lun. 19 juin 2017 16:02, Pascal Lamblin <lamb...@iro.umontreal.ca> a 
>> écrit :
>>
>>>
>>>
>>> On Monday, June 19, 2017 at 3:39:17 PM UTC-4, Pascal Lamblin wrote:
>>>>
>>>> Hi,
>>>>
>>>> Unfortunately, it looks like a runtime issue in cuDNN rather than 
>>>> somehting in the Theano wrapper, but I could be wrong.
>>>> A recent PR introduced more algorithms that you can specify for 
>>>> dnn.conv.algo_bwd_filter. In particular, 
>>>> dnn.conv.algo_bwd_filter=fft_tiling should be deterministic as well.
>>>>
>>>
>>> Actually, I just realized the value gets rejected by the configuration, 
>>> but if we bypass it in theano/configdefaults.py it should work. This should 
>>> be fixed soon.
>>>  
>>>
>>>>
>>>> Does it work with an input and kernel that are smaller than 541211 on 
>>>> that dimension?
>>>> Does it work using corrMM instead of cuDNN?
>>>>
>>>> On Wednesday, June 7, 2017 at 11:19:31 AM UTC-4, Fabian Stemmer wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm using theano.tensor.nnet.conv2d in my model and I want to set 
>>>>> dnn.conv.algo_bwd_filter=deterministic to make this run deterministically 
>>>>> on GPUs. I work on three different GPU architectures (K10, M40, P6000) 
>>>>> and 
>>>>> setting the mentioned flag works well on the K10, but fails with error 
>>>>> message CUDNN_STATUS_EXECUTION_FAILED on the other two. I have tried 
>>>>> several combinations of theano, nvidia driver and cuDNN versions, but 
>>>>> none 
>>>>> fix the issue. 
>>>>>
>>>>> Below are details about the respective GPU configurations I tried and 
>>>>> the full error message. Any help you can give me is greatly appreciated.
>>>>>
>>>>> Thanks
>>>>> Fabian
>>>>>
>>>>>
>>>>> *Shared setup (all GPUs):*Theano 0.8.2 / 0.9.0 / 0.10.0.dev1 (commit 
>>>>> 6b59449186b04225484b98951192c5867e0719ca, which was the latest at the 
>>>>> time 
>>>>> of this writing)
>>>>> cuda 8.0
>>>>> cuDNN 5105
>>>>> THEANO_FLAGS=mode=FAST_RUN,floatX=float32,lib.cnmem=1,
>>>>> *dnn.conv.algo_bwd_filter=deterministic*,device=cuda //device=gpu for 
>>>>> theano 0.8.2
>>>>>
>>>>> *GPU and Nvidia driver:*
>>>>> Tesla K10 Architecture (Driver 361.93.03)
>>>>> Tesla M40 Architecture (Driver: 375.26)
>>>>> Quadro P6000 (Driver 375.26)
>>>>>
>>>>> Alternative driver versions (all tested on Tesla M40):
>>>>>
>>>>>1. 361.93.03 - Current Production Driver on K10/K20/K80 servers - 
>>>>>No difference. Application fails on the M40 node
>>>>>2. 375.26 - Current Production driver on M40/P100/P6000 servers - 
>>>>>App fails
>>>&

[theano-users] Re: CUDNN_STATUS_EXECUTION_FAILED when using dnn.conv.algo_bwd_filter=deterministic

[Pascal Lamblin]

On Monday, June 19, 2017 at 3:39:17 PM UTC-4, Pascal Lamblin wrote:
>
> Hi,
>
> Unfortunately, it looks like a runtime issue in cuDNN rather than 
> somehting in the Theano wrapper, but I could be wrong.
> A recent PR introduced more algorithms that you can specify for 
> dnn.conv.algo_bwd_filter. In particular, 
> dnn.conv.algo_bwd_filter=fft_tiling should be deterministic as well.
>

Actually, I just realized the value gets rejected by the configuration, but 
if we bypass it in theano/configdefaults.py it should work. This should be 
fixed soon.
 

>
> Does it work with an input and kernel that are smaller than 541211 on that 
> dimension?
> Does it work using corrMM instead of cuDNN?
>
> On Wednesday, June 7, 2017 at 11:19:31 AM UTC-4, Fabian Stemmer wrote:
>>
>> Hi,
>>
>> I'm using theano.tensor.nnet.conv2d in my model and I want to set 
>> dnn.conv.algo_bwd_filter=deterministic to make this run deterministically 
>> on GPUs. I work on three different GPU architectures (K10, M40, P6000) and 
>> setting the mentioned flag works well on the K10, but fails with error 
>> message CUDNN_STATUS_EXECUTION_FAILED on the other two. I have tried 
>> several combinations of theano, nvidia driver and cuDNN versions, but none 
>> fix the issue. 
>>
>> Below are details about the respective GPU configurations I tried and the 
>> full error message. Any help you can give me is greatly appreciated.
>>
>> Thanks
>> Fabian
>>
>>
>> *Shared setup (all GPUs):*Theano 0.8.2 / 0.9.0 / 0.10.0.dev1 (commit 
>> 6b59449186b04225484b98951192c5867e0719ca, which was the latest at the time 
>> of this writing)
>> cuda 8.0
>> cuDNN 5105
>> THEANO_FLAGS=mode=FAST_RUN,floatX=float32,lib.cnmem=1,
>> *dnn.conv.algo_bwd_filter=deterministic*,device=cuda //device=gpu for 
>> theano 0.8.2
>>
>> *GPU and Nvidia driver:*
>> Tesla K10 Architecture (Driver 361.93.03)
>> Tesla M40 Architecture (Driver: 375.26)
>> Quadro P6000 (Driver 375.26)
>>
>> Alternative driver versions (all tested on Tesla M40):
>>
>>1. 361.93.03 - Current Production Driver on K10/K20/K80 servers - No 
>>difference. Application fails on the M40 node
>>2. 375.26 - Current Production driver on M40/P100/P6000 servers - App 
>>fails
>>3. 375.51 - Most recent driver with CUDA Repo equivalent - App fails
>>4. 375.66 - Most recent official driver for Quadro/Tesla cards - App 
>>fails
>>
>> I also tried upgrading to cuDNN 6.0 and still got the same error.
>>
>>
>> *Full error message (on Quadro P6000, using theano 0.10.0.dev1:*
>>
>> Using cuDNN version 5105 on context None
>> Mapped name None to device cuda: Quadro P6000 (:04:00.0)
>> Traceback (most recent call last):
>>   File 
>> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/py/bin/n3lu_train",
>>  
>> line 9, in 
>> load_entry_point('n3lu', 'console_scripts', 'n3lu_train')()
>>   File 
>> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/n3lu/n3lu/training.py",
>>  
>> line 507, in main
>> valid_error, test_error = exp.run()
>>   File 
>> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/n3lu/n3lu/training.py",
>>  
>> line 475, in run
>> return self.run_one(self.train_corpus, self.valid_corpus)
>>   File 
>> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/n3lu/n3lu/training.py",
>>  
>> line 384, in run_one
>> learner.run()
>>   File 
>> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/n3lu/n3lu/learning.py",
>>  
>> line 448, in run
>> train_outputs = self.train(*batch)
>>   File 
>> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/py/lib/python2.7/site-packages/theano/compile/function_module.py",
>>  
>> line 898, in __call__
>> storage_map=getattr(self.fn, 'storage_map', None))
>>   File 
>> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/py/lib/python2.7/site-packages/theano/gof/link.py",
>>  
>> line 325, in raise_with_op
>> reraise(exc_type, exc_value, exc_trace)
>>   File 
>> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/py/lib/python2.7/site-packages/theano/compile/function_module.py",
>>  
>> line 884, in __call__
>> self.fn() if output_subset is None else\
>> *RuntimeError: error doing operation: CUDNN_STATUS_EXECUTION_FAILED*
>

[theano-users] Re: CUDNN_STATUS_EXECUTION_FAILED when using dnn.conv.algo_bwd_filter=deterministic

[Pascal Lamblin]

Hi,

Unfortunately, it looks like a runtime issue in cuDNN rather than somehting 
in the Theano wrapper, but I could be wrong.
A recent PR introduced more algorithms that you can specify for 
dnn.conv.algo_bwd_filter. In particular, 
dnn.conv.algo_bwd_filter=fft_tiling should be deterministic as well.

Does it work with an input and kernel that are smaller than 541211 on that 
dimension?
Does it work using corrMM instead of cuDNN?

On Wednesday, June 7, 2017 at 11:19:31 AM UTC-4, Fabian Stemmer wrote:
>
> Hi,
>
> I'm using theano.tensor.nnet.conv2d in my model and I want to set 
> dnn.conv.algo_bwd_filter=deterministic to make this run deterministically 
> on GPUs. I work on three different GPU architectures (K10, M40, P6000) and 
> setting the mentioned flag works well on the K10, but fails with error 
> message CUDNN_STATUS_EXECUTION_FAILED on the other two. I have tried 
> several combinations of theano, nvidia driver and cuDNN versions, but none 
> fix the issue. 
>
> Below are details about the respective GPU configurations I tried and the 
> full error message. Any help you can give me is greatly appreciated.
>
> Thanks
> Fabian
>
>
> *Shared setup (all GPUs):*Theano 0.8.2 / 0.9.0 / 0.10.0.dev1 (commit 
> 6b59449186b04225484b98951192c5867e0719ca, which was the latest at the time 
> of this writing)
> cuda 8.0
> cuDNN 5105
> THEANO_FLAGS=mode=FAST_RUN,floatX=float32,lib.cnmem=1,
> *dnn.conv.algo_bwd_filter=deterministic*,device=cuda //device=gpu for 
> theano 0.8.2
>
> *GPU and Nvidia driver:*
> Tesla K10 Architecture (Driver 361.93.03)
> Tesla M40 Architecture (Driver: 375.26)
> Quadro P6000 (Driver 375.26)
>
> Alternative driver versions (all tested on Tesla M40):
>
>1. 361.93.03 - Current Production Driver on K10/K20/K80 servers - No 
>difference. Application fails on the M40 node
>2. 375.26 - Current Production driver on M40/P100/P6000 servers - App 
>fails
>3. 375.51 - Most recent driver with CUDA Repo equivalent - App fails
>4. 375.66 - Most recent official driver for Quadro/Tesla cards - App 
>fails
>
> I also tried upgrading to cuDNN 6.0 and still got the same error.
>
>
> *Full error message (on Quadro P6000, using theano 0.10.0.dev1:*
>
> Using cuDNN version 5105 on context None
> Mapped name None to device cuda: Quadro P6000 (:04:00.0)
> Traceback (most recent call last):
>   File 
> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/py/bin/n3lu_train",
>  
> line 9, in 
> load_entry_point('n3lu', 'console_scripts', 'n3lu_train')()
>   File 
> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/n3lu/n3lu/training.py",
>  
> line 507, in main
> valid_error, test_error = exp.run()
>   File 
> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/n3lu/n3lu/training.py",
>  
> line 475, in run
> return self.run_one(self.train_corpus, self.valid_corpus)
>   File 
> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/n3lu/n3lu/training.py",
>  
> line 384, in run_one
> learner.run()
>   File 
> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/n3lu/n3lu/learning.py",
>  
> line 448, in run
> train_outputs = self.train(*batch)
>   File 
> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/py/lib/python2.7/site-packages/theano/compile/function_module.py",
>  
> line 898, in __call__
> storage_map=getattr(self.fn, 'storage_map', None))
>   File 
> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/py/lib/python2.7/site-packages/theano/gof/link.py",
>  
> line 325, in raise_with_op
> reraise(exc_type, exc_value, exc_trace)
>   File 
> "/gpfs/hcnlp/data/users/fabian_stemmer/n3lu/environments/n3lu_0.5.2/py/lib/python2.7/site-packages/theano/compile/function_module.py",
>  
> line 884, in __call__
> self.fn() if output_subset is None else\
> *RuntimeError: error doing operation: CUDNN_STATUS_EXECUTION_FAILED*
> Apply node that caused the error: GpuDnnConvGradW{algo='deterministic', 
> inplace=True}(GpuContiguous.0, GpuContiguous.0, 
> GpuAllocEmpty{dtype='float32', context_name=None}.0, 
> GpuDnnConvDesc{border_mode=(1, 0), subsample=(1, 1), conv_mode='cross', 
> precision='float32'}.0, Constant{1.0}, Constant{0.0})
> Toposort index: 234
> Inputs types: [GpuArrayType(float32, (True, True, False, False)), 
> GpuArrayType(float32, (True, False, False, False)), 
> GpuArrayType(float32, (False, True, False, False)), 
> , Scalar(float32), 
> Scalar(float32)]
> Inputs shapes: [(1, 1, 541211, 10), (1, 50, 541211, 1), (50, 1, 3, 10), 
> 'No shapes', (), ()]
> Inputs strides: [(21648440, 21648440, 40, 4), (108242200, 2164844, 4, 4), 
> (120, 120, 40, 4), 'No strides', (), ()]
> Inputs values: ['not shown', 'not shown', 'not shown',  NULL at 0x7ff55d00fe10>, 1.0, 0.0]
> Outputs clients: [[GpuIncSubtensor{Inc;::, ::, ::, 
> int64:int64:}(GpuAlloc{memset_0=True}.0, 
> GpuDnnConvGradW{algo='deterministic', inplace=True}.0, Constant{0}, 
> 

Re: [theano-users] Theano: MissingInputError

[Pascal Lamblin]

The error is raised by the line that says:
fz=theano.function([], z, updates=updates)

because inputs are missing there.

Moreover, even if you provided inputs, the "out" returned would just be
a numpy array, that would be treated like a constant by the rest of the
graph.

The thing you want to do is keeping the updates symbolic until you pass
then to "train", not try to apply them through another theano functin.

On Tue, May 09, 2017, Giuseppe Angora wrote:
> Hi,
> 
> this is a semplification of the original code that i'm trying to fix. I 
> have invented a problem that reflects what the original code does and which 
> gives me the same error but is written in a minimal form:
> 
> import theanoimport theano.tensor as Timport numpy as npimport copy
> theano.config.exception_verbosity = 'high'
> 
> class Test(object):
> def __init__(self):
> self.rate=0.01
> def start(self, x, y):
> W_val=40.89
> W=theano.shared(value=W_val, borrow=True)
> z=T.mean(x*W/y)
> gz=T.grad(z, W)
> updates=[(W, W-self.rate*gz)]
> 
> fz=theano.function([], z, updates=updates)
> for i in range(100):
> out=fz()
> self.out=out
> return out
> 
>  x_set=np.random.rand(10)
>  y_set=np.random.randint(low=0, high=5, size=10, dtype=int)
>  batch_size=2
> 
>  x = T.dvector('x')
>  y = T.ivector('y')
>  index = T.lscalar()
> 
> test = Test()
> cost=test.start(x,y)
> 
> train = theano.function(
> inputs=[index],
> outputs=cost,
> givens={
> x: x_set[index * batch_size: (index + 1) * batch_size],
> y: y_set[index * batch_size: (index + 1) * batch_size]
>})
> for i in range(5):
> result=train(i)
> print(result)
> 
> 
> This is the TraceBack:
> 
> Traceback (most recent call last):
>   File "functions.py", line 33, in 
> cost=test.start(x,y)
>   File "functions.py", line 18, in start
> fz=theano.function([], z, updates=updates)
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\function.py", line 320, in 
> function
> output_keys=output_keys)
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\pfunc.py", line 479, in pfunc
> output_keys=output_keys)
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 
> 1776, in orig_function
> output_keys=output_keys).create(
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 
> 1428, in __init__
> accept_inplace)
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 
> 177, in std_fgraph
> update_mapping=update_mapping)
>   File "C:\Program Files\Anaconda3\lib\site-packages\theano\gof\fg.py", line 
> 171, in __init__
> self.__import_r__(output, reason="init")
>   File "C:\Program Files\Anaconda3\lib\site-packages\theano\gof\fg.py", line 
> 360, in __import_r__
> self.__import__(variable.owner, reason=reason)
>   File "C:\Program Files\Anaconda3\lib\site-packages\theano\gof\fg.py", line 
> 465, in __import__
> detailed_err_msg)
> theano.gof.fg.MissingInputError: A variable that is an input to the graph was 
> neither provided as an input to the function nor given a value. A chain of 
> variables leading from this input to an output is [x, 
> Elemwise{mul,no_inplace}.0, Elemwise{true_div,no_inplace}.0, 
> Sum{acc_dtype=float64}.0, mean]. This chain may not be uniqueBacktrace when 
> the variable is created:
>   File "functions.py", line 28, in 
> x = T.dvector('x')
> 
> I do not know where to slam my head again... 
> 
> Thanks
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Bug with advanced indexing recently fixed in the dev branch

[Pascal Lamblin]

Hi all,

We wanted to let you know that we just fixed a bug that could let to
incorrect results when using indexing by an array of integers, when some
integers were duplicated.

More precisely, when an expression like this was detected:
inc_subtensor(zeros[idx], x)[idx]
where idx is an array of ints, it was always replaced by "x".
However, that is incorrect when some indices in idx are duplicated.

In the development version, there is now a warning if that optimization
would have been applied, but whether it actually let to incorrect
results depend on whether there actually are duplicates in the indices.

That issue had been introduced back in 2014.
Thanks to Dzmitry Bahdanau for reporting and pinpointing the issue.

-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Question about using T.grad when the cost cannot be provided, but only some gradient

[Pascal Lamblin]

Also, if the final cost has some direct contribution from the
parameters, that do not go through the output (for instance, weight
norm penalty), then this will give different results, unless that
contribution to the global cost is passed as the cost (instead of None).

On Thu, May 11, 2017, Pascal Lamblin wrote:
> On Sun, May 07, 2017, Vena Li wrote:
> > I have a question about computing gradient. In general, I can use the 
> > theano.gradient.grad to compute the gradient for the whole graph given the 
> > cost. In my situation, I cannot compute the cost, but I know the gradient 
> > of the weights last layer. Can I still automatically compute the gradient 
> > of previous layers? My understanding is that I should still use the same 
> > function, but put the cost as None and give the ones I know as known_grads.
> 
> You would probably need the gradient wrt the output of the last layer
> (the activations, or pre-activation values), rather than the parameters.
> In general, the gradient wrt the bias will correspond to the gradient
> wrt the pre-activation value.
> 
> > 
> > I tried a small example, where
> > 
> > The original gradient is the following
> > 
> > original_all_grad = T.grad(cost=loss_train, wrt=params)
> > 
> > These are the parameters
> > params = lasagne.layers.get_all_params(model, trainable=True)
> > last_layer_params =model.get_params(trainable=True)
> > other_param = params[0:-1]
> > 
> > I computed the last layer gradient still using the same cost, although 
> > later this would change.
> > known_grad = T.grad(loss_train, last_layer_params)
> > 
> > Compute the gradient with respect to the known gradient.
> > output = lasagne.layers.get_output(model)
> > dic = OrderedDict([(output, known_grad[0])])
> > all_grad = T.grad(cost=None,wrt=other_param, known_grads=dic)
> > 
> > The surprising result is that the all_grad and original_all_grad will not 
> > be identical values for other params. I am not sure what I did wrong here.
> > 
> > I am really gratefully for any help.
> 
> My guess is that you gave the gradient wrt the bias as a "known
> gradient" wrt the post-activation output, but it corresponds to the
> pre-activation value.
> 
> If you want to use the post-activation output, then you can do something
> like this (I'm not sure if it will work exactly like that since I'm not
> sure how Lasagne builds its graph):
> 
> known_grad = T.grad(loss_train, output)
> dic = OrderedDict([(output, known_grad)])
> all_grad = T.grad(cost=None, wrt=..., known_grads=dic)
> 
> or, if you want to use the pre-activation one, you would have to get the
> Theano variable that is the input of the activation function of the last
> layer, say we call it preact_output, and then:
> 
> all_grad = T.grad(cost=None, wrt=..., known_grads=[(preact_output, 
> known_grad)]
> 
> > 
> > Vena
> > 
> > 
> > -- 
> > 
> > --- 
> > You received this message because you are subscribed to the Google Groups 
> > "theano-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an 
> > email to theano-users+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> 
> 
> -- 
> Pascal
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: GpuArrayException

[Pascal Lamblin]

Here is the beginning of the error message:
> An unexpected GpuArrayException exception occurred. Traceback will be 
> written to debug log. The error message was: b'out of memory'

The output of your dot product would be of size (482112, 4000) in float32,
for a total of 7.7 GB.

It may very well exceed the amount of free memory on the GPU.

On Wed, May 10, 2017, Siva Reddy Gangireddy wrote:
> Hi,
> 
> I am getting the GpuArrayException while validating and saving the model. 
> Not during the training.
> 
> I couldn't understand the error.  Could you look and tell me what might be 
> the problem.
> 
> Thank you.
> 
> Log:
> 
> Using cuDNN version 5103 on context None
> Mapped name None to device cuda0: Tesla K80 (:05:00.0)
> /scratch/elec/puhe/Modules/opt/Theano/Theano-0c53fb5/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/sandbox/rng_mrg.py:1521:
>  
> UserWarning: MRG_RandomStreams.multinomial_wo_replacement() is deprecated 
> and will be removed in the next release of Theano. Please use 
> MRG_RandomStreams.choice() instead.
>   warnings.warn('MRG_RandomStreams.multinomial_wo_replacement() is '
> An unexpected GpuArrayException exception occurred. Traceback will be 
> written to debug log. The error message was: b'out of memory'
> Apply node that caused the error: GpuDot22(GpuReshape{2}.0, 
> layers/hidden_layer_1/layer_input/W)
> Toposort index: 62
> Inputs types: [GpuArrayType(float32, (False, False)), 
> GpuArrayType(float32, (False, False))]
> Inputs shapes: [(482112, 200), (200, 4000)]
> Inputs strides: [(800, 4), (16000, 4)]
> Inputs values: ['not shown', 'not shown']
> Inputs type_num: [11, 11]
> Outputs clients: [[GpuReshape{3}(GpuDot22.0, MakeVector{dtype='int64'}.0)]]
> 
> Debugprint of the apply node: 
> GpuDot22 [id A]  ''   
>  |GpuReshape{2} [id B]  ''   
>  | |GpuAdvancedSubtensor1 [id C]  False))> ''   
>  | | |layers/projection_layer/W [id D]  False))>
>  | | |GpuContiguous [id E]  ''   
>  | |   |GpuReshape{1} [id F]  ''   
>  | | |GpuSubtensor{:int64:} [id G]  False))> ''   
>  | | | |GpuFromHost [id H]  False))> ''   
>  | | | | |textscorer/batch_word_ids [id I] 
>  | | | |Constant{-1} [id J] 
>  | | |TensorConstant{(1,) of -1} [id K] 
>  | |MakeVector{dtype='int64'} [id L]  ''   
>  |   |Elemwise{mul,no_inplace} [id M]  ''   
>  |   | |Elemwise{Switch}[(0, 2)] [id N]  ''   
>  |   | | |Elemwise{eq,no_inplace} [id O]  ''   
>  |   | | | |Elemwise{Switch}[(0, 2)] [id P]  '' 
>   
>  |   | | | | |Elemwise{lt,no_inplace} [id Q]  '' 
>   
>  |   | | | | | |Elemwise{add,no_inplace} [id R]  
> ''   
>  |   | | | | | | |TensorConstant{-1} [id S] 
>  |   | | | | | | |Shape_i{0} [id T]  ''   
>  |   | | | | | |   |textscorer/batch_word_ids [id I]  matrix)>
>  |   | | | | | |TensorConstant{0} [id U] 
>  |   | | | | |TensorConstant{0} [id U] 
>  |   | | | | |Elemwise{add,no_inplace} [id R]  
> ''   
>  |   | | | |TensorConstant{-1} [id V] 
>  |   | | |Elemwise{switch,no_inplace} [id W]  '' 
>   
>  |   | | | |Elemwise{lt,no_inplace} [id Q]  ''   
>  |   | | | |TensorConstant{0} [id U] 
>  |   | | | |Elemwise{Composite{((i0 * (i1 * i2)) // (i1 * i2))}}[(0, 0)] 
> [id X]  ''   
>  |   | | |   |Shape_i{1} [id Y]  ''   
>  |   | | |   | |layers/projection_layer/W [id D] 
> 
>  |   | | |   |Shape_i{1} [id Z]  ''  
>  |   | | |   | |textscorer/batch_word_ids [id I] 
>  |   | | |   |Elemwise{add,no_inplace} [id R]  
> ''   
>  |   | | |Elemwise{Switch}[(0, 2)] [id P]  ''   
>  |   | |Shape_i{1} [id Z]  ''   
>  |   |Elemwise{switch,no_inplace} [id W]  ''   
>  |layers/hidden_layer_1/layer_input/W [id BA]  (False, False))>
> 
> Storage map footprint:
>  - GpuReshape{2}.0, Shape: (482112, 200), ElemSize: 4 Byte(s), TotalSize: 
> 385689600 Byte(s)
>  - layers/hidden_layer_1/step_input/W, Shared Input, Shape: (1000, 4000), 
> ElemSize: 4 Byte(s), TotalSize: 1600 Byte(s)
>  - layers/highway_layer_1/input/W, Shared Input, 

Re: [theano-users] Error "pygpu.gpuarray.GpuArrayException: invalid resource handle" when using multiple GPUs with new backend

[Pascal Lamblin]

Unfortunately, model parallelism using different GPUs in the same
process is still experimental, and not a current focus for development.


On Wed, May 10, 2017, T Qri wrote:
> Hi everyone,
> 
> In brief, I'm implementing a model with 2 seq2seq components which share an 
> encoder. Each of the 2 components are distributed across different GPU 
> devices and the shared encoder is also placed on one of these 2 devices.
> 
> When config.optimizer is set to None, the error doesn't exist anymore but 
> the model runs on multiple CPUs quite slowly.
> 
> The detailed Theano flags are described in `theano-flags` file. And the 
> corresponding error message is in `error-message` file.
> 
> I tried to disable constant_folding and elemwise optimization with 
> `optimizer_excluding=constant_folding elemwise` set but it didn't work.
> 
> I'm using  Theano 0.9.0 version on Ubuntu 14.04. There are 4 GPUs available 
> and 2 of them are involved in my model.
> 
> Thanks for your help.
> 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.




-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Question about using T.grad when the cost cannot be provided, but only some gradient

[Pascal Lamblin]

On Sun, May 07, 2017, Vena Li wrote:
> I have a question about computing gradient. In general, I can use the 
> theano.gradient.grad to compute the gradient for the whole graph given the 
> cost. In my situation, I cannot compute the cost, but I know the gradient 
> of the weights last layer. Can I still automatically compute the gradient 
> of previous layers? My understanding is that I should still use the same 
> function, but put the cost as None and give the ones I know as known_grads.

You would probably need the gradient wrt the output of the last layer
(the activations, or pre-activation values), rather than the parameters.
In general, the gradient wrt the bias will correspond to the gradient
wrt the pre-activation value.

> 
> I tried a small example, where
> 
> The original gradient is the following
> 
> original_all_grad = T.grad(cost=loss_train, wrt=params)
> 
> These are the parameters
> params = lasagne.layers.get_all_params(model, trainable=True)
> last_layer_params =model.get_params(trainable=True)
> other_param = params[0:-1]
> 
> I computed the last layer gradient still using the same cost, although later 
> this would change.
> known_grad = T.grad(loss_train, last_layer_params)
> 
> Compute the gradient with respect to the known gradient.
> output = lasagne.layers.get_output(model)
> dic = OrderedDict([(output, known_grad[0])])
> all_grad = T.grad(cost=None,wrt=other_param, known_grads=dic)
> 
> The surprising result is that the all_grad and original_all_grad will not be 
> identical values for other params. I am not sure what I did wrong here.
> 
> I am really gratefully for any help.

My guess is that you gave the gradient wrt the bias as a "known
gradient" wrt the post-activation output, but it corresponds to the
pre-activation value.

If you want to use the post-activation output, then you can do something
like this (I'm not sure if it will work exactly like that since I'm not
sure how Lasagne builds its graph):

known_grad = T.grad(loss_train, output)
dic = OrderedDict([(output, known_grad)])
all_grad = T.grad(cost=None, wrt=..., known_grads=dic)

or, if you want to use the pre-activation one, you would have to get the
Theano variable that is the input of the activation function of the last
layer, say we call it preact_output, and then:

all_grad = T.grad(cost=None, wrt=..., known_grads=[(preact_output, known_grad)]

> 
> Vena
> 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] how to Rearrange columns of 2D array ?

[Pascal Lamblin]

You can use the same syntax as numpy.

On Sat, May 06, 2017, aLEE wrote:
> 
> as this question about numpy  [link 
> 
> ]
> 
> # a  b  c  d  e  f
> A = numpy.array([[0, 1, 2, 3, 4, 5],
>  [1, 0, 3, 4, 5, 6],
>  [2, 3, 0, 5, 6, 7],
>  [3, 4, 5, 0, 7, 8],
>  [4, 5, 6, 7, 0, 9],
>  [5, 6, 7, 8, 9, 0]])
> #a  d  b  e  c  f
> new_order = [0, 3, 1, 4, 2, 5]
> A1 = A[:, new_order][new_order]
> 
> 
> 
> 
> for a theano.tensor, how to implement this ? 
> 
> thanks 
> 
> 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: Variable number of arguments in scan

[Pascal Lamblin]

On Fri, May 05, 2017, Gilles Degottex wrote:
> Sorry, slight confusion, by "chose the order arbitrarily" I didn't mean on 
> the fly. I meant be able to selection what ever order, but once for all 
> once running.
> I'm not actually changing anything once compiled.

Then it should work fine.

> 
> 
> 
> On Friday, 5 May 2017 02:56:03 UTC+1, Adam Becker wrote:
> >
> > I don't think this works. The inner function of scan will be converted to 
> > a graph, then getting compiled inside ScanOp. If you change nonlocal 
> > variable "order" on the fly, the change won't be reflected on the compiled 
> > function.
> >
> > If the inner loop itself can be written as scan, you can just make a 
> > nested scan instead. Compile the innermost graph with scan first (by hand), 
> > then pass it as fn to the outer scan.
> >
> > On Wednesday, May 3, 2017 at 8:04:46 PM UTC+8, Gilles Degottex wrote:
> >>
> >> Hi,
> >>
> >> I'm new on this list, so I hope I'm not duplicating a subject that has 
> >> already been covered and answered.
> >>
> >> I'm trying to implement High-Order RNN (https://arxiv.org/abs/1605.00064) 
> >> and I would like to let me chose the order arbitrarily, which changes the 
> >> number parameters to pass to the inner scan function.
> >> For this purpose, I'm doing something like:
> >>
> >> def step(in_t, *args):
> >>
> >> W_xi_ = args[-2]
> >> b_i_ = args[-1]
> >> args = args[:-2]
> >>
> >> h_t = T.dot(in_t, W_xi_)
> >>
> >> for p in xrange(order):
> >> h_t += T.dot(args[p], args[order+p])
> >>
> >> h_t = nonlinearity(h_t + b_i_)
> >>
> >> return h_t
> >>
> >> h, _ = theano.scan(step, sequences=[invalues],
> >> outputs_info=[dict(initial=hid_init, taps=range(-order,0))],
> >> non_sequences=W_hip+[W_xi, b_i], strict=True)
> >>
> >> W_hip being a list of shared matrices, one for each tap.
> >>
> >> Basically, it compiles, I can train such a model, so it looks like it 
> >> works.
> >>
> >> However, I've strictly no clue if the code is doing what I'm expecting it 
> >> to do.
> >> Is is okay to use a for loop in th inner scan function?
> >> Does scan behave nicely with a variable number of argument?
> >>
> >> Any tips to help verifying/checking that a code is doing what it is 
> >> supposed to do is also very welcome.
> >>
> >> Cheers,
> >>
> >>
> >>
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] GpuArrayException: b'an illegal memory access was encountered'

[Pascal Lamblin]

Could you try running the whole thing inside cuda-memcheck?

On Wed, Apr 05, 2017, Sergey Ovcharenko wrote:
> Hi,
> 
> I'm struggling to get a theano graph spread over two GPU's working, but I 
> keep encountering the GpuArrayException: b'an illegal memory access was 
> encountered' error (full traceback is in the end of this email).
> The basic idea is to do a forward pass through two neural networks, each 
> located on a separate device each and combine the outputs.
> 
> I'm using the latest Theano, libgpuarray and Lasagne to build the networks, 
> and have hacked Lasagne a bit to able to pass target='device' to the shared 
> variable constructor during weights initialization. 
> 
> I have THEANO_FLAGS="contexts=dev1->cuda1;dev2->cuda2" and the output after 
> theano import is:
> Using cuDNN version 5005 on context None 
> Mapped name None to device cuda: GeForce GTX 980 (:0A:00.0) 
> Using cuDNN version 5005 on context dev1 
> Mapped name dev1 to device cuda1: GeForce GTX 980 (:09:00.0) 
> Using cuDNN version 5005 on context dev2 
> Mapped name dev2 to device cuda2: GeForce GTX 980 (:06:00.0)
> 
> 
> The networks definition is quite lengthy (and doesn't always reproduce on 
> toy graphs), so I'm providing a simpified example of what I'm doing. 
> inp_0 = T.tensor4('inp0')
> r0 = build_model('dev1', input_var=inp_0)
> inp_1 = T.tensor4('inp1')
> r1 = build_model("dev2", input_var=inp_1)
> 
> r0_out = lasagne.layers.get_output(r0['fc6'], deterministic=False)
> r1_out = lasagne.layers.get_output(r1['fc6'], deterministic=False)
> 
> train_r0 = theano.function(
> [inp_0, inp_1],
> [r0_out, r1_out]
> )
> 
> result0 = train_r0(x, x2)
> This code fails with the aforementioned error.
> 
> I've also tried to compile a separate function for each of the networks, 
> like
> train_r0 = theano.function(
> [inp_0],
> [r0_out]
> )
> 
> train_r1 = theano.function(
> [inp_1],
> [r1_out]
> )
> 
> And running either train_r0 or train_r1 fails. But compiling and running a 
> single function (no matter train_r0 or train_r1) works just fine.
> Could someone help me debug this? Please let me know if I should provide 
> additional code/info.
> 
> Thanks,
> Sergey.
> 
> The full traceback:
> 
> RuntimeError  Traceback (most recent call last)
> /home/facenx/.virtualenvs/multitheano/lib/python3.5/site-packages/theano/compile/function_module.py
>  in __call__(self, *args, **kwargs)
> 883 outputs =\
> --> 884 self.fn() if output_subset is None else\
> 885 self.fn(output_subset=output_subset)
> 
> RuntimeError: Error in the elemwise call
> 
> During handling of the above exception, another exception occurred:
> 
> GpuArrayException Traceback (most recent call last)
>  in ()
> > 1 result0 = train_r0(x, x2)
>   2 #result1 = train_r1(x2)
> 
> /home/facenx/.virtualenvs/multitheano/lib/python3.5/site-packages/theano/compile/function_module.py
>  in __call__(self, *args, **kwargs)
> 896 node=self.fn.nodes[self.fn.position_of_error],
> 897 thunk=thunk,
> --> 898 storage_map=getattr(self.fn, 'storage_map', None))
> 899 else:
> 900 # old-style linkers raise their own exceptions
> 
> /home/facenx/.virtualenvs/multitheano/lib/python3.5/site-packages/theano/gof/link.py
>  in raise_with_op(node, thunk, exc_info, storage_map)
> 139 
> 140 hints = []
> --> 141 detailed_err_msg = "\nApply node that caused the error: " + 
> str(node)
> 142 if exc_value.__applynode_index__ is not None:
> 143 detailed_err_msg += "\nToposort index: %d" % node_index
> 
> /home/facenx/.virtualenvs/multitheano/lib/python3.5/site-packages/theano/gof/graph.py
>  in __str__(self)
> 178 
> 179 def __str__(self):
> --> 180 return op_as_string(self.inputs, self)
> 181 
> 182 def __repr__(self):
> 
> /home/facenx/.virtualenvs/multitheano/lib/python3.5/site-packages/theano/gof/graph.py
>  in op_as_string(i, op, leaf_formatter, node_formatter)
>1256 between i and o
>1257 """
> -> 1258 strs = as_string(i, op.inputs, leaf_formatter, node_formatter)
>1259 return node_formatter(op, strs)
>1260 
> 
> /home/facenx/.virtualenvs/multitheano/lib/python3.5/site-packages/theano/gof/graph.py
>  in as_string(i, o, leaf_formatter, node_formatter)
>1336 return leaf_formatter(r)
>1337 
> -> 1338 return [describe(output) for output in o]
>1339 
>1340 
> 
> /home/facenx/.virtualenvs/multitheano/lib/python3.5/site-packages/theano/gof/graph.py
>  in (.0)
>1336 return leaf_formatter(r)
>1337 
> -> 1338 return [describe(output) for output in o]
>1339 
>1340 
> 
> /home/facenx/.virtualenvs/multitheano/lib/python3.5/site-packages/theano/gof/graph.py
>  in describe(r)
>1334 

Re: [theano-users] [BUG or FEATURE?] gradient wrt shared after changing shape of storage

[Pascal Lamblin]

The problem is that your test values are no longer consistent with each other.

On Fri, Mar 24, 2017, Maxim Kochurov wrote:
> Hi, I'm trying to get gradient wrt shared variable after setting new value 
> and get an error. Am I doing something wrong?
> Minimal example to play with
> 
> from theano import theano, tensor as tt
> abc = tt.fvector()
> abc.tag.test_value = np.array([1.,2.,3.], dtype='float32')
> X = theano.shared(np.ones((4,3), 'float32'))
> y = tt.dot(X, abc)
> X.set_value(np.ones((1, 3), 'float32'))
> y.sum().eval({abc:np.array([1.,2.,3.], dtype='float32')}) # works
> g = theano.grad(y.sum(), X)
> 
> ...
> 
> ValueError: shapes (3,1) and (4,) not aligned: 1 (dim 1) != 4 (dim 0)
> 
> 
> theano.version 0.9.0.dev-dded8355679b510acf392ce35753cbc7856ba69f
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Error while executing Alexnet with theano v0.9.0

[Pascal Lamblin]

Hi,

You seem to be initializing both the new back-end and the old back-end
at the same time, which is not really supported.

Regarding the NotImplementedError, it is raised when calling T.min()
on something that is neither a signed int nor a float, as it currently
instantiates (-T.max(-x)), which may not be safe.

Here, the argument is the output of T.neq(...), which used to return an
int8, but now returns a bool.

I think the easiest fix for that code would be to use boolean operations
instead of min(), since the equivalent would be all(...).
The following should all work:

  T.mean(T.neq(y_pred_top_x, y_top_x).all(axis=1))
  T.mean(T.min(T.neq(y_pred_top_x, y_top_x).astype('int8'), axis=1))

On Tue, Mar 28, 2017, Deepali Patel wrote:
> Hello,
> 
> While executing alexnet with theano v0.9.0 I observed the following error:
> 
> THEANO_FLAGS=mode=FAST_RUN,floatX=float32 python train.py
> WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be 
> removed in the next release (v0.10).  Please switch to the gpuarray 
> backend. You can get more information about how to switch at this URL:
>  
> https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
> 
> Using cuDNN version 5105 on context None
> Mapped name None to device cuda: Tesla K80 (0004:03:00.0)
> Using gpu device 0: Tesla K80 (CNMeM is disabled, cuDNN 5105)
> ... building the model
> conv (cudnn) layer with shape_in: (3, 227, 227, 128)
> conv (cudnn) layer with shape_in: (96, 27, 27, 128)
> conv (cudnn) layer with shape_in: (256, 13, 13, 128)
> conv (cudnn) layer with shape_in: (384, 13, 13, 128)
> conv (cudnn) layer with shape_in: (384, 13, 13, 128)
> fc layer with num_in: 9216 num_out: 4096
> dropout layer with P_drop: 0.5
> fc layer with num_in: 4096 num_out: 4096
> dropout layer with P_drop: 0.5
> softmax layer with num_in: 4096 num_out: 1000
> Process Process-1:
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in 
> _bootstrap
> self.run()
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
> self._target(*self._args, **self._kwargs)
>   File "train.py", line 54, in train_net
> model = AlexNet(config)
>   File "/alexnet-theano/alex_net.py", line 130, in __init__
> self.errors_top_5 = softmax_layer8.errors_top_x(y, 5)
>   File "./lib/layers.py", line 312, in errors_top_x
> return T.mean(T.min(T.neq(y_pred_top_x, y_top_x), axis=1))
>   File "/site-packages/theano/tensor/basic.py", line 1742, in min
> raise NotImplementedError()
> NotImplementedError
> 
> 
> Please let me know if there is a known solution to this issue.
> 
> Regards
> Deepali 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] pygpu.gpuarray.GpuArrayException: Unknown device error: -1

[Pascal Lamblin]

Are you able to run other CUDA programs on that GPU, for instance devicQuery 
(source included in cuda SDK)?
What does deviceQuery return, vs nvidia-smi?

On Thu, Mar 30, 2017, Øystein Schønning-Johansen wrote:
> 
> 
> Hi all!
> 
> Today I got the 'strange' error message when importing theano. This has 
> worked perfectly before,
> 
> I've built theano and libgpuarray from git sources. I've even recompiled from 
> the today's repo code, but I get the same error every time.
> 
> Any hints?
> 
> 
> Thanks,
> 
> -Øystein
> 
> 
> [oystein@jupiter libgpuarray-git]$ python
> Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
> [GCC 6.3.1 20170109] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import theano
> ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
> Traceback (most recent call last):
>   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 
> 164, in 
> use(config.device)
>   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 
> 151, in use
> init_dev(device)
>   File "/usr/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 
> 60, in init_dev
> sched=config.gpuarray.sched)
>   File "pygpu/gpuarray.pyx", line 614, in pygpu.gpuarray.init 
> (pygpu/gpuarray.c:9419)
>   File "pygpu/gpuarray.pyx", line 566, in pygpu.gpuarray.pygpu_init 
> (pygpu/gpuarray.c:9110)
>   File "pygpu/gpuarray.pyx", line 1021, in 
> pygpu.gpuarray.GpuContext.__cinit__ (pygpu/gpuarray.c:13472)
> pygpu.gpuarray.GpuArrayException: Unknown device error: -1
> >>> 
> 
> [oystein@jupiter libgpuarray-git]$ cat ~/.theanorc
> [global]
> floatX = float32
> device = cuda
> allow_gc = False
> [lib]
> cnmem=1.0
> 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] TypeError: ARange{dtype='int64'}.grad illegally returned an integer-valued variable

[Pascal Lamblin]

Right, that's a bug (https://github.com/Theano/Theano/issues/5790).

However, for some reason `seq` seems to be defined as int64, which I
doubt is what you want since you are taking a gradient wrt it, which
would be 0.

Moreover, the expression for the gradient does not look right. It
currently returns 0 for the boundaries, which is not correct.
It should be 0 for param2 (changing it by epsilon would not change
the output most of the time), and the sum of the output gradients for
param1, at least when step > 0, which is your case.

On Mon, Mar 27, 2017, Julia K wrote:
> Hi,
> 
> I'm getting an error when trying to get the gradient of a function. 
> 
> Basically, I want to use two parameters to define the lower and upper bound 
> of an iterator.
> For easier understanding I simplified my code:
> 
> from theano import theano, tensor as tt
> 
> def get_sum(param1, param2):
> seq = tt.arange(param1, param2)
> 
> components, _ = theano.scan(lambda index: index**2,
>outputs_info=None,
>sequences=[seq])
> return tt.sum(components)
> 
> 
> param1 = tt.dscalar('param1')
> param1.tag.test_value = 0.0
> 
> param2 = tt.dscalar('param2')
> param2.tag.test_value = 10.0
> 
> objective = get_sum(param1, param2)
> 
> obj_function = theano.function(inputs=[param1, param2],outputs=objective)
> 
> obj_gradient = theano.gradient.jacobian(objective,[param1])
> gradient_function = theano.function(inputs=[param1, param2], 
> outputs=obj_gradient)
> 
> ---
> 
> ...
> 
> TypeError: ARange{dtype='int64'}.grad illegally  returned an integer-valued 
> variable. (Input index 0, dtype int64)
> 
> 
> The function that is called in theano.scan (which is index**2 in the 
> example) could be any function that uses the index as one of its parameters.
> 
> What do I need to change about the code to make it work?
> 
> The same error occurs when using theano version 0.8.2 or 
> 0.9.0.dev-b3d19ce8e8d7cdf87c3f84599be81647803aa1f6
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] How to get gradients from Theano in numpy type?

[Pascal Lamblin]

If you enabled the GPU back-end, then shared variables will live on the GPU, 
and the gradients will have the same type.

You have two options at this time:
- either call `.transfer('cpu')` on the symbolic gradients before using them as 
outputs
- call numpy.asarray() on the GpuArrays returned by the Theano function

On Mon, Mar 27, 2017, duy kien nguyen wrote:
> 
> Hi all,
> 
> I tried to get the gradients from Theano in numpy type. But, when I defined 
> the function, it returned pygpu.gpuarray.GpuArray. How can I solve this 
> problem?
> 
> Here is my function:
> 
> cost  = classifier.negative_log_likelihood(answer, answer_mask)
> grads = T.grad(cost, list(classifier.tparams.values()))
> 
> optim = optimizer()
> # updates = optim(itemlist(classifier.tparams), grads)
> 
> train_model = theano.function(
> inputs=[],
> outputs=[cost] + grads,
> # updates=updates,
> givens={
> img: shared_img, ques: shared_ques_casted, ans: shared_ans_casted[:-1],
> m_ques: shared_m_ques, m_ans: shared_m_ans[:-1],
> answer: shared_ans_casted[1:], answer_mask: shared_m_ans[1:]
> }
> )
> 
> test_model = theano.function(
> inputs=[],
> outputs=classifier.samples(length_answer-1),
> givens={
> img: shared_img, ques: shared_ques_casted,
> m_ques: shared_m_ques
> }
> )
> --
> 
> Here is the type of the returned gradients (the first one is in numpy type 
> but the other are not):
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Thank you in advance
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Differentiating w.r.t matrix row results in DisconnectedInputError

[Pascal Lamblin]

You can take a subtensor of the whole gradient instead, it should be 
mathematically equivalent.

It may not get completely optimized, though, and parts of the whole gradient 
may be computed and end up not being used.

On Thu, Mar 30, 2017, Jan Kukačka wrote:
> Dear all,
> 
> I have the following problem: When I want to differentiate a row of an 
> input matrix w.r.t. a row of an output matrix, I get a 
> DisconnectedInputError. The minimum scenario is in the attached file. I 
> suspect theano does not consider theta and theta[i] to be the same 
> variable, rather take theta[i] to be a different variable depending on 
> theta.
> 
> Long story:
> I have a neural network, using M-dimensional input and N-dimensional output 
> and having batch size B. Input matrix for my net is then (B,M) dimensional 
> and the output matrix is (B,N) dimensional. For every batch element I want 
> to compute the jacobbian of the network, that is, M*N matrix with gradients 
> of element outputs depending on the element inputs. In the scan function I 
> can compute jacobbians of individual rows wrt. whole input (thus having B 
> matrices of size (B*M, N) but when I try computing the jacobbian only wrt a 
> row of the input, I get the DisconnectedInputError.
> 
> The question is:
> Is there a way to differentiate w.r.t. a subtensor?
> 
> Best regards,
> Jan
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

> import theano
> from theano import tensor as T
> import numpy as np
> 
> n_samples = 10
> theta_single = T.as_tensor_variable([0.2, 0.15, 0.4, 0.05, 0.2])
> theta = T.tile(theta_single,(n_samples, 1))
> print theta_single.eval().shape
> print theta.eval().shape
> 
> # differentiating with the original theta_single
> J1, _ = theano.scan(lambda i, theta, theta_single : 
> theano.gradient.jacobian(theta[i,:], theta_single),
> sequences=T.arange(theta.shape[0]),
> non_sequences=[theta, theta_single])
> J1.eval()
> 
> # same as before, just differentiating wrt to single row of theta taken from 
> that same array\n",
> J2, _ = theano.scan(lambda i, theta, theta_single : 
> theano.gradient.jacobian(theta[i,:], theta[i,:]),
> sequences=T.arange(theta.shape[0]),
> non_sequences=[theta, theta_single])
> J2.eval()


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Straight-through gradient for stochastic node in theano

[Pascal Lamblin]

You can try OpFromGraph with inline=True, and specify
override_gradients=... with the right expression.
This is still experimental.

On Wed, Mar 29, 2017, nokunok...@gmail.com wrote:
> Hi guys!
> 
> I recently started using theano and am struggling to implement custom 
> gradient for stochastic node. Can anyone help me?
> 
> What I want is an op that produces one hot vector whose hot element is 
> sampled from softmax distribution.
> The op is not differentiable, but I want to "fake" as if its gradient is 
> softmax's one ("straight through estimator").
> Below is the minimum code that perform forward path, which raises 
> DisconnectedInputError due to missing gradient.
> 
> 
> import theano
> 
> import theano.tensor as T
> 
> import numpy as np
> 
> 
> logits_values = np.random.uniform(-1, 1, size=3)
> 
> logits = theano.shared(logits_values, 'logits')
> 
> probabilities = T.nnet.softmax(logits)
> 
> print('probabilities', probabilities.eval())
> 
> # result: probabilities [[ 0.55155489  0.2907730.15767211]]
> 
> 
> random_streams = T.shared_randomstreams.RandomStreams()
> 
> index = random_streams.choice(size=(1,), a=3, p=probabilities[0])
> 
> samples = T.extra_ops.to_one_hot(index, logits.shape[-1])
> 
> print('samples', samples.eval())
> 
> # result: samples [[ 1.  0.  0.]]
> 
> 
> # We want to use gradient of probabilities instead of samples!
> 
> samples_grad = T.grad(samples[0][0], logits)
> 
> # result: raise DisconnectedInputError
> 
> 
> The node is not the final layer, so I can't use categorical cross entropy 
> loss for training it.
> 
> I am trying to implement custom op (see attached stochastic_softmax.py), but 
> it is not working in practice.
> 
> Since I have working expression for forward path, can I simply override 
> gradient of existing expression?
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

> import numpy as np
> import theano
> import theano.tensor as T
> 
> 
> class StochasticSoftmax(theano.Op):
> def __init__(self, random_state=np.random.RandomState()):
> self.random_state = random_state
> 
> def make_node(self, x):
> x = T.as_tensor_variable(x)
> return theano.Apply(self, [x], [x.type()])
> 
> def perform(self, node, inputs, output_storage):
> # Gumbel-max trick
> x, = inputs
> z = self.random_state.gumbel(loc=0, scale=1, size=x.shape)
> indices = (x + z).argmax(axis=-1)
> y = np.eye(x.shape[-1], dtype=np.float32)[indices]
> output_storage[0][0] = y
> 
> def grad(self, inp, grads):
> x, = inp
> g_sm, = grads
> 
> sm = T.nnet.softmax(x)
> return [T.nnet.softmax_grad(g_sm, sm)]
> 
> def infer_shape(self, node, i0_shapes):
> return i0_shapes


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Does the Theano 0.9(the lastest version) support python 2.7?

[Pascal Lamblin]

Python 2.7 is still supported.

On Wed, Mar 29, 2017, xqg mickle wrote:
> Does the new version only support python>=3.3?
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] How to apply Deconv using kernel of multiple channels ?

[Pascal Lamblin]

I'm not sure I understand your question.
Have you read the convolution arithmetic tutorial [1]? It might help.

[1] http://theano.readthedocs.io/en/master/tutorial/conv_arithmetic.html

On Wed, Dec 21, 2016, Feras Almasri wrote:
> What is saw by using conv2d_grad_wrt_inputs *it is to do de**convolution** on 
> feature maps by using a 2D kernel. in the case where you have multiple 
> channels who is this applied? *
> *I did think of using conv2d function with stride and padding but then the 
> idea how to match the assertion between between the input channels and the 
> filter size. If I chose one feature map to deconv and the output should be 
> RGB image ? *
>  
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] how to edit conv function to normalize input for each patch

[Pascal Lamblin]

On Tue, Dec 20, 2016, Eric Huang wrote:
> Thank you very much.
> It is right but seems difficult to achieve...

You could also try images2neibs, then apply normalization, then a dot
product to simulate the convolution.

Although it could be slower, and the gradient may not be implemented in all 
cases.

> 
> 在 2016年12月21日星期三 UTC+8上午5:57:08，Pascal Lamblin写道：
> >
> > Hi, 
> >
> > Not all implementations of convolutions used in Theano can be modified. 
> > For instance, cuDNN ones are defined externally, and we do not have the 
> > source code. 
> >
> > In your case, you could probably copy / edit the CorrMM version, and add 
> > the normalization of patches before the call to im2col. You may actually 
> > have to reverse the role of image and kernel for that. 
> > Also, you would have to redefine the gradients as well. 
> >
> > On Tue, Dec 20, 2016, Eric Huang wrote: 
> > > 
> > > 
> > > I wanna to normalize (substract mean, divide standard deviation) input 
> > in 
> > > each patch during convolution. 
> > > 
> > > 
> > > For example, 
> > > Input: (1,224, 224) 
> > > kernel: (64,5,5) 
> > > stride: 1 
> > > During the calculation of the first feature map, I'd like to do the 
> > > following operations for each position: 
> > > feaMap[0, 0, 0] = conv( (Input[0, 0:5, 0:5] - mean)/std, kernel[0, :, :] 
> > ) 
> > > feaMap[0, 0, 1] = conv( (Input[0, 0:5, 1:6] - mean)/std, kernel[0, :, :] 
> > ) 
> > > feaMap[0, 0, 2] = conv( (Input[0, 0:5, 2:7] - mean)/std, kernel[0, :, :] 
> > ) 
> > > 
> > > ... 
> > > where mean is a fix matrix of shape (1, 5, 5), std is a fix matrix of 
> > shape 
> > > (1, 5, 5). 
> > > 
> > > It is great cost if I write my own code to extract each patch, do 
> > > normalization then conduct convolution. So I just want to edit the conv 
> > > function to add "substract mean", "divide standard deviation" operation. 
> > > But it seems difficult to edit the conv function to achieve my goal. 
> > > 
> > > How can I achieve it ? Any idea? 
> > > 
> > > -- 
> > > 
> > > --- 
> > > You received this message because you are subscribed to the Google 
> > Groups "theano-users" group. 
> > > To unsubscribe from this group and stop receiving emails from it, send 
> > an email to theano-users...@googlegroups.com . 
> > > For more options, visit https://groups.google.com/d/optout. 
> >
> >
> > -- 
> > Pascal 
> >
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] how to edit conv function to normalize input for each patch

[Pascal Lamblin]

Hi,

Not all implementations of convolutions used in Theano can be modified.
For instance, cuDNN ones are defined externally, and we do not have the
source code.

In your case, you could probably copy / edit the CorrMM version, and add
the normalization of patches before the call to im2col. You may actually
have to reverse the role of image and kernel for that.
Also, you would have to redefine the gradients as well.

On Tue, Dec 20, 2016, Eric Huang wrote:
> 
> 
> I wanna to normalize (substract mean, divide standard deviation) input in 
> each patch during convolution.
> 
> 
> For example,
> Input: (1,224, 224)
> kernel: (64,5,5)
> stride: 1
> During the calculation of the first feature map, I'd like to do the 
> following operations for each position:
> feaMap[0, 0, 0] = conv( (Input[0, 0:5, 0:5] - mean)/std, kernel[0, :, :] )
> feaMap[0, 0, 1] = conv( (Input[0, 0:5, 1:6] - mean)/std, kernel[0, :, :] )
> feaMap[0, 0, 2] = conv( (Input[0, 0:5, 2:7] - mean)/std, kernel[0, :, :] )
> 
> ...
> where mean is a fix matrix of shape (1, 5, 5), std is a fix matrix of shape 
> (1, 5, 5).
> 
> It is great cost if I write my own code to extract each patch, do 
> normalization then conduct convolution. So I just want to edit the conv 
> function to add "substract mean", "divide standard deviation" operation.
> But it seems difficult to edit the conv function to achieve my goal.
> 
> How can I achieve it ? Any idea?
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Current best practices for saving compiled functions with updates to shared variables

[Pascal Lamblin]

Pickling the functions should work fine, as long as they are unpickled
on the same kind of hardware (GPUs of the same generation for instance),
and the same version of Theano.

You can then use the function's .copy() method [1] to swap the shared
variable for the local one.

That being said, if your compilation cache is full (which should happen
after the first launch), then locking should not be an issue.

[1] 
http://deeplearning.net/software/theano/library/compile/function.html#theano.compile.function_module.Function.copy

On Wed, Dec 07, 2016, Michael Harradon wrote:
> Hello all,
> 
> I've found a number of threads on this, but most of them are rather old or 
> refer to functions without updates, so I was hoping to check for some new 
> advice.
> 
> I have a function that performs some training via updates that I run on 16 
> different processes running 16 different GPUs - right now it takes about 
> 5-10 minutes to compile for each process, and it happens serially due to 
> locking in the cache directory, so the GPUs come online one-by-one. Is 
> pickling functions currently a recommended practice? If so, how would one 
> connect to the shared variables in the updates? Is it possible to reassign 
> the update variables in the function object to local shared variables?
> 
> Sorry if this is an already answered question!
> 
> Thanks,
> Michael
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Missing Input to Computation graph while using scan

[Pascal Lamblin]

This would be really difficult to help debugging without a full,
runnable script.

The only meaningful difference between train_mb and validate_mb_accuracy
seens to be the output: does self.layers[-1].accuracy(self.y) depend on
any other variable than self.x and self.y?

The missing input looks like an unnamed matrix, that gets reshaped.

On Sat, Dec 10, 2016, Daksh Varshneya wrote:
> Hi, here is a piece of code I have been working on - 
> 
> self.x = T.tensor3("x")
> self.y = T.ivector("y")
> self.sequences = sequences
> # Check this step for general dimshuffle
> if self.sequences:
> self.x_shuffled = self.x.dimshuffle((1,0,2))
> 
> 
> # self.output = T.tensor3()
> self.output,self.scan_updates = 
> theano.scan(self.step,sequences=self.x_shuffled
>  # ,non_sequences=weights
>  ,outputs_info=None)
> 
> def step(self,x):
> init_layer = self.layers[0]
> init_layer.set_inpt(x, self.mini_batch_size)
> for j in range(1, len(self.layers)):
> prev_layer, layer = self.layers[j-1], self.layers[j]
> layer.set_inpt(
> prev_layer.output, self.mini_batch_size)
> self.output = self.layers[-1].output
> return self.output
> 
> 
> # define the (regularized) cost function, symbolic gradients, and updates
> l2_norm_squared = sum([(layer.w**2).sum() for layer in self.layers])
> cost = self.layers[-1].cost(self)+\
>0.5*lmbda*l2_norm_squared/num_training_batches
> grads = T.grad(cost, self.params)
> updates = [(param, param-eta*grad)
>for param, grad in zip(self.params, grads)]
> 
> # define functions to train a mini-batch, and to compute the
> # accuracy in validation and test mini-batches.
> i = T.lscalar() # mini-batch index
> train_mb = theano.function(
> [i], cost,
> updates=updates + self.scan_updates,
> givens={
> self.x:
> training_x[i*self.mini_batch_size: (i+1)*self.mini_batch_size],
> self.y:
> training_y[i*self.mini_batch_size: (i+1)*self.mini_batch_size]
> },on_unused_input='ignore')
> 
> validate_mb_accuracy = theano.function(
> [i], self.layers[-1].accuracy(self.y),
> 
> updates=self.scan_updates,
> 
>givens={
> self.x:
> validation_x[i*self.mini_batch_size: (i+1)*self.mini_batch_size],
> self.y:
> validation_y[i*self.mini_batch_size: (i+1)*self.mini_batch_size]
> },on_unused_input='ignore')
> 
> 
> I have shown only the relevant part of the code here.
> I am working with sequences of data points and hence you can see the step 
> function which i am calling repeatedly using theano.scan. The 'train_mb' 
> function works perfectly fine when I call it. The error is thrown in the 
> validate_mb_accuracy function. 
> The complete error with exception_verbosity=high is :
> 
> theano.gof.fg.MissingInputError: A variable that is an input to the graph 
> >> was neither provided as an input to the function nor given a value. A 
> >> chain 
> >> of variables leading from this input to an output is [ >> matrix)>, Reshape{2}.0, dot.0, Elemwise{add,no_inplace}.0, sigmoid.0, 
> >> Reshape{2}.0, dot.0, Elemwise{add,no_inplace}.0, sigmoid.0, Reshape{2}.0, 
> >> dot.0, Elemwise{add,no_inplace}.0, Softmax.0, argmax, 
> >> Elemwise{eq,no_inplace}.0, Sum{acc_dtype=int64}.0, mean]. This chain may 
> >> not be unique
> >
> > Backtrace when the variable is created:
> >
> >   File "/home/daksh/PycharmProjects/nn/theano_ann_sequences/tester.py", 
> >> line 65, in 
> >
> > net = shallow()
> >
> >   File "/home/daksh/PycharmProjects/nn/theano_ann_sequences/tester.py", 
> >> line 36, in shallow
> >
> > 
> >> FullyConnectedLayer(n_in=100,n_out=10,activation_fn=softmax)],mini_batch_size,sequences
> >>  
> >> = True)
> >
> >   File "/home/daksh/PycharmProjects/nn/theano_ann_sequences/network.py", 
> >> line 35, in __init__
> >
> > ,outputs_info=None)
> >
> >
> Line 35 in the error points to the validation_mb function.
> Any clues on why I am getting this error? I am even recording the updates 
> of the scan and passing them to the theano.function call as updates(This 
> was pointed out as the probable cause in some of the blogs.
> 
> Any help will be appreciated.
> Thanks 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] indexing a list with a tensor scalar variable in theano

[Pascal Lamblin]

Hi,

Types other than regular tensors (with fixed shape along each dimension)
are not supported in scan.

Moreover, the input and output values of the step function of scan must
always have the same shapes across iterations.

Common workarounds include padding and keeping a mask, or a list of lengths.

On Sun, Dec 11, 2016, 马凯 wrote:
> 
> 
> I have a list :
> 
> my_list = [[1,2,3,4], [4,5,6], [1,2,1,2,1,2]]
> 
> 
> I also have a tensor variable:
> 
> a_tensor = theano.tensor.ivector("tensor")
> 
> 
> now I want to use theano.scan to get the corresponding item in the list 
> given the index:
> 
> result, _ = theano.scan(fn=lambda idx, a_list:a_list[idx],
> sequences=[a_tensor], 
> non_sequences=theano.shared(np.array(my_list)))
> 
> 
> but got the error: *TypeError: The generic 'SharedVariable' object is not 
> subscriptable. This shared variable contains a Numpy array with dtype: 
> 'object'. This data type is not currently recognized by Theano tensors: 
> please cast your data into a supported numeric type if you need Theano 
> tensor functionalities. *
> 
> 
> I'm very new to Theano so I'm probably thinking about this problem in the 
> wrong way. I would really appreciate any advice.
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano optimization

[Pascal Lamblin]

On Wed, Dec 07, 2016, 'Adepu Ravi Sankar' via theano-users wrote:
> I see no big improvement even after using MRG_RandomStreams.

Can you try profiling to see what the bottleneck is?

> 
> On Wednesday, December 7, 2016 at 1:28:14 PM UTC+5:30, Pascal Lamblin wrote:
> >
> > If you are running on GPU, use MRG_RandomStreams. 
> >
> > RandomStreams.normal will execute all sampling on CPU using NumPy's rng, 
> > so transfers can kill your speed, but the MRG variant can sample in 
> > parallel on GPU. 
> >
> > On Mon, Dec 05, 2016, 'Adepu Ravi Sankar' via theano-users wrote: 
> > > I have made the following changes to mlp.py theano example. 
> > > 
> > > rv={} 
> > > for i in range(len(gparams)): 
> > > srng=RandomStreams() 
> > > rv[i]=srng.normal(gparams[i].shape) 
> > > gparams[i]=gparams[i]+0.1*rv[i] 
> > > 
> > > it is just adding the noise to the gradient for noisy-gradient 
> > > implementation. 
> > > 
> > > But the code is running very slow after this modification. 
> > > 
> > > Any pointers on this could be of great help. 
> > > 
> > > The entire source code is attached.(Modification is from line 306 to 
> > 310) 
> > > 
> > > Thanks. 
> > > 
> > > -- 
> > > 
> > > --- 
> > > You received this message because you are subscribed to the Google 
> > Groups "theano-users" group. 
> > > To unsubscribe from this group and stop receiving emails from it, send 
> > an email to theano-users...@googlegroups.com . 
> > > For more options, visit https://groups.google.com/d/optout. 
> >
> > > """ 
> > > This tutorial introduces the multilayer perceptron using Theano. 
> > > 
> > >  A multilayer perceptron is a logistic regressor where 
> > > instead of feeding the input to the logistic regression you insert a 
> > > intermediate layer, called the hidden layer, that has a nonlinear 
> > > activation function (usually tanh or sigmoid) . One can use many such 
> > > hidden layers making the architecture deep. The tutorial will also 
> > tackle 
> > > the problem of MNIST digit classification. 
> > > 
> > > .. math:: 
> > > 
> > > f(x) = G( b^{(2)} + W^{(2)}( s( b^{(1)} + W^{(1)} x))), 
> > > 
> > > References: 
> > > 
> > > - textbooks: "Pattern Recognition and Machine Learning" - 
> > >  Christopher M. Bishop, section 5 
> > > 
> > > """ 
> > > 
> > > from __future__ import print_function 
> > > 
> > > __docformat__ = 'restructedtext en' 
> > > 
> > > 
> > > import os 
> > > import sys 
> > > import timeit 
> > > 
> > > import numpy 
> > > 
> > > import theano 
> > > import theano.tensor as T 
> > > 
> > > 
> > > from logistic_sgd import LogisticRegression, load_data 
> > > 
> > > 
> > > # start-snippet-1 
> > > class HiddenLayer(object): 
> > > def __init__(self, rng, input, n_in, n_out, W=None, b=None, 
> > >  activation=T.tanh): 
> > > """ 
> > > Typical hidden layer of a MLP: units are fully-connected and 
> > have 
> > > sigmoidal activation function. Weight matrix W is of shape 
> > (n_in,n_out) 
> > > and the bias vector b is of shape (n_out,). 
> > > 
> > > NOTE : The nonlinearity used here is tanh 
> > > 
> > > Hidden unit activation is given by: tanh(dot(input,W) + b) 
> > > 
> > > :type rng: numpy.random.RandomState 
> > > :param rng: a random number generator used to initialize weights 
> > > 
> > > :type input: theano.tensor.dmatrix 
> > > :param input: a symbolic tensor of shape (n_examples, n_in) 
> > > 
> > > :type n_in: int 
> > > :param n_in: dimensionality of input 
> > > 
> > > :type n_out: int 
> > > :param n_out: number of hidden units 
> > > 
> > > :type activation: theano.Op or function 
> > > :param activation: Non linearity to be applied in the hidden 
> > >layer 
> > > """ 
> > > self.input = input 
> > 

Re: [theano-users] Theano optimization

[Pascal Lamblin]

If you are running on GPU, use MRG_RandomStreams.

RandomStreams.normal will execute all sampling on CPU using NumPy's rng,
so transfers can kill your speed, but the MRG variant can sample in
parallel on GPU.

On Mon, Dec 05, 2016, 'Adepu Ravi Sankar' via theano-users wrote:
> I have made the following changes to mlp.py theano example.
> 
> rv={}
> for i in range(len(gparams)):
> srng=RandomStreams()
> rv[i]=srng.normal(gparams[i].shape)
> gparams[i]=gparams[i]+0.1*rv[i]
> 
> it is just adding the noise to the gradient for noisy-gradient 
> implementation.
> 
> But the code is running very slow after this modification.
> 
> Any pointers on this could be of great help.
> 
> The entire source code is attached.(Modification is from line 306 to 310)
> 
> Thanks.
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

> """
> This tutorial introduces the multilayer perceptron using Theano.
> 
>  A multilayer perceptron is a logistic regressor where
> instead of feeding the input to the logistic regression you insert a
> intermediate layer, called the hidden layer, that has a nonlinear
> activation function (usually tanh or sigmoid) . One can use many such
> hidden layers making the architecture deep. The tutorial will also tackle
> the problem of MNIST digit classification.
> 
> .. math::
> 
> f(x) = G( b^{(2)} + W^{(2)}( s( b^{(1)} + W^{(1)} x))),
> 
> References:
> 
> - textbooks: "Pattern Recognition and Machine Learning" -
>  Christopher M. Bishop, section 5
> 
> """
> 
> from __future__ import print_function
> 
> __docformat__ = 'restructedtext en'
> 
> 
> import os
> import sys
> import timeit
> 
> import numpy
> 
> import theano
> import theano.tensor as T
> 
> 
> from logistic_sgd import LogisticRegression, load_data
> 
> 
> # start-snippet-1
> class HiddenLayer(object):
> def __init__(self, rng, input, n_in, n_out, W=None, b=None,
>  activation=T.tanh):
> """
> Typical hidden layer of a MLP: units are fully-connected and have
> sigmoidal activation function. Weight matrix W is of shape 
> (n_in,n_out)
> and the bias vector b is of shape (n_out,).
> 
> NOTE : The nonlinearity used here is tanh
> 
> Hidden unit activation is given by: tanh(dot(input,W) + b)
> 
> :type rng: numpy.random.RandomState
> :param rng: a random number generator used to initialize weights
> 
> :type input: theano.tensor.dmatrix
> :param input: a symbolic tensor of shape (n_examples, n_in)
> 
> :type n_in: int
> :param n_in: dimensionality of input
> 
> :type n_out: int
> :param n_out: number of hidden units
> 
> :type activation: theano.Op or function
> :param activation: Non linearity to be applied in the hidden
>layer
> """
> self.input = input
> # end-snippet-1
> 
> # `W` is initialized with `W_values` which is uniformely sampled
> # from sqrt(-6./(n_in+n_hidden)) and sqrt(6./(n_in+n_hidden))
> # for tanh activation function
> # the output of uniform if converted using asarray to dtype
> # theano.config.floatX so that the code is runable on GPU
> # Note : optimal initialization of weights is dependent on the
> #activation function used (among other things).
> #For example, results presented in [Xavier10] suggest that you
> #should use 4 times larger initial weights for sigmoid
> #compared to tanh
> #We have no info for other function, so we use the same as
> #tanh.
> if W is None:
> W_values = numpy.asarray(
> rng.uniform(
> low=-numpy.sqrt(6. / (n_in + n_out)),
> high=numpy.sqrt(6. / (n_in + n_out)),
> size=(n_in, n_out)
> ),
> dtype=theano.config.floatX
> )
> if activation == theano.tensor.nnet.sigmoid:
> W_values *= 4
> 
> W = theano.shared(value=W_values, name='W', borrow=True)
> 
> if b is None:
> b_values = numpy.zeros((n_out,), dtype=theano.config.floatX)
> b = theano.shared(value=b_values, name='b', borrow=True)
> 
> self.W = W
> self.b = b
> 
> lin_output = T.dot(input, self.W) + self.b
> self.output = (
> lin_output if activation is None
> else activation(lin_output)
> )
> # parameters of the model
> self.params = [self.W, self.b]
> 
> 
> # start-snippet-2
> class MLP(object):
> 

Re: [theano-users] theano.tensor.extra_ops.to_one_hot(): nb_class how to make as tensor variable

[Pascal Lamblin]

The error message indicates that the index variable (x_index_true) has
to have an integer dtype.

The issue in that case is that its dtype is float64, since mask has been
defined as a dmatrix(). If you define it as imatrix() or lmatrix(), then
it should work.

On Tue, Dec 06, 2016, Lijun Wu wrote:
> But when I try to feed in with M.shape[0], it failed, my code is:
> 
> x = tensor.dmatrix('x')
> mask = tensor.dmatrix('m')
> mask_sum = mask.sum(axis=0)
> mask_sum_gt_1 = tensor.gt(mask_sum, 1)
> x_index= mask.sum - 2
> x_index_true = x_index * mask_sum_gt_1
> one_hot_matrix = tensor.extra_ops.to_one_hot(x_index_true, mask.shape[0])
> 
> then it posted error:
> raise TypeError('index must be integers')
> 
> am I doing anything wrong?
> 
> 
> On Wednesday, December 7, 2016 at 6:47:34 AM UTC+8, Pascal Lamblin wrote:
> >
> > Theano definitely accepts 'nb_class' as a symbolic scalar in to_one_hot(). 
> >
> > >>> a = tensor.ivector() 
> > >>> i = tensor.iscalar() 
> > >>> b = to_one_hot(a, i) 
> > >>> b.eval{a: [3], i: 5}) 
> > array([[ 0.,  0.,  0.,  1.,  0.]]) 
> > >>> b.eval({a: [3], i: 4}) 
> > array([[ 0.,  0.,  0.,  1.]]) 
> >
> >
> > On Tue, Dec 06, 2016, Lijun Wu wrote: 
> > > Hi All, 
> > > 
> > > I want to implement the need of one_hot with variable length, so I want 
> > to 
> > > feed in the nb_class with a tensorVariable, but how to do this? Is there 
> > > any other way? 
> > > 
> > > What my need is following: 
> > > I have matrix A, example: 
> > > [[0.1, 0.2, 0.3] 
> > >  [0.2, 0.1, 0.1] 
> > >  [0.1, 0.2, 0.2]] 
> > > 
> > > and one mask matrix M: 
> > > [[1, 1, 1] 
> > >  [1, 0, 1] 
> > >  [0, 0, 0]] 
> > > 
> > > and I want to get the last one in each column of M, and get the 
> > > corresponding value in A. e.g, here is 
> > > [[0, 0.2, 0] 
> > >  [0.2, 0, 0.1] 
> > >  [0, 0, 0]] 
> > > 
> > > My solution is first get y=M.sum(axis=0), then feed y to create one_hot 
> > > matrix using extra_ops.to_one_hot(), but since my M.shape[0] will be 
> > > different, so I want to feed in np_class as M.shape[0], but I don't know 
> > > how to do this, one_hot() can not feed in 'nb_class' as tensorvariable. 
> > > 
> > > Can anyone help me work on this? Thanks pretty much. 
> > > 
> > > -- 
> > > 
> > > --- 
> > > You received this message because you are subscribed to the Google 
> > Groups "theano-users" group. 
> > > To unsubscribe from this group and stop receiving emails from it, send 
> > an email to theano-users...@googlegroups.com . 
> > > For more options, visit https://groups.google.com/d/optout. 
> >
> >
> > -- 
> > Pascal 
> >
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano code run with gpu produces NaN with nvcc.fastmath = True

[Pascal Lamblin]

Unfortunately, fastmath can do that. One of the possible reasons is
that denormalized numbers are not used, so you can have small non-zero
numbers become zero. It may also be something else that makes your
learning diverge. I would suspect you could get the same issues on CPU
if using -ffast-math for g++.

Out of curiosity, how much of a speed-up do you get using nvcc.fastmath?

On Tue, Dec 06, 2016, Alexander McDowell wrote:
> For some reason when I try to run this 
> 
>  
> code with the gpu with nvcc.fastmath = True, it runs fine, but eventually 
> starts producing NaNs as a loss. It works fine when I run it on cpu but not 
> on the gpu. If I try to run it with nvcc.fastmath = False, it runs 
> perfectly well but the cpu version is considerably faster than the gpu 
> version. Does anyone know why this is?
> 
> GPU result message (with fastmath = True):
> 
> Building Models
> 
> Training Model!
> 
> Training with device = gpu
> 
> Training on iteration #0
> 
> Receiver Training Error: nan. Interceptor Training Error: 1.004785
> 
> Training on iteration #100
> 
> Receiver Training Error: nan. Interceptor Training Error: nan
> 
> 
> ... (keeps going)
> 
> 
> GPU result message (with fastmath = False):
> 
> 
> Using gpu device 0: GeForce GT 650M (CNMeM is disabled, cuDNN not available)
> 
> Building Models
> 
> Training Model!
> 
> Training with device = gpu
> 
> Training on iteration #0
> 
> Receiver Training Error: 0.995444. Interceptor Training Error: 1.002399
> 
> Training on iteration #100
> 
> Receiver Training Error: 0.990433. Interceptor Training Error: 1.002779
> 
> Training on iteration #200
> 
> Receiver Training Error: 0.991761. Interceptor Training Error: 1.000185
> 
> 
> ... (keeps going)
> 
> 
> CPU result message:
> 
> 
> Building Models
> 
> Training Model!
> 
> Training with device = cpu
> 
> Training on iteration #0
> 
> Receiver Training Error: 0.994140. Interceptor Training Error: 1.002878
> 
> Training on iteration #100
> 
> Receiver Training Error: 1.004477. Interceptor Training Error: 0.997820
> 
> Training on iteration #200
> 
> Receiver Training Error: 0.998176. Interceptor Training Error: 1.001941
> 
> 
> ... (keeps going)
> 
> 
> I also have my .theanorc file:
> 
> 
> [global]
> 
> device = gpu
> 
> floatX = float32
> 
> cxx = /Library/Developer/CommandLineTools/usr/bin/clang++
> 
> optimizer=fast_compile
> 
> 
> [blas]
> 
> blas.ldflags=
> 
> 
> [nvcc]
> 
> fastmath = True
> 
> nvcc.flags = -D_FORCE_INLINES
> 
> 
> [cuda]
> 
> root = /usr/local/cuda/
> 
> 
> 
> I also ran the CPU and GPU on the GPU Test program from here 
>  and got 
> the following results:
> 
> GPU (with fastmath = True):
> 
> Using gpu device 0: GeForce GT 650M (CNMeM is disabled, cuDNN not available)
> 
> [GpuElemwise{exp,no_inplace}(), 
> HostFromGpu(GpuElemwise{exp,no_inplace}.0)]
> 
> Looping 1000 times took 0.856593 seconds
> 
> Result is [ 1.23178029  1.61879349  1.52278066 ...,  2.20771813  2.29967761
> 
>   1.62323296]
> 
> Used the gpu
> 
> 
> GPU (with fastmath = False):
> 
> Using gpu device 0: GeForce GT 650M (CNMeM is disabled, cuDNN not available)
> 
> [GpuElemwise{exp,no_inplace}(), 
> HostFromGpu(GpuElemwise{exp,no_inplace}.0)]
> 
> Looping 1000 times took 0.872737 seconds
> 
> Result is [ 1.23178029  1.61879349  1.52278066 ...,  2.20771813  2.29967761
> 
>   1.62323296]
> 
> Used the gpu
> 
> 
> CPU (using .theanorc):
> 
> [Elemwise{exp,no_inplace}()]
> 
> Looping 1000 times took 2.067907 seconds
> 
> Result is [ 1.23178029  1.61879337  1.52278066 ...,  2.20771813  2.29967761
> 
>   1.62323284]
> 
> Used the cpu
> 
> CPU (without .theanorc):
> 
> [Elemwise{exp,no_inplace}()]
> 
> Looping 1000 times took 16.824746 seconds
> 
> Result is [ 1.23178032  1.61879341  1.52278065 ...,  2.20771815  2.29967753
> 
>   1.62323285]
> 
> Used the cpu
> 
> 
> 
> I also have my computer specs if needed:
> 
> Mac OS Sierra, Version 10.12.1
> Processor: 2.9 GHz Intel Core i5
> 
> Memory: 8 GB 1600 MHz DDR3
> 
> Graphics Card: NVIDIA GeForce GT 650M 512 MB
> 
> Thanks in advance!
> - Alexander McDowell
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] lazy evaluation of a convnet with ifelse

[Pascal Lamblin]

It is also possible that the gradient through the non-executed branch
tries to explicitly backpropagate zeros through the convolution.

In that case, an option would be to have an optimization replacing
ConvGradI(zeros(...), w) by zeros(right_shape).

On Tue, Dec 06, 2016, Frédéric Bastien wrote:
> Hi,
> 
> using ifelse do not make sure computation don't get executed. There is
> interaction with another optimization that sometime trigger the execution
> of some/all node in the not executed branch. This happen in particular
> during training.
> 
> I think that you you disable inplace optimization, it should fix that. Can
> you try that? Just use this Theano flag:
> 
> optimizer_excluding=inplace
> 
> For the python execution, this only add an extra overhead. If you don't see
> it in the Theano profiler output, you don't need to take care of this.
> Otherwise, we need to make a C interface for the lazy op, as we don't have
> one now.
> 
> Fred
> 
> On Sun, Dec 4, 2016 at 1:51 AM, Emmanuel Bengio  wrote:
> 
> > Hi everyone,
> >
> > I'm trying to train a deep convnet with residual connections, where some
> > layers are lazily evaluated (and the residual is always forward-propagated).
> >
> > When only doing the forward pass, simply wrapping each layer's output as:
> >   ifelse(cond, conv(x,W) + x, x)
> > works.
> >
> > When doing the backwards pass, things get trickier. I need to wrap the
> > update of W in an ifelse as well:
> >   updates += [(W, ifelse(cond, W - lr*T.grad(W), W)]
> >
> > but it seems simply doing this is not enough. If I look at the
> > profile.op_callcounts(), I still get too many DnnConvs and ConvGradI/W
> > being exectued.
> >
> > If I do the following for the layer output:
> >   ifelse(cond, ifelse(cond, conv(x,W), x) + x, x)
> > now only the right number of DnnConv and ConvGradW are executed.
> > Having taken I peek at ifelse.py, I suspect that is necessary because the
> > optimization that lifts the ifelse is both unaware of convolutions and,
> > most importantly, not activated.
> >
> > Somehow though, all the ConvGradI ops are still being executed.
> > I am basing this on some "minimal" code I made: https://gist.github.com/
> > bengioe/edf82104a391bf54bb8776d8b211e87c
> > With all the ifelse's I'm using this results in:
> > If eval
> > (, 8)
> > (, 16)
> > (, 7)
> > (, 8)
> > If no eval
> > (, 2)
> > (, 10)
> > (, 7)
> > (, 2)
> >
> > There are 6 lazy layers, so this shows the correct number of DnnConv and
> > ConvGradW being run, but the wrong number of ConvGradI.
> >
> > I'm wondering at this point if there's anything to do without delving deep
> > into the ifelse/lazy evaluation code. I'm really not sure what is going on.
> > Please let me know if you have any ideas or suggestions.
> >
> > Thanks!
> >
> > ps: Aside thought: Since IfElse is still in Python, and I am doing lots of
> > call to it, I'm afraid it might slow down computation. I was wondering if
> > there would be a totally different way of doing what I'm trying to do maybe
> > with a radically different computational structure?
> > --
> > Emmanuel Bengio
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "theano-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to theano-users+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> >
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: pow op on cpu instead of gpu

[Pascal Lamblin]

It may be possible that ultra_fast_sigmoid has no implementation on the
GPU, or that the corresponding optimization is not applied for some
reason.

On Tue, Dec 06, 2016, Bogdan Budescu wrote:
> Could it be that when specifying 
> optimizer_including=local_ultra_fast_sigmoid in theano's config flags, the 
> gpu optimization is disabled?
> 
> On Tuesday, December 6, 2016 at 5:40:23 PM UTC+2, Bogdan Budescu wrote:
> >
> > When trying to compile a (rather large) neural net with device=cuda, at 
> > some point I get the following error:
> >
> > ImportError: ('libamdlibm.so: cannot open shared object file: No such file 
> > or directory', '[Elemwise{pow,no_inplace}(, 
> > )]')
> >
> > Now, I don't mind the error itself, as it's probably caused by not running 
> > ldconfig, but this suggests that the optimizer might want to run the op on 
> > cpu instead of gpu, and I assume that this might be the reason for which my 
> > net's training runs so slow (I assume that this also implies some redundant 
> > copies between cpu and gpu memory). I also observe that the training 
> > process takes 100% of the processor (or, rather, of a single core, as 
> > python is not multi-threaded).
> >
> > How can I tell whether there are any ops running on cpu after a successful 
> > compilation (I already set assert_no_cpu_op='raise'), and how can I force 
> > the ops to be executed on gpu instead?
> >
> >
> >
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] theano.tensor.extra_ops.to_one_hot(): nb_class how to make as tensor variable

[Pascal Lamblin]

Theano definitely accepts 'nb_class' as a symbolic scalar in to_one_hot().

>>> a = tensor.ivector()
>>> i = tensor.iscalar()
>>> b = to_one_hot(a, i)
>>> b.eval{a: [3], i: 5})
array([[ 0.,  0.,  0.,  1.,  0.]])
>>> b.eval({a: [3], i: 4})
array([[ 0.,  0.,  0.,  1.]])


On Tue, Dec 06, 2016, Lijun Wu wrote:
> Hi All,
> 
> I want to implement the need of one_hot with variable length, so I want to 
> feed in the nb_class with a tensorVariable, but how to do this? Is there 
> any other way? 
> 
> What my need is following:
> I have matrix A, example:
> [[0.1, 0.2, 0.3]
>  [0.2, 0.1, 0.1]
>  [0.1, 0.2, 0.2]]
> 
> and one mask matrix M:
> [[1, 1, 1]
>  [1, 0, 1]
>  [0, 0, 0]]
> 
> and I want to get the last one in each column of M, and get the 
> corresponding value in A. e.g, here is 
> [[0, 0.2, 0]
>  [0.2, 0, 0.1]
>  [0, 0, 0]]
> 
> My solution is first get y=M.sum(axis=0), then feed y to create one_hot 
> matrix using extra_ops.to_one_hot(), but since my M.shape[0] will be 
> different, so I want to feed in np_class as M.shape[0], but I don't know 
> how to do this, one_hot() can not feed in 'nb_class' as tensorvariable. 
> 
> Can anyone help me work on this? Thanks pretty much. 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: nvcc compiler not found on $PATH but I can run nvcc -V

[Pascal Lamblin]

Hi,

I think Fred meant submitting a pull request that would add a
summary of what is in your previous thread in theano/doc/faq.txt,
so that people looking for that information would find it on
http://deeplearning.net/software/theano/faq.html

On Thu, Dec 01, 2016, Mustg Oplay wrote:
> Hi,
> 
> I bumped the thread.  I'm not sure what you mean by resume and add to 
> Theano doc?
> 
> On Monday, November 28, 2016 at 11:15:37 AM UTC-5, nouiz wrote:
> >
> > Could you resume that and add that to Theano doc? It could be in the file 
> > doc/faq.txt. It happen frequently that people have problems with that.
> >
> > thanks
> >
> > Fred
> >
> > On Sun, Nov 27, 2016 at 10:12 PM, Mustg Oplay  > > wrote:
> >
> >> Hi,
> >>
> >> Search for my post about Pycharm and theano.  I got it to work and also 
> >> had path issues.
> >>
> >> Marc
> >>
> >>
> >> On Sunday, November 20, 2016 at 7:11:28 PM UTC-5, 王烨 wrote:
> >>>
> >>> Dear folks:
> >>> I recently met a question.
> >>>
> >>> I think I did set everything all right in theano. because when I import 
> >>> my theano in python terminal, it shows: 
> >>> Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
> >>> [GCC 5.4.0 20160609] on linux2
> >>> Type "help", "copyright", "credits" or "license" for more information.
> >>> >>> import theano
> >>> Using gpu device 0: GeForce GTX 1060 6GB (CNMeM is enabled with initial 
> >>> size: 80.0% of memory, cuDNN 5105)
> >>>
> >>>
> >>> When I run nvcc -C , it shows:
> >>> ye@ye-desktop:/home$ nvcc -V
> >>> nvcc: NVIDIA (R) Cuda compiler driver
> >>> Copyright (c) 2005-2016 NVIDIA Corporation
> >>> Built on Sun_Sep__4_22:14:01_CDT_2016
> >>> Cuda compilation tools, release 8.0, V8.0.44
> >>>
> >>>
> >>> But when I try to use gensim and theano in Pycharm, It shows:
> >>> ERROR (theano.sandbox.cuda): nvcc compiler not found on $PATH. Check 
> >>> your nvcc installation and try again.
> >>>
> >>> Any suggestions?
> >>>
> >> -- 
> >>
> >> --- 
> >> You received this message because you are subscribed to the Google Groups 
> >> "theano-users" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an 
> >> email to theano-users...@googlegroups.com .
> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> >
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] import theano problem

[Pascal Lamblin]

The error message mentions being out of memory for cnmem.
How much memory do you have on your GPU? How much free memory?
What is the value you gave to lib.cnmem?
Does it still happen if you set lib.cnmem to 0?

On Thu, Dec 01, 2016, Sandipan Haldar wrote:
> *Can somebody help me with this error?*
> 
> 
> 
> >>> import theano
> ERROR (theano.sandbox.cuda): ERROR: Not using GPU. Initialisation of device 
> gpu failed:
> initCnmem: cnmemInit call failed! Reason=CNMEM_STATUS_OUT_OF_MEMORY. 
> numdev=1
> 
> Traceback (most recent call last):
>   File 
> "/home/sandipan/.local/lib/python3.5/site-packages/theano/compile/function_module.py",
>  
> line 859, in __call__
> outputs = self.fn()
> RuntimeError: Cuda error: 
> kernel_reduce_ccontig_node_meb404c8cd39208f6884dd773b584b7d7_0: out of 
> memory. (grid: 1 x 1; block: 256 x 1 x 1)
> 
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "", line 1, in 
>   File 
> "/home/sandipan/.local/lib/python3.5/site-packages/theano/__init__.py", 
> line 111, in 
> theano.sandbox.cuda.tests.test_driver.test_nvidia_driver1()
>   File 
> "/home/sandipan/.local/lib/python3.5/site-packages/theano/sandbox/cuda/tests/test_driver.py",
>  
> line 38, in test_nvidia_driver1
> if not numpy.allclose(f(), a.sum()):
>   File 
> "/home/sandipan/.local/lib/python3.5/site-packages/theano/compile/function_module.py",
>  
> line 871, in __call__
> storage_map=getattr(self.fn, 'storage_map', None))
>   File 
> "/home/sandipan/.local/lib/python3.5/site-packages/theano/gof/link.py", 
> line 314, in raise_with_op
> reraise(exc_type, exc_value, exc_trace)
>   File "/home/sandipan/.local/lib/python3.5/site-packages/six.py", line 
> 685, in reraise
> raise value.with_traceback(tb)
>   File 
> "/home/sandipan/.local/lib/python3.5/site-packages/theano/compile/function_module.py",
>  
> line 859, in __call__
> outputs = self.fn()
> RuntimeError: Cuda error: 
> kernel_reduce_ccontig_node_meb404c8cd39208f6884dd773b584b7d7_0: out of 
> memory. (grid: 1 x 1; block: 256 x 1 x 1)
> 
> Apply node that caused the error: 
> GpuCAReduce{add}{1}()
> Toposort index: 0
> Inputs types: [CudaNdarrayType(float32, vector)]
> Inputs shapes: [(1,)]
> Inputs strides: [(1,)]
> Inputs values: ['not shown']
> Outputs clients: [[HostFromGpu(GpuCAReduce{add}{1}.0)]]
> 
> HINT: Re-running with most Theano optimization disabled could give you a 
> back-trace of when this node was created. This can be done with by setting 
> the Theano flag 'optimizer=fast_compile'. If that does not work, Theano 
> optimizations can be disabled with 'optimizer=None'.
> HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and 
> storage map footprint of this apply node.
> >>> 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] theano beginner problem

[Pascal Lamblin]

Since identity is a numpy array, you should not pass it as an input to 
function().

On Wed, Nov 30, 2016, Mohammed Rashid Chowdhury wrote:
> i was trying to simply compute the dot product of two matrices using theano.
> my code is very simple.
> 
> 
> import theano
> import theano.tensor as T
> import numpy as np
> from theano import function
> 
> def covarience(array):
> 
> input_array=T.matrix('input_array')
> deviation_matrix = T.matrix('deviation_matrix')
> matrix_filled_with_1s=T.matrix('matrix_filled_with_1s')
> z = T.dot(input_array, matrix_filled_with_1s)
> 
> 
> identity=np.ones((len(array),len(array)))
> 
> f=function([array,identity],z)
> # print(f)
> 
> 
> covarience(np.array([[2,4],[6,8]]))
> 
> 
> 
> 
> but the problem is each time i run this code , i get error message like 
> "TypeError: Unknown parameter type: "
> 
> i am kind of lost here. can anyone tell me whats wrong with my code ?. Any 
> kind of help from any one of you would be highly appreciated. 
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Softmax is a classifier?

[Pascal Lamblin]

The softmax layer (softmax(wx + b) is a classifier, that is trained on
the last fully-connected layer, and backpropagates a gradient so that
the rest of the network is trained as well.

SVM is a different classifier, that they connected to the same input
(x, the output of the last fully-connected layer) and that they trained
(without backpropagation I think).

There is sometimes confusion in the literature between the softmax
operation itself (exp(x) / exp(x).sum(), that converts unnormalized
log-probabilities into a probability vector) and the "softmax layer", or
"logistic regression layer" (softmax(Wx+b)).

On Thu, Nov 24, 2016, Beatriz G. wrote:
> Hi Everyone, I am trying to build a cnn based in imagenet. The paper which 
> I am following sais that the architecture is formed by convolutional layers 
> and fully connected layers, and in the last layer, i.e. output layer is 
> followed by softmax. Then, it sais that after extracting the features from 
> the last fully connected layer, uses a SVM as a classifier.
> 
> I do not know if the input of the classifier is the output of the softmax.
> 
> And I thought that the softmax was a classifier, and I must be wrong
> 
> 
> Regards.
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.