the message about ".... Access is denied ..." occurs on both machine, even when the scripts work.
the workaround is to use a different sub-folder for each script inside the AppData -> thanks so much for pointing to this solution. It will be beneficial if this finding can be added to Theano documentation - it is very helpful for those - like me - who live in windows. On Tuesday, May 8, 2018 at 2:43:39 PM UTC-5, Pascal Lamblin wrote: > > The new error is: > > Failed to run "C:/Temp/Anaconda3/Library/mingw-w64/bin" (Access is > denied. > >). > > Which may be consistent with the OS being stricter on security on the > laptop. > > In any case, did I understand correctly that indicating a specific > subdirectory inside AppData works for both versions? > If so, is that enough to let you run different jobs with different cache > directories? > > On 2018-05-08 10:30 AM, [email protected] <javascript:> wrote: > > I am using two machines: a laptop and a desktop for testing Theano > > installation. Both machines have windows 7 and it seems to me that they > > were configured identically (including antivirus) but I will check with > > IT people. > > > > I decided to do the following: remove anaconda and related files from > > both machines, and install anaconda 5.1 - to have a fresh start. Note > > that in the laptop, I installed Theano 0.9.0. While on the desktop, I > > installed Theano 1.0.1 > > > > Then, I conducted these tests: > > > > A) Both laptop and desktop: os.environ["THEANO_FLAGS"]= > > > r"base_compiledir=C:\\Users\\IMarroquin\\AppData\\local\\theano\\Model_1" > > > > -> It worked fine on both machines > > > > B) Both laptop and desktop: os.environ["THEANO_FLAGS"]= > > r"base_compiledir=Model_1" > > > > -> on the laptop: it worked fine. The folder "Model_1" got created in > > the directory where the Python script is located. See file > > "Laptop_message.txt" for a listing of the last lines before the script > > started to run. > > > > -> on the desktop: it failed with the same issue, missing pyd file. See > > the compilation error file > > > > > > Hope this helps! > > > > > > > > > > On Monday, May 7, 2018 at 9:51:37 PM UTC-5, Pascal Lamblin wrote: > > > > It may be another Windows protection mechanism that prevents the > > creation of that .pyd file here, but allows it in the default > > compiledir. I can understand why such restrictions could exist, > since > > it's a binary created by another program. > > > > If running the command manually gives the same error, then it does > not > > seem to be specific to Theano, even though it is weird that version > 0.9 > > would work. > > > > Two things you could try in order to investigate: > > - Use a subdirectory of the _default_ base_compiledir, with the full > > path. For instance, something like > > "C:\\Users\\IMarroquin\\AppData\\local\\theano\\Model_1" > > - Get the full compilation command line generated by Theano 0.9 (the > > one > > that works), and post both, so we can compare. > > > > > > On 2018-05-04 08:43 AM, [email protected] <javascript:> wrote: > > > Hi Pascal, > > > > > > I tried the full path with your suggestion. This time, windows > > didn't > > > compliant about the length of the filename and the sub-folder > > "Model_1" > > > got created. Unfortunately, I still got the same issue about the > > missing > > > *.pyd file (I attached again the compilation error file) > > > > > > Then, I ran the command (the version in the attached file) on a > > conda > > > window. I got the following output: > > > > > > > > > C:/Temp/Anaconda3/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: > > > > > > > cannot open output file > > > C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_c > > > > > > ontest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmpyuw3l9gk\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd: > > > > > > > No such file or directory > > > collect2.exe: error: ld returned 1 exit status > > > > > > Hope this information helps, > > > > > > Ivan > > > > > > > > > On Thursday, May 3, 2018 at 5:32:20 PM UTC-5, Pascal Lamblin > wrote: > > > > > > On 2018-05-03 04:50 PM, [email protected] <javascript:> > > wrote: > > > > Thanks for the suggestion. > > > > > > > > I checked the disk space and have ~700Gbytes. Then, I > > tested with > > > > "base_compiledir" using the entire path. > > > > > > > > This time, a sub-folder with this name was created: > > > > > > > > > > UsersIMarroquinDocumentsMy_Python_ScriptsMLPSEG_contestIndependent_ScriptsModel_1 > > > > > > > > > > > > > backslashes may have not been escaped enough, maybe with > > > os.environ["THEANO_FLAGS"] = r"basecompiledir=C:\\Users\\ > ..." > > > > > > > I am running the command: python <script.py> from a conda > > window. I > > > > attached the compilation error by Theano and the problem > > occurs > > > when it > > > > tries to open the *.pyd file. > > > > > > OK, the log file says the attempt command was: > > > > "C:\Temp\Anaconda3\Library\mingw-w64\bin\g++.exe" -shared > > -g -O3 > > > -fno-math-errno -Wno-unused-label -Wno-unused-variable > > > -Wno-write-strings -march=haswell -mmmx -mno-3dnow -msse > > -msse2 -msse3 > > > -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha > -mpclmul > > > -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 > > -mno-tbm > > > -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle > > -mrdrnd -mf16c > > > -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave > > -mxsaveopt > > > -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf > > -mno-prefetchwt1 > > > -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq > > -mno-avx512bw > > > -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb > > -mno-pcommit > > > -mno-mwaitx --param l1-cache-size=32 --param > > l1-cache-line-size=64 > > > --param l2-cache-size=25600 -mtune=haswell > > > -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 > > > -I"C:\Temp\Anaconda3\lib\site-packages\numpy\core\include" > > > -I"C:\Temp\Anaconda3\include" > > > -I"C:\Temp\Anaconda3\lib\site-packages\theano\gof\c_code" > > > -L"C:\Temp\Anaconda3\libs" -L"C:\Temp\Anaconda3" -o > > > > > > "C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd" > > > > > > > > > > > > > "C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\mod.cpp" > > > > > > > > > > -lpython36 > > > > > > The name of the "tmp..." directory changed, but that is > > expected (it is > > > randomly generated each time). > > > The file name for the .pyd is the same as before, since it > > only depends > > > on the file's content. > > > > > > Can you try running that command line from the conda window? > > > > > > > > > > > Also, I tried with this > command os.environ["THEANO_FLAGS"]= > > > > 'base_compiledir=Model_1' and unfortunately, I ended with > > the same > > > > problem (see attached file). > > > > > > > > It is interesting that it used to work on Theano 0.9.0 > > > > > > > > I am hoping that you have other suggestions. > > > > > > > > Thanks for all, > > > > Ivan > > > > > > > > > > > > > > > > On Thursday, May 3, 2018 at 3:01:29 PM UTC-5, Pascal > > Lamblin wrote: > > > > > > > > On 2018-05-03 02:00 PM, [email protected] > > <javascript:> > > > wrote:> > > > > A) Added the > > > > option nocleanup=True to the .theanorc file > > > > > > > > Thanks. This makes sure the "mod.c" file you mention > > later is > > > not > > > > deleted. > > > > > > > > > B) The backslash before "Model 1". In my previous > > attempts > > > in Theano > > > > > 0.9.0, this was the way that I found to make it > work. > > > > > > > > We have added quotation marks and such in order to > better > > > handle paths > > > > on Windows since 0.9, so I'm not surprised. > > > > > > > > > Following your suggestion, I edited the header of > the > > > script to > > > > > this: os.environ["THEANO_FLAGS"]= > > "base_compiledir=Model_1" > > > > > > > > > > and ran the script. This time, I noticed an extra > > > sub-folder under > > > > > > > > > > > > > > > "Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64". > > > > > > > > > > > > > > > > The sub-folder is named "tmpgqmnjspd", which only > > contains a > > > > mod.c file > > > > > > > > That makes sense. > > > > > > > > > The compilation ended with this error: > > > > > > > > > > Exception: ('The following error happened while > > compiling > > > the node', > > > > > InplaceDimShuffle{x,0}(layer:tanh-1/bias), '\n', > > > 'Compilation failed > > > > > (return status=1): > > > C:/Temp/Anaconda3/Library/mingw-w64/bin/../ > > > > > > > > > > > > > > > lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: > > > > > > > > > > > > > > cannot open output file > > > > > > > > > > > > > > > C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1\ > > > > > > > > > > > > > > > > > > > > > > > > > > \compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\\tmpgqmnjspd\\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd: > > > > > > > > > > > > > > > > No such file or directory\r. collect2.exe: error: > ld > > > returned 1 exit > > > > > status\r. ', > > '[InplaceDimShuffle{x,0}(layer:tanh-1/bias)]') > > > > > > > > > > It seems that the source of the problem is a > > missing file > > > with > > > > extension > > > > > *.pyd > > > > > > > > From my understanding, the process is not able to > > open the > > > *output* > > > > file (the .pyd) in order to *write* to it, and so the > > > compilation > > > > cannot > > > > proceed. > > > > > > > > I find that surprising, since it was able to create > > the mod.c > > > file. > > > > Maybe it's a quota problem, or not enough free disk > > space? > > > > > > > > Can you specify a full path, for instance: > > > > > > > > os.environ["THEANO_FLAGS"] = > > > > > > > > > > "base_compiledir=C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1" > > > > > > > > > > > > > > > > > > > Another thing to try would be to put a breakpoint or > > try to > > > print the > > > > compilation command line that is executed, and try to > > execute it > > > > directly from a shell (and the same working > directory), > > > rather than > > > > inside Theano. > > > > > > > > > > > > > > > > > > > > > > C) "compiler" instead of "compiledir". Sorry for > the > > > confusion, I > > > > made a > > > > > mistake during the copy/paste > > > > > > > > > > Many thanks, > > > > > Ivan > > > > > > > > > > -- > > > > > > > > > > --- > > > > > You received this message because you are > > subscribed to > > > the Google > > > > > Groups "theano-users" group. > > > > > To unsubscribe from this group and stop receiving > > emails > > > from it, > > > > send > > > > > an email to [email protected] > > <javascript:> > > > > > <mailto:[email protected] > <javascript:> > > <javascript:> > > > <javascript:> <javascript:>>. > > > > > For more options, visit > > https://groups.google.com/d/optout < > https://groups.google.com/d/optout> > > > <https://groups.google.com/d/optout > > <https://groups.google.com/d/optout>> > > > > <https://groups.google.com/d/optout > > <https://groups.google.com/d/optout> > > > <https://groups.google.com/d/optout > > <https://groups.google.com/d/optout>>>. > > > > > > > > -- > > > > Pascal Lamblin > > > > > > > > -- > > > > > > > > --- > > > > You received this message because you are subscribed to > > the Google > > > > Groups "theano-users" group. > > > > To unsubscribe from this group and stop receiving emails > > from it, > > > send > > > > an email to [email protected] > <javascript:> > > > > <mailto:[email protected] > <javascript:> > > <javascript:> <javascript:>>. > > > > For more options, visit https://groups.google.com/d/optout > > <https://groups.google.com/d/optout> > > > <https://groups.google.com/d/optout > > <https://groups.google.com/d/optout>>. > > > > > > -- > > > Pascal Lamblin > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the > Google > > > Groups "theano-users" group. > > > To unsubscribe from this group and stop receiving emails from it, > > send > > > an email to [email protected] <javascript:> > > > <mailto:[email protected] <javascript:> > <javascript:>>. > > > For more options, visit https://groups.google.com/d/optout > > <https://groups.google.com/d/optout>. > > > > -- > > Pascal Lamblin > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > > Groups "theano-users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [email protected] <javascript:> > > <mailto:[email protected] <javascript:>>. > > For more options, visit https://groups.google.com/d/optout. > > -- > Pascal Lamblin > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
