It may be another Windows protection mechanism that prevents the creation of that .pyd file here, but allows it in the default compiledir. I can understand why such restrictions could exist, since it's a binary created by another program.

If running the command manually gives the same error, then it does not seem to be specific to Theano, even though it is weird that version 0.9 would work.

Two things you could try in order to investigate:
- Use a subdirectory of the _default_ base_compiledir, with the full path. For instance, something like "C:\\Users\\IMarroquin\\AppData\\local\\theano\\Model_1" - Get the full compilation command line generated by Theano 0.9 (the one that works), and post both, so we can compare.


On 2018-05-04 08:43 AM, ivan.d.marroq...@gmail.com wrote:
Hi Pascal,

I tried the full path with your suggestion. This time, windows didn't compliant about the length of the filename and the sub-folder "Model_1" got created. Unfortunately, I still got the same issue about the missing *.pyd file (I attached again the compilation error file)

Then, I ran the command (the version in the attached file) on a conda window. I got the following output:

C:/Temp/Anaconda3/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot open output file C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_c ontest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmpyuw3l9gk\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd: No such file or directory
collect2.exe: error: ld returned 1 exit status

Hope this information  helps,

Ivan


On Thursday, May 3, 2018 at 5:32:20 PM UTC-5, Pascal Lamblin wrote:

    On 2018-05-03 04:50 PM, ivan.d.m...@gmail.com <javascript:> wrote:
     > Thanks for the suggestion.
     >
     > I checked the disk space and have ~700Gbytes. Then, I tested with
     > "base_compiledir" using the entire path.
     >
     > This time,  a sub-folder with this name was created:
     >
    
UsersIMarroquinDocumentsMy_Python_ScriptsMLPSEG_contestIndependent_ScriptsModel_1


    backslashes may have not been escaped enough, maybe with
    os.environ["THEANO_FLAGS"] = r"basecompiledir=C:\\Users\\ ..."

     > I am running the command: python <script.py> from a conda window. I
     > attached the compilation error by Theano and the problem occurs
    when it
     > tries to open the *.pyd file.

    OK, the log file says the attempt command was:
      > "C:\Temp\Anaconda3\Library\mingw-w64\bin\g++.exe" -shared -g -O3
    -fno-math-errno -Wno-unused-label -Wno-unused-variable
    -Wno-write-strings -march=haswell -mmmx -mno-3dnow -msse -msse2 -msse3
    -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul
    -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm
    -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c
    -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt
    -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1
    -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw
    -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit
    -mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64
    --param l2-cache-size=25600 -mtune=haswell
    -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64
    -I"C:\Temp\Anaconda3\lib\site-packages\numpy\core\include"
    -I"C:\Temp\Anaconda3\include"
    -I"C:\Temp\Anaconda3\lib\site-packages\theano\gof\c_code"
    -L"C:\Temp\Anaconda3\libs" -L"C:\Temp\Anaconda3" -o
    
"C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd"

    
"C:\Users\IMarroquin\Documents\My_Python_Scripts\MLP\SEG_contest\Independent_Scripts\Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\tmp1vufnfg1\mod.cpp"

    -lpython36

    The name of the "tmp..." directory changed, but that is expected (it is
    randomly generated each time).
    The file name for the .pyd is the same as before, since it only depends
    on the file's content.

    Can you try running that command line from the conda window?

     >
     > Also, I tried with this command os.environ["THEANO_FLAGS"]=
     > 'base_compiledir=Model_1' and unfortunately, I ended with the same
     > problem (see attached file).
     >
     > It is interesting that it used to work on Theano 0.9.0
     >
     > I am hoping that you have other suggestions.
     >
     > Thanks for all,
     > Ivan
     >
     >
     >
     > On Thursday, May 3, 2018 at 3:01:29 PM UTC-5, Pascal Lamblin wrote:
     >
     >     On 2018-05-03 02:00 PM, ivan.d.m...@gmail.com <javascript:>
    wrote:>
     >     A) Added the
     >     option nocleanup=True to the .theanorc file
     >
     >     Thanks. This makes sure the "mod.c" file you mention later is
    not
     >     deleted.
     >
     >      > B) The backslash before "Model 1". In my previous attempts
    in Theano
     >      > 0.9.0, this was the way that I found to make it work.
     >
     >     We have added quotation marks and such in order to better
    handle paths
     >     on Windows since 0.9, so I'm not surprised.
     >
     >      > Following your suggestion, I edited the header of the
    script to
     >      > this: os.environ["THEANO_FLAGS"]= "base_compiledir=Model_1"
     >      >
     >      > and ran the script. This time, I noticed an extra
    sub-folder under
     >      >
> "Model_1\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64".

     >
     >      > The sub-folder is named "tmpgqmnjspd", which only contains a
     >     mod.c file
     >
     >     That makes sense.
     >
     >      > The compilation ended with this error:
     >      >
     >      > Exception: ('The following error happened while compiling
    the node',
     >      > InplaceDimShuffle{x,0}(layer:tanh-1/bias), '\n',
    'Compilation failed
     >      > (return status=1):
    C:/Temp/Anaconda3/Library/mingw-w64/bin/../
     >      >
> lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:

     >
     >      > cannot open output file
     >      >
> C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1\

     >
     >      >
> \compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_63_Stepping_2_GenuineIntel-3.6.1-64\\tmpgqmnjspd\\m885ff006a95d626dac547a7bdfdb471bbf058622ece2b4435e42316c4012ea56.pyd:

     >
     >      > No such file or directory\r. collect2.exe: error: ld
    returned 1 exit
     >      > status\r. ', '[InplaceDimShuffle{x,0}(layer:tanh-1/bias)]')
     >      >
     >      > It seems that the source of the problem is a missing file
    with
     >     extension
     >      > *.pyd
     >
     >       From my understanding, the process is not able to open the
    *output*
     >     file (the .pyd) in order to *write* to it, and so the
    compilation
     >     cannot
     >     proceed.
     >
     >     I find that surprising, since it was able to create the mod.c
    file.
     >     Maybe it's a quota problem, or not enough free disk space?
     >
     >     Can you specify a full path, for instance:
     >
     >     os.environ["THEANO_FLAGS"] =
> "base_compiledir=C:\\Users\\IMarroquin\\Documents\\My_Python_Scripts\\MLP\\SEG_contest\\Independent_Scripts\\Model_1"

     >
     >
     >     Another thing to try would be to put a breakpoint or try to
    print the
     >     compilation command line that is executed, and try to execute it
     >     directly from a shell (and the same working directory),
    rather than
     >     inside Theano.
     >
     >
     >
     >      >
     >      > C) "compiler" instead of "compiledir". Sorry for the
    confusion, I
     >     made a
     >      > mistake during the copy/paste
     >      >
     >      > Many thanks,
     >      > Ivan
     >      >
     >      > --
     >      >
     >      > ---
     >      > You received this message because you are subscribed to
    the Google
     >      > Groups "theano-users" group.
     >      > To unsubscribe from this group and stop receiving emails
    from it,
     >     send
     >      > an email to theano-users...@googlegroups.com <javascript:>
     >      > <mailto:theano-users+unsubscr...@googlegroups.com
    <javascript:> <javascript:>>.
     >      > For more options, visit https://groups.google.com/d/optout
    <https://groups.google.com/d/optout>
     >     <https://groups.google.com/d/optout
    <https://groups.google.com/d/optout>>.
     >
     >     --
     >     Pascal Lamblin
     >
     > --
     >
     > ---
     > You received this message because you are subscribed to the Google
     > Groups "theano-users" group.
     > To unsubscribe from this group and stop receiving emails from it,
    send
     > an email to theano-users...@googlegroups.com <javascript:>
     > <mailto:theano-users+unsubscr...@googlegroups.com <javascript:>>.
     > For more options, visit https://groups.google.com/d/optout
    <https://groups.google.com/d/optout>.

-- Pascal Lamblin

--

---
You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com <mailto:theano-users+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.

--
Pascal Lamblin

--

--- You received this message because you are subscribed to the Google Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to