The "-march=native" is a good fix depending of the cause. There is 2 causes for this problems:
1) a bug in g++. Then this is a good work around. 2) your home directory is shared on many computer with different CPU. In some cases, our cache should recompile some of the files and sadly don't do it. The fix is to force the cache local to each computer with the flag: base_compiledir=/tmp/... Fred On Tue, Jan 24, 2017 at 3:14 PM, Jack Hessel <[email protected]> wrote: > For reference, it looks like all my CPUs are > > Intel(R) Xeon(R) CPU E5-2665 0 @ 2.40GHz > > so maybe I should try core-avx2 (though I am not sure if this is the > right instruction, either...) > > is > > -march=native > > the accepted solution right now? I've also tried this and am running into > similar errors. > > My g++ is 5.2 > > > On Tuesday, January 24, 2017 at 3:07:29 PM UTC-5, Jack Hessel wrote: >> >> I am trying to use theano on a redhat linux cluster, and I am running >> into a few problems. >> >> Specifically, I am getting an error about an undefined instruction. This >> error happens only sporadically (even on the same machine), which makes >> debugging it even harder. >> >> Here's the exception: >> >> Exception: ('The following error happened while compiling the node', >> Gemm{no_inplace}(dense_1_target\ >> , TensorConstant{1.0}, dense_input_1, dense_1_W, TensorConstant{-1.0}), >> '\n', "Compilation failed (r\ >> eturn status=1): /var/opt/condor/execute/dir_122882/cc4LC9Rp.s: >> Assembler messages:. /var/opt/condor\ >> /execute/dir_122882/cc4LC9Rp.s:2579: Error: no such instruction: `shrx >> %rcx,(%r9),%rdi'. /var/opt/co\ >> ndor/execute/dir_122882/cc4LC9Rp.s:2581: Error: no such instruction: >> `shrx %rcx,8(%r9),%r10'. /var/o\ >> pt/condor/execute/dir_122882/cc4LC9Rp.s:2604: Error: no such >> instruction: `shrx %rdx,(%rcx),%rdx'. /\ >> var/opt/condor/execute/dir_122882/cc4LC9Rp.s:2785: Error: no such >> instruction: `shrx %rcx,(%r9),%rdi\ >> '. /var/opt/condor/execute/dir_122882/cc4LC9Rp.s:2787: Error: no such >> instruction: `shrx %rcx,8(%r9)\ >> ,%r10'. /var/opt/condor/execute/dir_122882/cc4LC9Rp.s:2810: Error: no >> such instruction: `shrx %rdx,(\ >> %rcx),%rdx'. ", '[Gemm{no_inplace}(dense_1_target, TensorConstant{1.0}, >> dense_input_1, dense_1_W, Te\ >> nsorConstant{-1.0})]') >> >> Here's the full compilation line (forgive my ugly paths -- It's a long >> story!) >> >> /home/jmh563/gcc-5.2.0/bin/g++ -shared -g -O3 -fno-math-errno >> -Wno-unused-label -Wno-unused-variable -Wno-write-strings >> -Wl,-rpath,/home/jmh563/otherInstalls/OpenBLAS/lib -march=haswell -mmmx >> -mno-3dnow \ >> -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes >> -mno-sha -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi >> -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm\ >> -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx >> -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd >> -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsa\ >> ves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma >> -mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx --param l1-cache-size=32 >> --param l1-cache-line-size=64 --param l2-cache-size=20480 -mtune=\ >> haswell -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC >> -I/home/jmh563/curPythonInstall/lib/python2.7/site-packages/ >> numpy-1.13.0.dev0+1c8ecc7-py2.7-linux-x86_64.egg/numpy/core/include >> -I/home/jmh5\ >> 63/curPythonInstall/include/python2.7 -I/home/jmh563/curPythonInstal >> l/lib/python2.7/site-packages/Theano-0.9.0.dev5-py2.7.egg/theano/gof >> -L/home/jmh563/otherInstalls/OpenBLAS/lib -L/home/jmh563/curPython\ >> Install/lib -fvisibility=hidden -o /var/tmp/theano-360142/compile >> dir_Linux-2.6-el6.x86_64-x86_64-with-redhat-6.6-Santiago- >> x86_64-2.7.13-64/tmpnoStZa/1ef31f0e43a0cb34a58a9d31cef1eb8d.so >> /var/tmp/theano-36\ >> 0142/compiledir_Linux-2.6-el6.x86_64-x86_64-with-redhat-6.6- >> Santiago-x86_64-2.7.13-64/tmpnoStZa/mod.cpp -lopenblas -lpython2.7 >> >> and here's my theanorc >> >> [global] >> device = cpu >> floatX = float32 >> cxx = /home/jmh563/gcc-5.2.0/bin/g++ >> gcc.cxxflags = '-march=core2' >> >> (I tried to add -march=core2 because I read in a PR somewhere that that >> would work) >> >> I have done some googling and this seems to be a relatively common >> problem for folks. Is there an accepted set of steps to address this sort >> of thing? Or just try out a bunch of different settings of gcc.cxxflags? >> >> Jack >> > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
