Well, it's Sunday and late so take this with a grain of salt but on a real application benchmark on epyc, and the only difference being using gomkl (with and without AVX2) or foss toolchain:
- without using MKL_DEBUG_CPU_TYPE (so, MKL using *AVX*, not AVX2) - the FFT dominated parts are ~1.6x faster with *gomkl* than with foss - but the linear algebra dominated parts are ~1.3x faster with *foss* than with gomkl. - using MKL_DEBUG_CPU_TYPE=5, MKL does use *AVX2*, and now - the FFT dominated parts are ~1.9x faster with *gomkl* than with foss - the linear algebra dominated parts are ~1.4x faster with *gomkl* than with foss (so almost 2x compared to gomkl with AVX, as expected) So Intel MKL seems to not only run fine on epyc but also to be the best solution (are we missing any optimizations in the foss toolchain on epyc, especially FFTW?) (Intel MPI, on the other hand, does not seem to run fine on epyc. I had tried first gimkl instead of gomkl but while single-core performance was better than foss, multicore performance was much worse, and that issue disappeared with gomkl, using OpenMPI) My two (sing)cents, Miguel On Fri, Aug 17, 2018 at 4:55 AM Mikael Öhman <[email protected]> wrote: > We bought a single epyc node for testing, but we only reused our existing > library (built on haswell with similar instruction set). > > At the last EB user meeting I got recommended to use the undocumented > MKL_DEBUG_CPU_TYPE with MKL in order to force it to use AVX2. > As one might expect, the binaries build for haswell couldn't run, because > intel "helpfully" puts a block that forces a cpu check before starting > whenever you compile with -xHost. > The VASP code, which we had only compiled with -xavx, ran fine. > > All the foss code I have tested seemed to run fine. We had a PhD student > benchmark OpenFOAM for us, and using the same binaries (as compiled for > Intel 2650v3), it ran ~ twice as fast on the 2x16 core EPYC node than on a > 2x10 core 2650v3 node. > > Best regards, Mikael > > > On Thu, Aug 16, 2018 at 2:38 PM Kenneth Hoste <[email protected]> > wrote: > >> Dear EasyBuilders, >> >> Does anyone here have experience with using EasyBuild on AMD Epyc systems? >> >> Do the common toolchains (foss/2018a, intel/2018a or older) work out of >> the box, or did you have to make some tweaks? >> In particular, did the default of compiling with -xHost with the Intel >> compilers work fine? >> >> In addition: any experiences on how the performance compares to recent >> Intel systems for particular applications? >> >> >> regards, >> >> Kenneth >> >
