On Tue, 21 Aug 2018, 00:22 Alvarez, Damian, <[email protected]> wrote:
> The MPI results are interesting. But I don’t understand why one runtime > would perform better in one processor than in another. I mean, MPI is a > communication runtime, the performance should be limited by algorithms and > IO subsystem, not by the microarchitecture. Could it be that the pinning > was wrongly set by Intel MPI in the EPYC processor? That would make more > sense to me. > At the time I did think of that and didn't see anything obviously wrong with the process placement, but I agree, that would be the simplest explanation (apart from me having messed up somehow :) ) Btw, going back to Kenneth's original question, the Intel compilers also worked fine for me without any tweaks, with small improvements vs GNU compilers (there was one huge improvement but I think it was because ifort replaced matmul with calls to mkl and I wasn't able to get gfortran's -fexternal-blas to do the same). In any case, the main question for me is still the low FFTW performance, and if this is specific to my test or more general... Cheers, Miguel > > Cheers, > > Damian > > > > *From: *<[email protected]> on behalf of Miguel Costa < > [email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Sunday, 19. August 2018 at 17:02 > *To: *"[email protected]" <[email protected]> > *Subject: *Re: [easybuild] experiences with EasyBuild on AMD Epyc? > > > > Well, it's Sunday and late so take this with a grain of salt but on a real > application benchmark on epyc, and the only difference being using gomkl > (with and without AVX2) or foss toolchain: > > > > - without using MKL_DEBUG_CPU_TYPE (so, MKL using *AVX*, not AVX2) > > > > - the FFT dominated parts are ~1.6x faster with *gomkl* than with foss > > > > - but the linear algebra dominated parts are ~1.3x faster with *foss* > than with gomkl. > > > > - using MKL_DEBUG_CPU_TYPE=5, MKL does use *AVX2*, and now > > > > - the FFT dominated parts are ~1.9x faster with *gomkl* than with foss > > > > - the linear algebra dominated parts are ~1.4x faster with *gomkl* than > with foss (so almost 2x compared to gomkl with AVX, as expected) > > > > So Intel MKL seems to not only run fine on epyc but also to be the best > solution (are we missing any optimizations in the foss toolchain on epyc, > especially FFTW?) > > > > (Intel MPI, on the other hand, does not seem to run fine on epyc. I had > tried first gimkl instead of gomkl but while single-core performance was > better than foss, multicore performance was much worse, and that issue > disappeared with gomkl, using OpenMPI) > > > > My two (sing)cents, > > Miguel > > > > > > On Fri, Aug 17, 2018 at 4:55 AM Mikael Öhman <[email protected]> wrote: > > We bought a single epyc node for testing, but we only reused our existing > library (built on haswell with similar instruction set). > > > > At the last EB user meeting I got recommended to use the undocumented > MKL_DEBUG_CPU_TYPE with MKL in order to force it to use AVX2. > > As one might expect, the binaries build for haswell couldn't run, because > intel "helpfully" puts a block that forces a cpu check before starting > whenever you compile with -xHost. > > The VASP code, which we had only compiled with -xavx, ran fine. > > > > All the foss code I have tested seemed to run fine. We had a PhD student > benchmark OpenFOAM for us, and using the same binaries (as compiled for > Intel 2650v3), it ran ~ twice as fast on the 2x16 core EPYC node than on a > 2x10 core 2650v3 node. > > > > Best regards, Mikael > > > > > > On Thu, Aug 16, 2018 at 2:38 PM Kenneth Hoste <[email protected]> > wrote: > > Dear EasyBuilders, > > Does anyone here have experience with using EasyBuild on AMD Epyc systems? > > Do the common toolchains (foss/2018a, intel/2018a or older) work out of > the box, or did you have to make some tweaks? > In particular, did the default of compiling with -xHost with the Intel > compilers work fine? > > In addition: any experiences on how the performance compares to recent > Intel systems for particular applications? > > > regards, > > Kenneth > > > > > ------------------------------------------------------------------------------------------------ > > ------------------------------------------------------------------------------------------------ > Forschungszentrum Juelich GmbH > 52425 Juelich > Sitz der Gesellschaft: Juelich > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher > Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), > Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, > Prof. Dr. Sebastian M. Schmidt > > ------------------------------------------------------------------------------------------------ > > ------------------------------------------------------------------------------------------------ > >
