Hi, On the underlying issue of instructions for FFTW, I have found this (http://www.fftw.org/fftw3_doc/Installation-on-Unix.html#Installation-on-Unix):
--enable-sse (single precision), --enable-sse2 (single, double), --enable-avx (single, double), --enable-avx2 (single, double), --enable-avx512 (single, double), --enable-avx-128-fma, --enable-kcvi (single), --enable-altivec (single), --enable-vsx (single, double), --enable-neon (single, double on aarch64), --enable-generic-simd128, and --enable-generic-simd256: Enable various SIMD instruction sets. You need compiler that supports the given SIMD extensions, but FFTW will try to detect at runtime whether the CPU supports these extensions. That is, you can compile with--enable-avx and the code will still run on a CPU without AVX support. * - These options require a compiler supporting SIMD extensions, and compiler support is always a bit flaky: see the FFTW FAQ for a list of compiler versions that have problems compiling FFTW. The way I read this, we should be able to move the flag to avx2 for modern Haswells and the code should not throw an “illegal instruction” error on earlier CPUs. Any experiences/comments? Any one with an SSE2 system likes to try? Best wishes Joachim On 06 Oct 2016, at 15:56, Joachim Hein <[email protected]<mailto:[email protected]>> wrote: On 04 Oct 2016, at 16:11, Jack Perdue <[email protected]<mailto:[email protected]>> wrote: Jack Perdue Lead Systems Administrator High Performance Research Computing TAMU Division of Research [email protected]<mailto:[email protected]> http://hprc.tamu.edu<http://hprc.tamu.edu/> HPRC Helpdesk: [email protected]<mailto:[email protected]> On 10/04/2016 07:13 AM, Kenneth Hoste wrote: Hi Joachim, On 03/10/16 12:01, Joachim Hein wrote: Hi, I am trying to a version push on a package that needs FFTW. I noticed that FFTW 3.3.5 is out now and there are FFTW 3.3.5 packages in EB 2.9.0 (for e.g foss/2016.09). Looking into the release node for FFTW 3.3.5, avx2 seems a big addition. Looking into the EB 2.9.0 configurations, it seems sse2 is hard coded. To my understanding that is a factor of 4 in theoretical performance (2x for register length and 2x for FMA). The current SSE2 should be a lowest common denominator for current hardware. I am pretty sure an AVX2 version will do a nice illegal instruction error on oldish hardware (e.g. Interlagos). Thank you very much for bringing this up, and carefully checking the FFTW release notes and verifying what the FFTW easyconfigs specify. This type of thing is exactly what makes the EasyBuild community so valuable... We now have a bit of a can of worms. I am not supporting any pre avx2 hardware any longer, but I am sure some people here do. For FFTW we could make that explicit, e.g. adding an avx2 to the name of the FFTW. But then I am not building FFTW for the sake of building FFTW, I want to build NAMD with it. If I now supply a NAMD using the avx2 FFTW into the git repository, it will fail on any system deploying pre-avx2 hardware. For the compiler we are dealing with these kind of things via -xHost and -march=native options. Is there an EB mechanism to deal with this kind of thing? To my recollection, the 'hardcoding' (via --enable-sse2) you refer to was not done for the sake of hardcoding to SSE2 as a common base, but more to ensure that the FFTW builds with single and double (default) precision are properly using *at least* SSE2 (which they don't seem to do by default). Of course, that does imply that AVX(2) is bypassed entirely... That does make me wonder how the higher precision builds (--enable-double-long and --enable-quad-precision) are handled on older systems... Anyway, you're right, the real problem to me seems to be that these configuration options are specified totally independent of the 'optarch' configuration option. The question is, how do we fix this... Do we implement a custom easyblock for FFTW that takes into account both the --optarch configuration setting, together with the features of the system it is being installed on? That way, we could probably do a more intelligent pick of configure options to use (keeping --enable-sse2 as the default, unless --optarch=GENERIC is used). That's probably easier said than done, but I don't see a better way since this is quite specific to FFTW? Don' forget altivec on Power. :) http://www.siliconslick.com/easybuild/ebfiles_repo_cleaned/curie/FFTW/FFTW-3.3.4-gompi-2016a.eb Anyway, it demonstrates one approach (querying the build system CPU). Hi, I was away for a few days. I’d like to raise a few points to consider: * FFTW is a crucial ingredient of the beloved foss toolchain (one of the two most important ones if you ask me). Because so many packages depend on a foss, we can need a mechanism, that a single easyconfig/easyblock pair specifying e.g. foss/2016.09 will build an executable in a convenient fashion with either of SSE or AVX * Querying the host CPU is, in my view, not good enough. I feel EB needs to support cross compiling. While I currently do not need to support a system with different CPUs on the front- and back-end, I had to do so in the past. * An idea might be querying the host for a “default” and having the option to overwrite that default in the config file. * Can EB handle multiple config files for those poor animals who have to support systems with different kinds of back-end nodes? * A simpler solution might be, that easy building the relevant packages stop and ask for user input (e.g. instruction set options relevant to the package). One would need to flag that in the dry-run, that one expect an OpenFOAM build ontop a foss toolchain on an idle machine to succeed without human intervention. * I think this is an important issue that needs a good solution instead of a quick fix. * An issue I increasingly see with EB is complication and documentation. Adding any of above ideas will add to it. I spent a fair amount of time with the EB docs, and still I often get to hear: It is all there already, use this option and do that. Let’s hear what you think - or should we take that to a different forum instead of the mailing list? Best wishes Joachim jack
