Re: build system option to allow CPU optimizations?
Hi, Ricardo Wurmus skribis: > Ludovic Courtès writes: [...] > It may very well be the wrong approach in principle, but I also think > that it’s a neat escape hatch for specific use cases. Separating > reproducibility patching makes the package transformation mechanism > more powerful and appealing. Much like respecting TESTS? makes it > easy for users of modified packages to bypass a failing test suite, > making patching of Makefiles to remove CPU tuning conditional would > make for much less complex custom package definitions. > >> I found one case though where this is not possible: C++ header-only >> libraries such as Eigen contain hand-optimized vectorized routines, >> selected at build time, but we end up compiling Eigen users as the >> x86_64/AArch64 baseline, which is a waste. (If you do know of other >> problematic cases, I’m interested in taking a look!) >> >> My solution to that is “package multi-versioning” via a >> transformation >> option. Hopefully I’ll submit preliminary patches within a week or >> so! > > Oh, exciting! I forgot to mention it here, but it’s available for testing and probably even ready to merge: https://issues.guix.gnu.org/52283 I think it makes an option to dismiss ‘-march’ removal unnecessary; or, put differently, it achieves the same. I’m interested in seeing which packages people would mark as “tunable” and what performance gains it gives! Thanks, Ludo’.
Re: build system option to allow CPU optimizations?
Hi Ludovic, Ludovic Courtès writes: > Hi, > > zimoun skribis: > >> On Wed, 24 Nov 2021 at 13:10, Ricardo Wurmus wrote: >> >>> The build phases that patch out these features would have to check >>> for that build system option, much like they check the TESTS? >>> option before attempting to run tests. >> >> Then it could be a transformation. The idea sounds good to me. > > I’ve been working on it last week with my HPC hat on. > > To be clear, I think in may cases, passing ‘-march’ like you suggest is > the wrong approach; instead software should use (and usually does use) > function multi-versioning: > > https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/ > > I found one case though where this is not possible: C++ header-only > libraries such as Eigen contain hand-optimized vectorized routines, > selected at build time, but we end up compiling Eigen users as the > x86_64/AArch64 baseline, which is a waste. (If you do know of other > problematic cases, I’m interested in taking a look!) I think that 'atlas' is such an example of a package that uses multi-versioning but fails to build reproducibly depending on the exact CPU it was built. I've reported that here [0]. [0] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=51536 Thank you, Maxim
Re: build system option to allow CPU optimizations?
Ludovic Courtès writes: Hi, zimoun skribis: On Wed, 24 Nov 2021 at 13:10, Ricardo Wurmus wrote: The build phases that patch out these features would have to check for that build system option, much like they check the TESTS? option before attempting to run tests. Then it could be a transformation. The idea sounds good to me. I’ve been working on it last week with my HPC hat on. To be clear, I think in may cases, passing ‘-march’ like you suggest is the wrong approach; instead software should use (and usually does use) function multi-versioning: https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/ It may very well be the wrong approach in principle, but I also think that it’s a neat escape hatch for specific use cases. Separating reproducibility patching makes the package transformation mechanism more powerful and appealing. Much like respecting TESTS? makes it easy for users of modified packages to bypass a failing test suite, making patching of Makefiles to remove CPU tuning conditional would make for much less complex custom package definitions. I found one case though where this is not possible: C++ header-only libraries such as Eigen contain hand-optimized vectorized routines, selected at build time, but we end up compiling Eigen users as the x86_64/AArch64 baseline, which is a waste. (If you do know of other problematic cases, I’m interested in taking a look!) My solution to that is “package multi-versioning” via a transformation option. Hopefully I’ll submit preliminary patches within a week or so! Oh, exciting! -- Ricardo
Re: build system option to allow CPU optimizations?
Hi, zimoun skribis: > On Wed, 24 Nov 2021 at 13:10, Ricardo Wurmus wrote: > >> The build phases that patch out these features would have to check >> for that build system option, much like they check the TESTS? >> option before attempting to run tests. > > Then it could be a transformation. The idea sounds good to me. I’ve been working on it last week with my HPC hat on. To be clear, I think in may cases, passing ‘-march’ like you suggest is the wrong approach; instead software should use (and usually does use) function multi-versioning: https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/ I found one case though where this is not possible: C++ header-only libraries such as Eigen contain hand-optimized vectorized routines, selected at build time, but we end up compiling Eigen users as the x86_64/AArch64 baseline, which is a waste. (If you do know of other problematic cases, I’m interested in taking a look!) My solution to that is “package multi-versioning” via a transformation option. Hopefully I’ll submit preliminary patches within a week or so! Thanks, Ludo’.
Re: build system option to allow CPU optimizations?
Hi Ricardo, On Wed, 24 Nov 2021 at 13:10, Ricardo Wurmus wrote: > The build phases that patch out these features would have to check > for that build system option, much like they check the TESTS? > option before attempting to run tests. Then it could be a transformation. The idea sounds good to me. Cheers, simon
build system option to allow CPU optimizations?
Hi Guix, currently we patch source code and build systems to ensure that no special instructions are used that would not be portable, e.g. AVX2, SSE4.1 etc. What do you think of adding a build system option that would allow users to restore these optimizations? The build phases that patch out these features would have to check for that build system option, much like they check the TESTS? option before attempting to run tests. What do you think? -- Ricardo
