For ordinary EasyBuild use, I agree it's quite useless.
However, when creating new easyblocks/easyconfigs, it can be quite handy to speed things up. Usually, it will involved build the same software several times until the new easyblock fully works. Ward ________________________________ From: [email protected] <[email protected]> on behalf of Xavier Besseron <[email protected]> Sent: Monday, June 27, 2016 2:56 PM To: [email protected] Subject: Re: [easybuild] ccache support in EasyBuild? Hi Easybuilders, My opinion is that ccache will be pretty useless for EasyBuild. The typical usecase of ccache is when you build the same source (in the same directory), with the same compiler, with the same options. Basically, it will speedup a "make ; make clean ; make". I don't think that's a common workflow with EasyBuild. https://ccache.samba.org/manual.html#_how_ccache_works Bonus: https://ccache.samba.org/performance.html Best regards, Xavier On Thu, Jun 23, 2016 at 8:59 AM, Kenneth Hoste <[email protected]<mailto:[email protected]>> wrote: (migrated from Lmod-users mailing list) Does anyone have experience with combining ccache with EasyBuild? Would it make sense to add support for something like 'eb --use-ccache', to make EasyBuild create symlinks to ccache for each of the compilers it may be using, and shove the path to them first in $PATH? regards, Kenneth On 22/06/16 21:56, Jack Perdue wrote: /me ponders whether Robert (or Kenneth) might think this discussion might not be better served on the EasyBuild list.... jack (who's down with Adam... and used ccache a lot while grinding out RPMs on local hardware but doesn't mess with it it much these days on our clusters.... http://siliconslick.com/papitools/epel/6/SRPMS/ ). On 06/22/2016 12:12 PM, Adam Huffman wrote: Used to be very grateful for ccache a few years ago when I was regularly building large Fedora RPMs at home. ccache -s shows you how well it's working. It is something I've wondered about in connection with EasyBuild. Does it impair strict reproducibility? On Wed, Jun 22, 2016 at 5:13 PM, Orion Poplawski <[email protected]<mailto:[email protected]>> wrote: On 06/22/2016 10:06 AM, Kenneth Hoste wrote: Hi Orion, How much speedup have you seen with this setup, do you have any numbers on that? How often is ccache able to recycle object files (I guess that's where the biggest speedup is coming from) when different compilers are involved? Sorry, I really don't have any numbers. Would be nice to have though. Any caveats at all with this, e.g. have you seen builds fail when using ccache but work fine without it? I've not seen any issues using ccache. This looks like it could be an interesting feature in EasyBuild (--use-ccache), which could injects the symlinks to ccache at runtime in $PATH... :-) regards, Kenneth On 22/06/16 17:27, Orion Poplawski wrote: Perhaps others might find this useful. ccache is a compiler cache that can speed up multiple builds of larger projects. In Fedora/EPEL the ccache package puts links of compiler names pointing to ccache in /usr/lib{,64}/ccache and adds /etc/profile.d/ccache.{c,}sh to prepend that directory to PATH. Unfortunately when loading a compiler module, the compiler's directory gets prepended to the PATH and ccache is no longer active. Locally I install this Core modulefile: $ cat /opt/modulefiles/Core/ccache.lua prepend_path{"PATH","/usr/lib64/ccache",priority=100} With the ccache module loaded then it keeps the /usr/lib64/ccache path at the front. Adjust priority as needed for your installation. And I also install links for icc/icpc, although I'm not particularly sure how much ccache works with icc/icpc though I came across an (older) Intel web page that mentioned using ccache with icc. $ ls -l /usr/lib64/ccache/icc /usr/lib64/ccache/icpc lrwxrwxrwx. 1 root root 16 Jun 2 15:03 /usr/lib64/ccache/icc -> ../../bin/ccache lrwxrwxrwx. 1 root root 16 Jun 2 15:03 /usr/lib64/ccache/icpc -> ../../bin/ccache HTH, Orion -- Orion Poplawski Technical Manager 303-415-9701 x222<tel:303-415-9701%20x222> NWRA, Boulder/CoRA Office FAX: 303-415-9702<tel:303-415-9702> 3380 Mitchell Lane [email protected]<mailto:[email protected]> Boulder, CO 80301 http://www.nwra.com ------------------------------------------------------------------------------ Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San Francisco, CA to explore cutting-edge tech and listen to tech luminaries present their vision of the future. This family event has something for everyone, including kids. Get more information and register today. http://sdm.link/attshape _______________________________________________ Lmod-users mailing list [email protected]<mailto:[email protected]> https://lists.sourceforge.net/lists/listinfo/lmod-users ------------------------------------------------------------------------------ Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San Francisco, CA to explore cutting-edge tech and listen to tech luminaries present their vision of the future. This family event has something for everyone, including kids. Get more information and register today. http://sdm.link/attshape _______________________________________________ Lmod-users mailing list [email protected]<mailto:[email protected]> https://lists.sourceforge.net/lists/listinfo/lmod-users ------------------------------------------------------------------------------ Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San Francisco, CA to explore cutting-edge tech and listen to tech luminaries present their vision of the future. This family event has something for everyone, including kids. Get more information and register today. http://sdm.link/attshape _______________________________________________ Lmod-users mailing list [email protected]<mailto:[email protected]> https://lists.sourceforge.net/lists/listinfo/lmod-users -- Dr Xavier BESSERON Research associate FSTC, University of Luxembourg Campus Kirchberg, Office E-007 Phone: +352 46 66 44 5418 http://luxdem.uni.lu/
