On Mon, Jun 27, 2016 at 4:33 PM, Ward Poelmans <[email protected]> wrote:
> However, when creating new easyblocks/easyconfigs, it can be quite handy > to speed things up. Usually, it will involved build the same software > several times until the new easyblock fully works. > I agree with this too. So, I guess it should just be properly documented so one does not think this option will speedup a 'normal' build. Xavier > Ward > ------------------------------ > *From:* [email protected] <[email protected]> on behalf > of Xavier Besseron <[email protected]> > *Sent:* Monday, June 27, 2016 2:56 PM > *To:* [email protected] > *Subject:* Re: [easybuild] ccache support in EasyBuild? > > Hi Easybuilders, > > My opinion is that ccache will be pretty useless for EasyBuild. > The typical usecase of ccache is when you build the same source (in the > same directory), with the same compiler, with the same options. > Basically, it will speedup a "make ; make clean ; make". > > I don't think that's a common workflow with EasyBuild. > > https://ccache.samba.org/manual.html#_how_ccache_works > > Bonus: > https://ccache.samba.org/performance.html > > > > Best regards, > > Xavier > > > > On Thu, Jun 23, 2016 at 8:59 AM, Kenneth Hoste <[email protected]> > wrote: > >> (migrated from Lmod-users mailing list) >> >> Does anyone have experience with combining ccache with EasyBuild? >> >> Would it make sense to add support for something like 'eb --use-ccache', >> to make EasyBuild create symlinks to ccache for each of the compilers it >> may be using, and shove the path to them first in $PATH? >> >> >> regards, >> >> Kenneth >> >> On 22/06/16 21:56, Jack Perdue wrote: >> >>> /me ponders whether Robert (or Kenneth) might think this discussion >>> might not be better served on the EasyBuild list.... >>> >>> jack (who's down with Adam... and used ccache a lot while >>> grinding out RPMs on local hardware but doesn't mess >>> with it it much these days on our clusters.... >>> http://siliconslick.com/papitools/epel/6/SRPMS/ ). >>> >>> On 06/22/2016 12:12 PM, Adam Huffman wrote: >>> >>>> Used to be very grateful for ccache a few years ago when I was >>>> regularly building large Fedora RPMs at home. >>>> >>>> ccache -s >>>> >>>> shows you how well it's working. >>>> >>>> It is something I've wondered about in connection with EasyBuild. >>>> >>>> Does it impair strict reproducibility? >>>> >>>> On Wed, Jun 22, 2016 at 5:13 PM, Orion Poplawski <[email protected]> >>>> wrote: >>>> >>>>> On 06/22/2016 10:06 AM, Kenneth Hoste wrote: >>>>> >>>>>> Hi Orion, >>>>>> >>>>>> How much speedup have you seen with this setup, do you have any >>>>>> numbers on that? >>>>>> >>>>>> How often is ccache able to recycle object files (I guess that's >>>>>> where the >>>>>> biggest speedup is coming from) when different compilers are involved? >>>>>> >>>>> Sorry, I really don't have any numbers. Would be nice to have though. >>>>> >>>>> Any caveats at all with this, e.g. have you seen builds fail when >>>>>> using ccache >>>>>> but work fine without it? >>>>>> >>>>> I've not seen any issues using ccache. >>>>> >>>>> This looks like it could be an interesting feature in EasyBuild >>>>>> (--use-ccache), which could injects the symlinks to ccache at runtime >>>>>> in >>>>>> $PATH... :-) >>>>>> >>>>>> >>>>>> regards, >>>>>> >>>>>> Kenneth >>>>>> >>>>>> On 22/06/16 17:27, Orion Poplawski wrote: >>>>>> >>>>>>> Perhaps others might find this useful. >>>>>>> >>>>>>> ccache is a compiler cache that can speed up multiple builds of >>>>>>> larger >>>>>>> projects. In Fedora/EPEL the ccache package puts links of compiler >>>>>>> names >>>>>>> pointing to ccache in /usr/lib{,64}/ccache and adds >>>>>>> /etc/profile.d/ccache.{c,}sh to prepend that directory to PATH. >>>>>>> >>>>>>> Unfortunately when loading a compiler module, the compiler's >>>>>>> directory gets >>>>>>> prepended to the PATH and ccache is no longer active. Locally I >>>>>>> install this >>>>>>> Core modulefile: >>>>>>> >>>>>>> $ cat /opt/modulefiles/Core/ccache.lua >>>>>>> prepend_path{"PATH","/usr/lib64/ccache",priority=100} >>>>>>> >>>>>>> With the ccache module loaded then it keeps the /usr/lib64/ccache >>>>>>> path at the >>>>>>> front. Adjust priority as needed for your installation. >>>>>>> >>>>>>> And I also install links for icc/icpc, although I'm not particularly >>>>>>> sure how >>>>>>> much ccache works with icc/icpc though I came across an (older) >>>>>>> Intel web page >>>>>>> that mentioned using ccache with icc. >>>>>>> >>>>>>> $ ls -l /usr/lib64/ccache/icc /usr/lib64/ccache/icpc >>>>>>> lrwxrwxrwx. 1 root root 16 Jun 2 15:03 /usr/lib64/ccache/icc -> >>>>>>> ../../bin/ccache >>>>>>> lrwxrwxrwx. 1 root root 16 Jun 2 15:03 /usr/lib64/ccache/icpc -> >>>>>>> ../../bin/ccache >>>>>>> >>>>>>> HTH, >>>>>>> >>>>>>> Orion >>>>>>> >>>>>>> -- >>>>> Orion Poplawski >>>>> Technical Manager 303-415-9701 x222 >>>>> NWRA, Boulder/CoRA Office FAX: 303-415-9702 >>>>> 3380 Mitchell Lane [email protected] >>>>> Boulder, CO 80301 http://www.nwra.com >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San >>>>> Francisco, CA to explore cutting-edge tech and listen to tech >>>>> luminaries >>>>> present their vision of the future. This family event has something for >>>>> everyone, including kids. Get more information and register today. >>>>> http://sdm.link/attshape >>>>> _______________________________________________ >>>>> Lmod-users mailing list >>>>> [email protected] >>>>> https://lists.sourceforge.net/lists/listinfo/lmod-users >>>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San >>>> Francisco, CA to explore cutting-edge tech and listen to tech luminaries >>>> present their vision of the future. This family event has something for >>>> everyone, including kids. Get more information and register today. >>>> http://sdm.link/attshape >>>> _______________________________________________ >>>> Lmod-users mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/lmod-users >>>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San >>> Francisco, CA to explore cutting-edge tech and listen to tech luminaries >>> present their vision of the future. This family event has something for >>> everyone, including kids. Get more information and register today. >>> http://sdm.link/attshape >>> _______________________________________________ >>> Lmod-users mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/lmod-users >>> >> >> > > > -- > Dr Xavier BESSERON > Research associate > FSTC, University of Luxembourg > Campus Kirchberg, Office E-007 > Phone: +352 46 66 44 5418 > http://luxdem.uni.lu/ > > -- Dr Xavier BESSERON Research associate FSTC, University of Luxembourg Campus Kirchberg, Office E-007 Phone: +352 46 66 44 5418 http://luxdem.uni.lu/
