Hi, I setup ccache with shared cache for some of our projects and I would like to learn how to do it correctly.
Projects here have 1-2 MLOC in 3-10k files (mostly C++) and are built via cmake files. We have around 20 devs active typically plus Jenkins and mostly they compile the same inputs (HEADs of active branches with only their few changes) in various configurations (targets, debug/release...), in total build output can be 5-30 GB. I made some tests and found ccache be very efficient (reducing total build duration by ten or so, depends on many factors of course). However, I still have some issues. Setup: using cmake wrapper scripts that exports: - CCACHE_BASEDIR - CCACHE_SLOPPINESS=file_macro,time_macros - CCACHE_CPP2=set and havine ccache.conf like: max_size = 25.0G # find $CCACHE_DIR -type d | xargs chmod g+s cache_dir=/local/users/zcone-pisint/tmp/ccache hard_link=false umask=002 The straight case ("ccache -Cz && make clean all && make clean all && ccache -s") works as expected, first build slow, second very fast, 50% hits. Is this so far reasonable? As workaround for a special unrelated issue currently we redefine __FILE__ (and try to remove that redefinition). I understand that ccache still works thanks to CCACHE_BASEDIR even for __FILE__ usage inside files. Is that correct? I understood that CCACHE_SLOPPINESS=file_macro means that cache results may be used even if __FILE__ is different, i.e. using a __FILE__ from another user (fine for our usecases), is this correct? NB: unfortunately cmake uses absolute paths, so __FILE__ contains user specific information (currently we redefine it not to do so, but we might drop this, because it harms other things). How to find a reasonable max_size? For now I just arbitrarily picked 25 GB (approximately the build tree size) and I never saw it "full" according to ccache -s. On build servers we usually run "make -j 25" (24 cores). Often, several such jobs are running by different users (and Jenkins; sometimes 400 compiler processes or even more). I assume ccache of course safely handles parallel invocation, is this correct? We have some team mates that have slow (old) laptops only. They benefit from using a network shared ccache. Technically, they "mount -t cifs" the cache_dir (NFS is firewalled unfortunately). We have different Ubuntu/Mint/Debian/Devuan machines, but exactly the same compilers (own toolchains). Is sharing via CIFS possibly at all or could it have bad effects? One issue that occures from time to time is that the ccache -s stats become zero (all values expect max cache size are 0). I first didn't notice because stats are shared so I assume someone zeroed the stats, but with alternate directories we found that it sometimes happens without ccache -s. "du -hs $CCACHE_DIR" still shows gigabytes used. We didn't find a cause yet, but several candidates exist. Can ccache be used on CIFS? Are cache and/or stats version dependent? I tried to deploy the same ccache everywhere (3.7.7, now 3.7.7+8_ge65d6c92), but maybe there is some host somewhere with an older version, hard to say. A few times we noticed that ccache -s reports few GB size but "du -hs" reports 40 or 50 GB, although "max_size = 25.0G". Is this expected? Could be a follow up problem from the one before. I'm also still facing scmake issues (using "physical" and "logical" in several mixed combinations). Complex topic. Any information / hints / pointers are appreciated! Best regards, Steffen _______________________________________________ ccache mailing list firstname.lastname@example.org https://lists.samba.org/mailman/listinfo/ccache