On 11 November 2010 10:56, Christopher Tate <[email protected]> wrote: > I don't want to rain on peoples' parade here, because ccache is a > great product that has real benefits, but I do want to share some of > our findings regarding the use of ccache in our very large product -- > we were surprised by them, and you may be as well. These findings are > specifically for *large products*. In our case, the total source code > file size is on the order of 3 gigabytes (which includes not only > C/C++ but also Java source files, a couple hundred thousand lines of > makefiles, etc). It's the Android mobile phone OS, fwiw: it builds > something like 1-2 gigabytes of .o files from C/C++ during a full > build, and does a ton of Java compilation, resource compilation, > Dalvik compilation, etc as well.
I'd love to know whether you also tried distcc for it, and if so what happened or what went wrong. (Obviously it can only help for the C/C++ phases.) Your basic point that ccache can only help if you're doing strictly unnecessary builds is quite correct; eliminating these at the build tool level may be more elegant or efficient. > The issue is around VM/file system buffer cache management. If you're > using ccache, then you'll effectively be doubling the number of .o > files that are paged into memory during the course of a build. I'm just trying to understand how this happens. Is it that when ccache misses it writes out an object file both to the cache directory and into the build directory, and both will be in the buffer cache? So it's not so much they're paged in, but they are dirtied in memory and will still be held there. Perhaps in an ideal world ccache could tell the kernel that it's less useful to hold the .ccache o file (which probably won't be used until the next rebuild) than it is to hold the working tree o file (which may be used soon by the kernel.) We could try madvise MADV_DONTNEED, at least to see what happens. It seems like turning on compression would reduce the effect. Turning on hardlinking might eliminate it altogether, though that could have other bad effects. -- Martin _______________________________________________ ccache mailing list [email protected] https://lists.samba.org/mailman/listinfo/ccache
