Re: [ccache] Combining multiple ccache into one
Den 2018-03-14 kl. 19:31, skrev Basile Starynkevitch via ccache: > > > On 03/14/2018 06:54 PM, Jason Zhou via ccache wrote: >> Hi, >> >> I am looking for an efficient way to correctly combine multiple ccache from >> hundreds of build machines into a single ccache to build a super set ccache. > > perhaps you should consider distcc https://github.com/distcc/distcc or > icecream https://github.com/icecc/icecream > The use of distcc is orthogonal to the use of ccache, it only applies in the case of a cache miss (compile) /Anders ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache - .d files with absolute system header paths
Karanam Suryanarayana Rao Venkata Krishna wrote: > Hello, > I think I discovered a scenario that results in cache misses in spite of > using CCACHE_BASEDIR. > Consider the following command: > > CCACHE_BASEDIR=$PWD /bin/bash -c "ccache clang++ > -fdebug-prefix-map=/proc/self/cwd= -g -c -MD -MF hello.d -o hello.o > hello.cpp" > > It seems to me that is is perfectly alright to ask for debug prefix mapping > like: "-fdebug-prefix-map=/proc/self/cwd=" > Unfortunately, ccache is ending up hashing gnu_getcwd(); thus, even though > we use CCACHE_BASEDIR setting, such a cache cannot be shared by other > users' from different workspaces resulting in cache misses. ... > > If the string after "=" in the mapping is null string, then, I hash "./". Can't you just use "$PWD" and ".", instead of this elaborate scheme ? Especially since using /proc/self/cwd doesn't even work, not with GCC... i.e. when you give that prefix, it will just look for that path string In the actual debug info, you will _still_ have a reference the cwd. /Anders ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] base_dir and symbolic links
Andreas Wettstein wrote: > In function `make_relative_path` (file `ccache.c`), the given > path is made "canonic" before it is converted to a path relative > to the current working directory. In particular, "canonic" means > that symbolic links are removed. I understand that it makes > sense to make the current working directory canonic, but I do not > see why this done for the path given to `make_relative_path`. Is > it really necessary? > > The reason why I am asking is that I have a usage scenario where > this reduces sharing a cache among different users. Seems like a pretty niche use case, but if you can come up with a patch that doesn't break too much of current behaviour - why not. You should still get preprocessor hits, but suppose you would rather get "direct" hits instead of having to read both files. Worst case, it could be an option (that defaulted to false) ? Then you could override it at runtime, for your use case... /Anders ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Swift Support?
Jimmy Yue wrote: > I was wondering if anybody in this list has tried support swift and maybe > even ibtool. I'm looking to implement this myself if there's a gap here, > but if anybody has tried it before and ran into problems I'd love to know. I think the only thing Swift has in common with C++ are the slow compile times, or else I'm missing something. Interface Builder is even further. But maybe you mean that it could use a similar (memoization) approach ? We had a similar question before, but I think that it went unanswered... https://lists.samba.org/archive/ccache/2016q3/001465.html I think it *could* probably be done, but it would look a lot different. The first step would be to get a list of all the file dependencies. There is no preprocessor, so no need to worry about (just modules) Apparently there is a "swiftc -emit-dependencies" (output depends), maybe the .d files could be parsed to generate something useful ? Might be some similarities to https://github.com/ccache/ccache/pull/130 And it would probably make for a separate program/project, as well... Even the Objective-C stuff (i.e. modules) is getting hard to support. /Anders ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Theoretical question regarding ccache
Aashaka Shah wrote: > I recently came across ccache as I have an assignment to think of a novel > compiler design or related problem, and having a cache for compilers was > the first thing that came to my mind. I thought of trying it out on > QuantLib, a financial computation library. Hi! It would interesting to have some more scientific research on this, since ccache is a rather pragmatic project. It's "memoization", I think. > I would like to know > > 1. Why multiple source files cannot take advantage of ccache ( Why other > types of compilations (multi-file compilation, linking, etc) will silently > fall back to running the real compiler) Well, depends on what you mean with "source" exactly. Basically when you have one source (.c), it would still read a lot of headers (.h) etc... But when you give the compiler more than one C/C++ source file, it will compile them one by one anyway. Looks like your build system is broken ? Do you have an example of the actual files you are trying to compile ? It would help when trying to understand what you are trying to do here. > 2. Where in the memory hierarchy(cache, main memory?) does the ccache > output reside? Does it work according to the default replacement policies > of the hardware cache? The ccache files are stored on the filesystem, so in the "secondary" ? Usually it's stored in RAM and written to disk, unless using a ramdisk. Using the memcached implementation also offers you other mixed options. https://en.wikipedia.org/wiki/Memcached (see the dev/memcached branch) /Anders ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
[ccache] Testing on other platforms (than Linux)
Hi ccache! We have earlier had some problems on testing on other *Unix* platforms, like Solaris or FreeBSD. Mostly because none of us are running those... https://github.com/ccache/ccache/issues/148 https://svnweb.freebsd.org/ports/head/devel/ccache/ We are also seeing some bugs that are "unique" to either Mac or Win, but hard to reproduce with regular Clang or MinGW running on Linux. https://github.com/ccache/ccache/issues/54 https://github.com/ccache/ccache/issues/156 https://github.com/ccache/ccache/issues/95 https://github.com/ccache/ccache/issues/122 Lately we had some discussion on this, with Joel explaining his view: https://github.com/ccache/ccache/pull/162#issuecomment-289215585 We could need some more ideas, on how to make this work better ? Meanwhile, I've updated Clang and MinGW so that they work again. I also added a Docker build to define the "basic platform" and plan on extending this with clang and mingw (like in Travis) Also updated some more of the "clang-analyzer" tests, and to the new 64-bit versions of MinGW-w64 and Wine64 (instead of 32-bit). But what would be the best option to keep the base project both clean and portable ? And how to make it easier to test/contribute. I think the current infrastructure with Travis, and now Docker, are a good baseline, maybe we could add some Mac or Win in the cloud ? I know that we have access to Appveyor (we tried to use it before), and that the new "MSYS2" is looking promising (although is very slow) https://github.com/ccache/ccache/pull/69 http://www.msys2.org/ (uses Pacman, yeah!) But are there any better options, or more people wanting to help out ? My plan was to try to get the memcached and compression working instead. https://github.com/ccache/ccache/pull/58 (+ https://github.com/ccache/ccache/pull/104) https://github.com/ccache/ccache/pull/118 (+ https://github.com/ccache/ccache/pull/81) /Anders ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Visual C/C++ compiler upgrade
Jean-Dominique GASCUEL wrote: > Dear ccache developers, > > I just started a fork to try to make ccache compatible with Visual C/C++ > compiler (cl), so one can use it with msbuild or nmake based projects... > You can review the current state here: https://github.com/jd-gascuel/ccache > > Current state: > - It starts to do something interesting. But more work is needed to > handle specific options (.sbr files, debug options, etc.) > - I found the Travis stuff very interesting. But because Travis do not > support windows, I am trying to make an AppVeyor similar stuff. > - There is probably issue about Visual vs. MinGW modes ... I did not > start to investigate that. > > Any advices ? We talked a bit about the Windows version and Appveyor, in https://github.com/ccache/ccache/issues/122 and https://github.com/ccache/ccache/pull/69 You might want to look into those, even if about MinGW... I suppose you have already seen other "inspired" work, like https://github.com/frerich/clcache or https://github.com/inorton/cclash Even though those are in different languages (Python/C#) The lack of autotools (i.e. no decent shell available) is going to be something of a problem, in the long run. The same goes for the test suite, that is also going to need some thinking if it should work on "real" Windows. Supposedly one could use something like CMake for this, but there is a risk of making things harder on Unix... Some other things can make you cringe, like the use of /options or \\directories, but goes with cross-platform. Think the ultimate question will be up to Joel to decide: "Are you happy to have a windows branch in the official ccache repo ? and to merge it to the default trunk once it works great ?" https://github.com/ccache/ccache/pull/162 /Anders ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
