Re: [ccache] build octave with ccache causes segfault when checkinstall
Sounds like a ccache bug. See if you can get a traceback or core file. On Fri, Jul 3, 2015, 5:15 AM Kim Sun kim@ericsson.com wrote: Dear maintainers, if I try to build the latest octave development source with ccache and afterwards checkinstall, the process fails on checkinstall. The whole procedure: apt-get install git mercurial autoconf automake flex bison gperf gnuplot libreadline5-dev libncurses5-dev zlib1g-dev libatlas-base-dev fftw3-dev libglpk-dev libsuitesparse-dev libarpack2-dev libhdf5-mpich-dev libpcre3-dev libcurl4-gnutls-dev libqhull-dev gfortran libfltk1.1-dev libgraphicsmagick++1-dev transfig epstool libhdf5-serial-dev libfontconfig1-dev texlive export CXX=ccache g++ export CC=ccache gcc /home/andy/src/build_octave/libtool: line 8970: 4125 Segmentation fault ccache g++ -fPIC -DPIC -shared -nostdlib /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.6.1/crtbeginS.o [removed lines] .libs/liboctave_la-smx-scm-cm.o .libs/liboctave_la-smx-bm-sbm.o .libs/liboctave_la-smx-sbm-bm.o -Wl,--whole-archive ../libcruft/.libs/libranlib.a ../libgnu/.libs/libgnu.a -Wl,--no-whole-archive -Wl,-rpath -Wl,/usr/local/lib/octave/3.5.0+ -L/usr/local/lib/octave/3.5.0+ -lcruft -lcholmod -lumfpack -lamd -lcamd -lcolamd -lccolamd -lcxsparse -lqrupdate -L/usr/lib -lfftw3 -lfftw3f -llapack -lcblas -lf77blas -latlas -lreadline -lncurses -lpcre -ldl -L/usr/lib/gcc/x86_64-linux-gnu/4.6.1 -L/usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../../../lib -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../.. -L/usr/lib/x86_64-linux-gnu -lgfortran -lquadmath -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/4.6.1/crtendS.o /usr/lib/x86_64-linux-gnu/crtn.o -O2 -O2 -pthread -O2 -Wl,-rpath -Wl,/usr/local/lib/octave/3.5.0+ -pthread -Wl,-soname -Wl,liboctave.so.0 -o .libs/liboctave.so.0.0.0 libtool: install: error: relink `liboctave.la' with the above command before installing it make[4]: *** [install-octlibLTLIBRARIES] Fehler 1 This looks like ccache causes a segfault. I have reproduced this on different machines over the last 4 months. If I run checkinstall without ccache it completes without error. So my big questions are: Am I doing something wrong in building octave? Do you have the same problems when using ccache and checkinstall. Should I fill a bug report against ccache (my guess) or checkinstall? BR Kim ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Unable to install CCache on SunOS 5.10
You don't need admin rights to debug the build. On Thu, Apr 23, 2015, 10:57 PM Jeya Ganesh jeyaganes...@gmail.com wrote: Ok. I am a C/C++ developer but for accessing this machine with admin rights is a big process which will take long time as well. Is there anything that I need to do without admin rights..? Please let me know that, I shall try and let you know the status. Thanks, S. Jeya Ganesh On Thu, Apr 23, 2015 at 10:40 PM, Martin Pool m...@sourcefrog.net wrote: You're not doing anything wrong but there seems to be a portability shortcoming. Is there someone in your workplace who knows C and has access to this machine and can send a patch? On Wed, Apr 22, 2015 at 3:17 AM, Jeya Ganesh jeyaganes...@gmail.com wrote: Hi Team, I am unable to install ccache on my SunOS mahine. My machine has SunOS 5.10 - Generic_142900-15 sun4u sparc SUNW,SPARC-Enterprise. I tried the following commands and getting the error as mentioned below. * [sxxj1@dvcz7937:~/CCache/ccache-3.2.1]make* make: Fatal error in reader: Makefile, line 69: Macro assignment on dependency line * gmake* gcc -DHAVE_CONFIG_H -DSYSCONFDIR=/nfs/home02/sxxj1/CCache/ccache-3.2.1/etc -I. -I. -g -O2 -Wall -W -c -o ccache.o ccache.c In file included from system.h:28, from ccache.h:4, from ccache.c:22: /serve/public/unsupported/utils/lib/gcc-lib/sparc-sun-solaris2.3/2.7.2/include/sys/stat.h:85: parse error before `timestruc_t' /serve/public/unsupported/utils/lib/gcc-lib/sparc-sun-solaris2.3/2.7.2/include/sys/stat.h:85: warning: no semicolon at end of struct or union /serve/public/unsupported/utils/lib/gcc-lib/sparc-sun-solaris2.3/2.7.2/include/sys/stat.h:86: warning: data definition has no type or storage class /serve/public/unsupported/utils/lib/gcc-lib/sparc-sun-solaris2.3/2.7.2/include/sys/stat.h:87: parse error before `st_ctim' /serve/public/unsupported/utils/lib/gcc-lib/sparc-sun-solaris2.3/2.7.2/include/sys/stat.h:87: warning: data definition has no type or storage class /serve/public/unsupported/utils/lib/gcc-lib/sparc-sun-solaris2.3/2.7.2/include/sys/stat.h:92: parse error before `}' In file included from /serve/public/unsupported/utils/lib/gcc-lib/sparc-sun-solaris2.3/2.7.2/include/sys/signal.h:110, from /serve/public/unsupported/utils/lib/gcc-lib/sparc-sun-solaris2.3/2.7.2/include/signal.h:15, from system.h:41, from ccache.h: . . .ccache.c:2566: warning: implicit declaration of function `putenv' gmake: *** [ccache.o] Error 1 *which make* /usr/ccs/bin/make *which gmake* /serve/public/unsupported/utils/bin/gmake Please let me know where I am doing mistake and assist me in solving this issue. Thanks. S. Jeya Ganesh ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Sharing ccache among different workspaces.
Do the various compilations produce the exact same binary output? There might be something in your tree that makes the output vary from one run to the next. On Apr 29, 2014 3:50 PM, Sunny Jose sunju_j...@yahoo.com wrote: Hi, I want to share the ccache built using one workspace to be utilized by another workspace, so that one copy of ccache data is used for compiling identical source code. E.x: 1) Created a workspace (home/user/source_code1). 2) synced the code here from the server. 3) Compiled the Code with USE_CCACHE=1 The above will build my ccache. Now I do the following Case 1: 1) Created a workspace (home/user/source_code2). 2) synced identical source code here from the server. 3) Did the same compilation as I did in the earlier workspace. Case 2: 1) Delete workspace home/user/source_code1 2) Re-create the earlier workspace (home/user/source_code1). 2) Re-synced the code here from the server. 3) Compiled the Code with USE_CCACHE=1 I want source_code2 compilation to use the ccache built from source_code1 and when I delete and re-create the same workspace, I need the compilation to happen with the ccache already built. Any pointers to achieve the same are welcome! The result that I see now in both the above cases is though ccache is built it is never used or there are no hits in the ccache, it keeps on getting built up whenever a new workspace is used, If I just clean the build and re-compile in the same workspace without deleting any earlier source code(home/user/source_code1), then ccache compilation jumps in and there are hits which help in reducing the build time. Any idea why I do not see any ccache hits whenever a brand new workspace is used? I always have the ccache miss incremented rather than the cache hit (direct) , or , cache hit (preprocessed) . Also the cache hit (direct) is always 0. I have set the following variables. CCACHE_BASEDIR= /home/user USE_CCACHE=1 CCACHE_DIR=/home/user/ccache CCACHE_SLOPPINESS=time_macros,file_macro Kindly please let me know if I'm missing something obvious here. ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Bug compiling package freetype
What kind of file is the ftsystem.lo mentioned in the error? (What does `file $PATH` say?) On 18 March 2013 10:30, Giacomo Comes co...@naic.edu wrote: Hi, I have found a bug in ccache as it fails to compile the package freetype. I have tested the bug on CentOS 5 and 6 and openSUSE 12.2 with ccache 3.1.9. This is how to reproduce the bug: download freetype 2.4.11, untar and run ./configure. run make compilation is succesful. Redo everything but this time in the last step run: make CC=ccache gcc compilation will fail with the error message: libtool: link: `/dev/shm/freetype-2.4.11/objs/ftsystem.lo' is not a valid libtool object make: *** [/dev/shm/freetype-2.4.11/objs/libfreetype.la] Error 1 I have no idea if this is due to an already known bug of ccache or it is something new. Anyway I'm reporting it. Regards. Giacomo ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] auto-disabling ccache when gcc gets plugins?
Disabling all caching because any plugins are used seems like a bit of overkill: while they _can_ be impure, I wonder if most will be? But probably disabling when MELT is present would be reasonable. There is existing code to disable caching when particular flags are present so it should be a one-line fix. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Why not cache compile failures?
I think caching errors would be worth trying. It may help with configure, and it may also help with people rebuilding trees that from time to time have errors in them. Depending on the compiler, it may be possible to see from the waitstatus that it was interrupted, and so to avoid caching the result. Or, perhaps you can do something with process groups to let ccache observe the interrupt signal itself and so be sure not to cache it. If you emit a message when printing a cached error it ought to be safe enough to experiment with. On 29 September 2012 20:07, Shentino shent...@gmail.com wrote: I'd have to agree that caching failures is tricky. Transient errors are temporary by definition and shouldn't be cached. What if compile failures were only cached for a limited amount of time, say, an hour or so? I suppose you are proposing this because you think the cache will often be wrong, and limiting the TTL will limit the damage. But, if the cache is unreliable, adding in an additional time-based factor will make things worse: suppose someone is trying to debug it and things suddenly start working again? As much as possible we want things to be pure functions of the input, both in the input to ccache, and in how ccache itself behaves. The vast majority of errors are due to actual errors in the source code in the context of the environment and flags. If the compiler is intermittently giving errors for other reasons, the user really needs to stabilize them before thinking about using ccache. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Making ccache and clang compatible
On 5 July 2012 21:35, Mike Frysinger vap...@gentoo.org wrote: On Wednesday 04 July 2012 09:53:46 Max Horn wrote: Using ccache with clang tends to generate tons of warnings, which can range from simply being annoying, to causing autoconf failures. This is because ccache runs the compiler with -E on preprocessed input, but also passes -I, -isystem etc. flags on to the compiler. In clang, this triggers warnings about unused arguments. err, -I/-isystem/-D/-U/etc... are preprocessor flags and are valid when running in preprocessor mode (-E). i don't know if clang is broken, or your analysis is incorrect, but certainly that behavior you describe is wrong. -mike I agree with you there, Mike, but I think the bug originally described in https://bugzilla.samba.org/show_bug.cgi?id=8460 could possibly be valid. If you are compiling from a .i or .ii file, the -D and -I options can't have any effect. It's reasonable for clang to emit a warning about it, and it would be reasonable for ccache to strip those options when compiling a preprocessed file. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] PATCH: Look at include files' mtimes
On 22 May 2012 12:00, Justin Lebar justin.le...@gmail.com wrote: I've been burned by mtime only checking before as (excluding some recent file systems) mtime has a resolution only down to one second. I tried to address this in the patch, although come to think of it, I did it wrong. The trick is only to *cache* mtimes that are at least one second older than now. Then the resolution of the clock isn't a problem. Better to do 2 seconds, since FAT (and maybe some other Windows related setups) has only a 2-second resolution. The other thing you can do is, on Unix, use the latest of ctime and mtime, which should catch cases where the mtime gets reset. But if the system clock is set back (e.g. by NTP), we're in trouble. That's true, but it's a pretty bad idea to make large time steps on a build machine... And hardlinks are often created without bumping the inode's mtime, which is also problematic. (It's problematic for make, too.) ctime will catch that. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] question about safe usage of ccache
That's fine. On May 5, 2012 5:25 PM, John da_audioph...@yahoo.com wrote: I compile kernel packages for older PCs on my workstation. The builds are highly similar differing only in the a few option within the .config file I use for them (mainly differ cpu families). Will ccache mess up anything or is it perfect for my use? Example: Kernel package 1: generic x86 Kernel package 2: core2 optimized x86 Kernel package 3: atom optimized x86 Kernel package 3: pentium4 optimized x86 Thank you! ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Suggested patch to add local strtok_r for systems lacking it
Thanks for the patch. I guess the definition ought to be guarded by HAVE_STRTOK_R, not _WIN32. Perhaps you also need to update configure.in to check for it? m ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Implementing a Read-only HTTP CCACHE_DIR
Hi, Richard, The general thing of network-wide caching using something other than a network filesystem sounds good. There was some discussion (or patches? maybe even merged?) a while ago about using memcached, which seems like a pretty good idea to me rather than layering on http. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] What to do when there are problems writing to the cache directory?
On 9 January 2012 22:34, Joel Rosdahl j...@rosdahl.net wrote: On the other hand, my general opinion is that tools should not fail silently. As a ccache user, I would like to be informed if something is badly configured, for instance if the cache directory is unwritable by me, so that I get alerted that something is wrong. What do you think? I agree with the general approach. I suspect a common case where people hit this is something like 'make sudo make install' which can end up with some files being owned by a different user. In a sense they ought to fix this but perhaps many people would prefer to just have some stuff uncached. I would be inclined to at least give a suppressible warning rather than just silently falling back. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] How to support another compiler?
On 18 November 2011 14:34, Zhou, Yang yang.z...@polycom.com wrote: Hi all, I want to use ccache to optimize the compilation of my project. However, another compiler(not gcc) is used for the compilation. The compiler options are different from those of gcc. But I think the compilation process should be similar. I tried to compile with ccache, from the log, every time the manifest file can't be found, and can't find the object file in manifest. After that, preprocessor runs, and got object file hash from preprocessor, but the object file is not in cache. So the real compiler runs every time. Seems the results of real compilation are not stored into the cache. If the command line arguments are different that's not surprising. Do you think it's possible to change ccache source code to support the my compiler? Yes. If possible, which part of code should be updated? Thanks a lot! The parts that deal with compiler argument parsing. Probably you will want to introduce an object layer that describes different compilers, and an environment variable to switch between them. What's the other compiler you are using? m ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] How to support another compiler?
Hi, Yang Zhou, If you get this working, or even just a start towards it, please post the patch here and hopefully we can integrate it to help others. Regards, Martin On 18 November 2011 14:52, Zhou, Yang yang.z...@polycom.com wrote: Hi Martin, Thanks for help. I'm using cl6x from Texas Instruments, it's used for TI C6000 DSPs. Best Regards, Yang Zhou -Original Message- From: martinp...@gmail.com [mailto:martinp...@gmail.com] On Behalf Of Martin Pool Sent: Friday, November 18, 2011 11:48 AM To: Zhou, Yang Cc: ccache@lists.samba.org Subject: Re: [ccache] How to support another compiler? On 18 November 2011 14:34, Zhou, Yang yang.z...@polycom.com wrote: Hi all, I want to use ccache to optimize the compilation of my project. However, another compiler(not gcc) is used for the compilation. The compiler options are different from those of gcc. But I think the compilation process should be similar. I tried to compile with ccache, from the log, every time the manifest file can't be found, and can't find the object file in manifest. After that, preprocessor runs, and got object file hash from preprocessor, but the object file is not in cache. So the real compiler runs every time. Seems the results of real compilation are not stored into the cache. If the command line arguments are different that's not surprising. Do you think it's possible to change ccache source code to support the my compiler? Yes. If possible, which part of code should be updated? Thanks a lot! The parts that deal with compiler argument parsing. Probably you will want to introduce an object layer that describes different compilers, and an environment variable to switch between them. What's the other compiler you are using? m ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Duplicate object files in the ccache - possible optimization?
On 5 November 2011 11:12, Frank Klotz frank.kl...@alcatel-lucent.com wrote: I used ccache at my previous employer, and was very convinced of its value. Now that I have started a new job, I am in the process of trying to bring the new shop on board with ccache, so I have been doing lots of test runs and looking at things. Here is one thing I am thinking could add some value. Looking through the ccache, I find many pairs of files which have different names (different hashes), but exactly identical content. This actually makes sense, as each file would have an index hash and a preprocessed hash, and since ccache needs to be able to find a match on either, then both need to be in the cache. What is an index hash? (Actually, thinking about it, I'm a little surprised that there are any files in the ccache that DON'T appear twice - shouldn't EVERY compilation have 2 hashes?) I don't understand why you would expect that. It seems like you expect there is another indirection layer by which ccache tries to find jobs that produce identical output. I don't think there is one at present. I don't think this would happen very often in reality, except perhaps for trivial cases like compiling empty files, and that's not so important to accelerate, and it will not use up much disk space. If you're getting duplicated cache files due to for instance doing builds in different directories or from different trees that produce identical output you could change the ccache options to make it less stringent. But it seems to me that it would make a lot of sense to store the data of these 2 files only once, by hard-linking the 2 names to the same inode. (For filesystems that support hard links, of course!) Every time ccache does an actual compilation and stores a file in the cache, it should store it under hard links for BOTH hashes - the indexed hash and the proprocessed hash. And if it gets a hash miss on the indexed hash but a hit on the preprocessed hash, then it should add the missed index hash as a hard link to the file found. So a given file (inode) in the cache could actually be referenced by MANY directory entries: one preprocessed hash, and multiple index hashes for various different combinations of source files and header files which end up producing the same output when passed through the preprocessor. This mail is the first time google has heard of ccache indexed hash... This could increase the storage efficiency of the ccache. Of course, since not every filesystem supports hard links, the simplest solution was of course just to have multiple file copies. So I guess adding code to do this would require some way to determine if the filesystem the cache is on can in fact support hardlinks. If you think this sounds like a good idea, but don't have bandwidth to do it, I would be willing to give it a try. Any hints on where to start would of course be welcome. Thanks, Frank Klotz ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache cache in RAM -- bypassing file cache?
I think Justin is right that there should be no copying; it would be fairly insane if there was. I would speculate that the cause is nothing to do with the cache, but rather just that on the larger machine, more of the source tree and the header files used by it can be kept in memory. You could try with tmpfs instead to see if it's different. Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] problems with buildworld gcc45 on ports
This answer may be too shallow, but: do you need to get rid of the whitespace around the CC=? That won't work in shell. I don't know what parses that file. Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Configuration file plan
I would think about not putting ccache.conf inside .ccache, because that directory might often be excluded from backups or in some other way treated as disposable. Aside from that it sounds really good. Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Issues using ccache
On 20 January 2011 11:20, NITIN KHOSLA, BLOOMBERG/ 731 LEXIN nkhos...@bloomberg.net wrote: I am using first time. I have read documentation but not found my answers. Simple command: ccache gcc -o 1.o 1.c ccache -V ccache version 3.1.4 a) Ran first time and then second time. What called for link means? called for link 1 called for link 2 It should find it from cache second time. It did not. cache hit (direct) 0 cache miss 0 Linking is when the toolchain assembles the object files into an executable. [1] ccache doesn't cache these. Your command line is wrong; you need a -c option too, otherwise you're writing an executable into a .o file. [1] http://en.wikipedia.org/wiki/Linker_(computing) b) I created a ccache log file(log below is same for exactly same above mentioned gcc command called multiple times). : [2011-01-20T12:01:14.788191 10060] Hostname: sun15 [2011-01-20T12:01:14.788261 10060] Working directory: /home/nkhosla [2011-01-20T12:01:14.788969 10060] No -c option found [2011-01-20T12:01:14.789013 10060] Failed; falling back to running the real comp iler [2011-01-20T12:01:14.789038 10060] Executing /opt/swt/bin/gcc -o 1.o 1.c [2011-01-20T12:01:14.790973 10060] Acquired lock /home/nkhosla/.ccache/a/stats.l ock [2011-01-20T12:01:14.798012 10060] Releasing lock /home/nkhosla/.ccache/a/stats. lock [2011-01-20T12:01:14.798080 10060] Unlink /home/nkhosla/.ccache/a/stats.lock (as -tmp) [2011-01-20T12:01:14.798677 10060] Result: called for link c) Running with solaris compiler (CC) somewhere calls ccfe. And for that, ccache says unsupported source language 1 I don't know if it works with Solaris cc. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Radically different ccache performance on Linux vs OSX
As a poorly-informed guess, maybe the OS X build is turning on pre-compiled headers in a way ccache 2.4 can't cope well with? -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] globally unique hashes
On 21 June 2010 01:26, Reuben Hawkins reuben...@gmail.com wrote: c) distcc takes too much configuration/coordination, however, multicasting takes zero configuration/coordination Really? You don't have to do much more than run a daemon on the servers, and presumably you would need a multicast daemon to serve ccache requests. I think distcc can find its servers through mdns. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] globally unique hashes
On 22 June 2010 11:42, Reuben Hawkins reuben...@gmail.com wrote: On 06/21/2010 06:06 PM, Martin Pool wrote: On 22 June 2010 10:18, Reuben Hawkins reuben...@gmail.com wrote: Hi Martin, Yes, there would need to be a daemon to respond to multicasts. I think distcc requires setting all the hosts manually on each client (correct me if I'm wrong). Maybe what I need is distcc servers to respond to multicast so I can avoid maintaining the servers manually on the clients... I see you're one of the distcc developers. Would that be a welcome enhancement? Using multicast would probably still require some configuration on the clients about which multicast group to use, etc. Is there anything wrong with using mdns? Maybe I'm misunderstanding exactly what you're suggesting. There's nothing wrong with using mdns, but my understanding is that even with mdns, the names must still be known by the client. For example... $ distcc-config --set-hosts localhost c1.local c2.local c3.local ... cN.local What I'd like to be able to do is... $ distcc-config --multicast So when distcc starts a build, it will call out to the local network, who's running distccd? and the servers will reply such that my local machine can automatically update its list of hosts. mdns has a mechanism to say who offers this service? Apple's distcc version in Xcode does this. I don't know how well that works on generic unix but it seems like a good place to start, and the distcc list would be a good place to ask about it. RFC 3307 suggest that permanent IPv6 multicast addresses can be assigned (which I haven't really looked into yet, I was just going to pick one at random for now). I was not planning on using IPv4 at all. Is that reasonable or am I missing something? I don't think it would be reasonable to have one IPv6 address that is the global ccache service for the whole world. If the address is intended to be just network-wide or organization-wide, then really you do have per-machine configuration, you're just choosing to hardcode it into the program. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] globally unique hashes
On 22 June 2010 15:36, Reuben Hawkins reuben...@gmail.com wrote: On Tue, Jun 22, 2010 at 4:39 AM, Martin Pool m...@sourcefrog.net wrote: On 22 June 2010 11:42, Reuben Hawkins reuben...@gmail.com wrote: On 06/21/2010 06:06 PM, Martin Pool wrote: On 22 June 2010 10:18, Reuben Hawkins reuben...@gmail.com wrote: Hi Martin, Yes, there would need to be a daemon to respond to multicasts. I think distcc requires setting all the hosts manually on each client (correct me if I'm wrong). Maybe what I need is distcc servers to respond to multicast so I can avoid maintaining the servers manually on the clients... I see you're one of the distcc developers. Would that be a welcome enhancement? Using multicast would probably still require some configuration on the clients about which multicast group to use, etc. Is there anything wrong with using mdns? Maybe I'm misunderstanding exactly what you're suggesting. There's nothing wrong with using mdns, but my understanding is that even with mdns, the names must still be known by the client. For example... $ distcc-config --set-hosts localhost c1.local c2.local c3.local ... cN.local What I'd like to be able to do is... $ distcc-config --multicast So when distcc starts a build, it will call out to the local network, who's running distccd? and the servers will reply such that my local machine can automatically update its list of hosts. mdns has a mechanism to say who offers this service? Apple's distcc version in Xcode does this. I don't know how well that works on generic unix but it seems like a good place to start, and the distcc list would be a good place to ask about it. RFC 3307 suggest that permanent IPv6 multicast addresses can be assigned (which I haven't really looked into yet, I was just going to pick one at random for now). I was not planning on using IPv4 at all. Is that reasonable or am I missing something? I don't think it would be reasonable to have one IPv6 address that is the global ccache service for the whole world. If the address is intended to be just network-wide or organization-wide, then really you do have per-machine configuration, you're just choosing to hardcode it into the program. -- Martin I also agree it's unreasonable to have a whole world multicast address for distcc. But I'm not suggesting a global multicast address, just a link local address that doesn't route. Something in the FF02:: scope. I could use the link local all nodes address FF02::1, but the it would get lots of unwanted traffic. Hardcoding should be ok even if it is FF02::1 (all link local nodes) as this is what multicast is for...right? [please reply on the list] This sounds a lot like a reinvention of mDNS DNS-SD, which provides a generalized way to find a service on the local network by multicast queries. See eg http://en.wikipedia.org/wiki/Apple_Rendezvous. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] About Win32 binary
On 11 May 2010 17:02, asm warrior asmwarr...@gmail.com wrote: I have just test two version of ccache.exe( one from you and the other from a codeblocks' forum user reckless). It seems I still can't use it. I have reported the error build message in the codeblocks' forum post: http://forums.codeblocks.org/index.php/topic,12285.msg84932.html#msg84932 It seems ccache can't find the cache folder like: Unable to determine cache directory Maybe you need to set the environment variable CCACHE_DIR or HOME? -- Martin http://launchpad.net/~mbp/ ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
[ccache] New ccache User - Got Some Questions
On 11/07/2006, at 2:55 AM, Naqib Khan wrote: Hi, I'm new to ccache - it does appear to be an extremely useful tool for compiling software. I have read through all the archives, but don't seem to specifically find the answers I'm looking for - here they are: 1) Does ccache only work with gcc... or can I also use it with any arbitrary compiler? If not, which other compilers does it specifically support? (Note: I did see a similar question in the archives, but that was about 2 years ago). It only works with gcc or compilers that have similar external behaviour. (There are some niche compilers based on gcc, and the intel cc tries to be like gcc, but apparently is not close enough.) 2) ccache usage - the documentation / information on the web indicates that ccache is best suited for: C/C++ compiles; clean builds. My question is are those the type of builds I should be targeting (i.e. C/C++ compiles; clean builds)? It would appear that ccache should be able to speed up compiles in general regardless, due to it's sophistication (i.e. hashing versus plain old timestamp checking). It's useful in general; it just will give the biggest proportional win when you do make; make clean; make 3) In-house we also use clearmake (DOs wink-ins) - does anybody have experience with incorporating ccache in this type of environment? Specifically... 3a) I assume since ccache creates it's own object files, that ccache would essentially _replace_ clearmake, correct? Yes, they're trying to do the same thing at different levels. Clearmake (as I understand) can cache all kinds of output, not just C/ C++ compilation; ccache is a bit simpler and doesn't depend on clearmake or clearcase, and can work with distcc for an even bigger win. 3b) Just checking here (just in case) - before I get started on this journey :) - has anyone successfully used ccache with clearmake? I haven't. Maybe someone else has. -- Martin
[ccache] Can multiple users share a single cache?
On 13/04/2006, at 10:01 AM, Kiel Sturm wrote: With this tool, can multiple users share a single cache, like the build cache feature of http://freshmeat.net/projects/makepp/ Yes. -- Martin
[ccache] Re: [distcc] distcc - ccache: no error messages
Tim Niemueller wrote: Martin Pool wrote: Tim Niemueller wrote: [...@columbus firevision]$ make -j --- Compiling main.cpp (C++) gcc: \uC/distccd_219a0097.ii: File or directory not found gcc: no input files distcc[17494] ERROR: compile main.cpp on lechuck failed make: *** [.obj/main.o] Fehler 1 Do you really get a \u character there? Is it printed as the backslash escape, or is that unicode character written into the output stream? No, but the output looks as if it was filled with tabs. LANG=de_DE.UTF-8, so yes, this probably is a unicode escape. I don't know why. Thunderbird writes it if I copy by marking the message in my xterm and pasting. So if you have a syntax error, you get a 'file not found', plus this \u? That's wierd. -- Martin
[ccache] Other compilers (MSVC, SNSystem ProDG, Metrowerks CW)
On 7 Sep 2004, Anuj Goyal anuj.go...@gmail.com wrote: I tried doing the port to win32, but did not get very far, windows does not have a nice fork() call like POSIX, one has to call CreateProcess and much with the Handle, it's not very pleasant. One also has to figure out howto do a mmap() call, I know one can use CreateMemoryMapping() or something to that effect, but I don't know the specifics, I was also trying to learn howto use the digitalmars compiler at the time (because it has syntax more similar to that of gcc), but I gave up in frustration. There was a LOT of code that needed to be ported and that is not fun work, and it leads to ifdef hell. Thanks for letting us know. I wish the companies that support open source (sun, ibm, hpux) would actually take note of ccache, it is an amazing tool and they only really have to modifiy ccache.c to get it working for their compiler. I suspect the parts of those big companies that support open source are somewhat separate from the parts that write the proprietary compilers. -- Martin linux.conf.au 2005 - http://linux.conf.au/ - Call For Papers now open! April 18th to 23rd - Canberra, Australia -closes on October 5 -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://lists.samba.org/archive/ccache/attachments/20040908/35c1575c/attachment.bin
[ccache] Re: ccache 2.3 and coverage [patch]
On 25 May 2004, Markus F.X.J. Oberhumer mar...@oberhumer.com wrote: The tiny patch below enables compiling coverage information when using ccache. BTW, now that CVS has gone, are there any chances for a arch/tla repository on samba.org ? Hi Markus, We still have CVS on samba.org for most samba.org projects. It is only Samba itself, the CIFS implementation, that has switched to svn. Anoncvs was turned off because of the recent bug but it should now be reachable again. Please let me know if not. Having said that I might start an arch or darcs repo on samba.org to hold proposed ccache changes until tridge makes a new release. -- Martin -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://lists.samba.org/archive/ccache/attachments/20040526/a400c2f2/attachment.bin
[ccache] ccache doesn't compile binutils
On 14 Apr 2004, Edward S. Peschko e...@pge.com wrote: ( ps - has anyone made a large integration test with ccache, ie: integrated it into the gentoo builds, etc? If you could compile an entire gentoo build using ccache, I'd be more comfortable with it. ) Yes, this has been done. You can build all of Gentoo using ccache, except for the parts with broken makefiles, where it is specifically disabled. What's wrong with the makefiles? I thought that cache was transparent if you made a link to gcc/etc. I think some versions of libtool have bugs that are tripped by ccache. I cannot recall any specific examples at the moment. Anyways, I double checked, and I'm indeed using gnu's ld, etc. Does anyone have a solaris install to confirm? Or how would I go about tracking this down? Maybe try setting some of the debugging variables in the manpage? Are you using NFS for your working directory or cache? You might have an NFS problem. -- Martin
[ccache] ccache doesn't compile binutils
On 13 Apr 2004, Edward S. Peschko e...@pge.com wrote: On Tue, Apr 06, 2004 at 11:30:55AM +1000, Martin Pool wrote: On 4 Apr 2004, Edward S. Peschko e...@pge.com wrote: hey all, I was trying to compile binutils using ccache (version 2.3) and am getting the following errors: Are you using ccache to wrap Sun CC? Maybe you're hitting a bug in Sun CC. no, I'm definitely using gcc and the associated binutils. Compiling the first time works, compiling the second breaks with the above error. binutils 2.14.90.0.4 builds correctly for me on debian sid i386 with ccache 2.2. The second time through gets lots of cache hits and is very quick. -- Martin
[ccache] PRB: ccache directory not available
On 2 Dec 2003, heiko_el...@arburg.com wrote: ccache: version 2.2 OS: Windows XP Professional cygwin: latest version Hello, first of all: we like ccache - its a great utlility - thanks a lot! We're using ccache to share compiler output between several people. All works fine - but if the ccache file server is not available - it would be nice if ccache compile local instead of generating an error message like the following: ccache: failed to create //ad26080/ccache (No such host or network path) Perhaps a new flag or environement variable or per default fall back to compile on local host without try fetching/saving files to ccache directory. I searched the man pages for such an option - but without success! Perhaps you can add a feature like this in the next release. I think that should be on by default: just emit a warning and then compile without a cache. Probably a one line change. -- Martin linux.conf.au -- Adelaide, January 2004 -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://lists.samba.org/archive/ccache/attachments/20031203/ce520a4d/attachment.bin
[ccache] Fw: [NFS] nfs/mmap/rename file corruption
In summary: to make ccache work safely on an NFS filesystem, the filesystem must be exported with no_subtree_check. Otherwise data can be lost when it renames recently-written files. :-( -- Begin forwarded message: Date: 27 Aug 2003 21:37:38 -0400 From: Trond Myklebust trond.mykleb...@fys.uio.no To: Martin Pool m...@sourcefrog.net Cc: n...@lists.sourceforge.net Subject: Re: [NFS] nfs/mmap/rename file corruption == Martin Pool m...@sourcefrog.net writes: - ccache runs distcc with output to a temporary file - distcc opens, mmaps, writes to, munmaps, and closes the temporary file - distcc exits - ccache renames the temporary file to its proper location in the ccache - ccache opens the file read only, and reads from it Is this a rename from one directory to the other? If so, are you using the 'no_subtree_check' option on the server? Without the latter option enabled, I would indeed expect the behaviour that you describe. Cheers, Trond -- Martin
[ccache] lilypond doesn't compile with ccache (DEPENDENCIES_OUTPUT)
On 10 Aug 2003, Koblinger Egmont egm...@uhulinux.hu wrote: ./out/axis.o: axis.cc include/axes.hh include/string.hh \ include/arithmetic-operator.hh include/flower-proto.hh include/real.hh \ include/string-handle.hh include/string-handle.icc \ include/string-data.hh include/string-data.icc include/international.hh \ include/compare.hh include/string.icc but when ccache is used, this file has one more line: ./out/axis.o: axis.cc include/axes.hh include/string.hh \ include/arithmetic-operator.hh include/flower-proto.hh include/real.hh \ include/string-handle.hh include/string-handle.icc \ include/string-data.hh include/string-data.icc include/international.hh \ include/compare.hh include/string.icc ./out/axis.o: /home/egmont/.ccache/tmp.stdout.boci.5961.ii All the other *.dep files have a similar extra line appended which point to some ccache-internal filename, which means that somehow ccache fails to remain transparent for lilypond. The command that generates this .dep file is: DEPENDENCIES_OUTPUT=./out/axis.dep ./out/axis.o c++ -c -DHAVE_CONFIG_H -DSTRING_UTILS_INLINED -Iinclude -I./out -I../flower/include -I../flower/./out -O2 -finline-functions -g -O2 -finline-functions -g -Wall -W -Wmissing-prototypes -Wconversion -o out/axis.o axis.cc Thankyou for reporting this. It looks like the problem is that DEPENDENCIES_OUTPUT is read by both cpp and cc1, so it gets dependencies for both .h-.i and .i-.o. Almost certainly you only want the first one. You can work around this in lilypond by using -MD -MF rather than DEPENDENCIES_OUTPUT. I think ccache (and distcc) ought to strip DEPENDENCIES_OUTPUT before invoking the compiler, so that you only get the first dependencies line. Please take a look at this problem. It seems to me that ccache should fix the DEPENDENCIES_OUTPUT files after running gcc itself, but real developers should know it better than me :-) Another trivial workaround might be to let an existing DEPENDENCIES_OUTPUT env variable simply imply CCACHE_DISABLE. A small patch is attached. I don't think such severe measures are needed. Have a nice holiday, Thankyou! How did you know? :-) -- Martin
[ccache] ccache doesn't handle .i files
If I run ccache 2.2 on a .i file, it thinks it's not a C/C++ file and it doesn't cache it. This could be naively fixed by updating check_extension(), but it would be far better for ccache to know that it doesn't need to run cpp on such files. distcc does so. In particular this means that if ccache is run by distcc or distccd it will never hit. This is not a terribly big deal but might be nice to fix. -- Martin
[ccache] CCACHE_PREFIX included in hash
On 31 Oct 2003, Brian Poynor bri...@redback.com wrote: Is CCACHE_PREFIX intentionally included in the hash? Yes. For my project it means that objects built locally never match objects built remotely with CCACHE_PREFIX=distcc, even though they are identical, and I'd like to avoid duplicate cache entries. If interested, I have a patch which avoids hashing it. Currently it is implemented as an alternate environment variable, CCACHE_NHPREFIX. Please post it. -- Martin linux.conf.au -- Adelaide, January 2004
[ccache] Re: disk full and distcc/ccache weirdness
On 23 Sep 2002, Tim Potter t...@samba.org wrote: Hi Tridge. I've just had some distcc/ccache wierdness that caused my ccache to be corrupted. One of the machines in our DISTCC_HOSTS list filled up /tmp and a whole bunch of object files were created containing text like: DOMAIN_GRP:t(239,22)=(239,21) domain_grp_member_info:T(239,23)=s257name:(211,2),0,2048;attr:(0,11),2048,8;; DOMAIN_GRP_MEMBER:t(239,24)=(239,23) time_info:T(239,25)=s4time:(0,4),0,32;; The exact error returned was: Linking bin/smbd lib/fsusage.o: file not recognized: File truncated I haven't been able to reproduce it by filling up /tmp again and rebuilding so I'm not sure what sort of bug it is. )-: The bug is that execute() in execute.c only checks WEXITSTATUS(), not WIFSIGNALLED(). If the compiler exits with a signal ccache will proceed as if it succeeded, and therefore probably corrupt its cache. Old versions of distcc would raise SIGABRT if something strange like ENOSPC when writing a file happened. (New ones are practically perfect in every possible way. :-) -- Martin
[ccache] ccache - -frepo
On 1 Oct 2003, andrew andrew.sla...@discus.anu.edu.au wrote:
Hi,
Should ccache always ignore compiles with (g++) -frepo, since object
files might have to be rebuilt at link time, due to c++ template object
code placement? I bypassed caching for -frepo'd source files so my
template stuff would compile.
Yes, I think it should.
:)
andrew
diff -Naur ccache-2.3/ccache.c ccache-2.3-nofrepo/ccache.c
+++ ccache-2.3-nofrepo/ccache.c 2003-10-01 14:06:12.0 +1000
@@ -630,6 +630,7 @@
/* these are too hard */
if (strcmp(argv[i], -fbranch-probabilities)==0 ||
+ strcmp(argv[i], -frepo) == 0 ||
strcmp(argv[i], -M) == 0 ||
strcmp(argv[i], -MM) == 0 ||
strcmp(argv[i], -x) == 0) {
___
ccache mailing list
ccache@lists.samba.org
http://lists.samba.org/mailman/listinfo/ccache
--
Martin
[ccache] Re: [distcc] ccache doesn't handle .i files
On 23 Sep 2003, ee...@gmx.net wrote:
on Mon, 21 Jul 2003 23:52:45 -0700, Martin Pool wrote:
If I run ccache 2.2 on a .i file, it thinks it's not a C/C++ file and
it doesn't cache it.
This could be naively fixed by updating check_extension(), but it
would be far better for ccache to know that it doesn't need to run cpp
on such files. distcc does so.
In particular this means that if ccache is run by distcc or distccd it
will never hit.
This is not a terribly big deal but might be nice to fix.
imho it's a better way to combine ccache and distcc to ccache on the
distcc hosts after distributing, as compiler upgrades are handled
correctly this way.
That was what I meant by running it from distccd.
maybe this will do(I know it's kludgy but to nicely implement it some
redesign of ccache would be needed):
@@ -64,19 +70,23 @@
static struct {
char *extension;
char *i_extension;
+ int preprocess;
Okay.
+ if (preprocess) {
+ /* now the run */
args_add(args, -E);
args_add(args, input_file);
status = execute(args-argv, path_stdout, path_stderr);
@@ -327,6 +339,12 @@
failed();
}
+ } else {
+ copy_file(input_file, path_stdout);
+ x_asprintf(command, touch %s, path_stderr);
+ system(command);
+ free(command);
+ }
Huh?
--
Martin
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url :
http://lists.samba.org/archive/ccache/attachments/20030924/d47b8340/attachment.bin
