Re: [ccache] ccache miss increasing on making same build
Follow-up: We continued debugging the problem off-list. There were actually two problems: 1. The cache size was too small so a second build never even had a chance to get cache hits. 2. There was a generated header file with unstable content (due to Perl 5.18's randomization of hashes http://search.cpan.org/dist/perl-5.18.0/pod/perldelta.pod#Hash_randomization ). -- Joel On 11 August 2015 at 21:45, Joel Rosdahl j...@rosdahl.net wrote: Hi Vijay, For me, ccache is not working properly on compiling the already compiled build again. I can see ccache miss count increasing instead of ccache hit count. To be able to understand why, it would help to get a ccache log that contains: 1. A compilation of a source file X, leading to a cache miss (and storage of the result in the cache). 2. A repeated compilation of the same source file X, leading to a cache miss (where you expected a cache hit of the previous result). The log file you attached only contains cache misses for two *different* source files. However, one thing I noticed in your log is that the manifest file already exists. This likely means that one of the include files has changed content between the two compilations. To find out which one, get the path of the manifest from the ccache log file (search for Looking for object file hash in .../hash-length.manifest) and dump the manifest with this script: https://git.samba.org/?p=ccache.git;a=blob_plain;f=dump-manifest;hb=326bcbfda8709db5ad6c21ff6b0e62f0441d27d9 (Or, upgrade to ccache 3.2 and run ccache --dump-manifest path/to/the.manifest.) You can send the dumped manifest to me (or to the list if it's not too large) and I'll have a look. -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache version 3.2.3 has been released
Joel Rosdahl j...@rosdahl.net writes: I'm happy to announce ccache version 3.2.3. Hate to be the bearer of bad news, but this fails to build for me on late-model OS X: $ ./configure $ make gcc -DHAVE_CONFIG_H -DSYSCONFDIR=/usr/local/etc -I. -I. -g -O2 -Wall -W -c -o main.o main.c ... gcc -DHAVE_CONFIG_H -DSYSCONFDIR=/usr/local/etc -I. -I. -g -O2 -Wall -W -c -o conf.o conf.c make: *** No rule to make target `-lz', needed by `ccache'. Stop. The reason appears to be that the Makefile gets generated like so: $ grep extra_libs Makefile extra_libs = -lz ccache$(EXEEXT): $(ccache_objs) $(extra_libs) $(CC) $(all_cflags) -o $@ $(ccache_objs) $(all_ldflags) $(extra_libs) $(LIBS) test/main$(EXEEXT): $(base_objs) $(test_objs) $(extra_libs) $(CC) $(all_cflags) -o $@ $(base_objs) $(test_objs) $(all_ldflags) $(extra_libs) $(LIBS) and of course -lz isn't a valid dependency. Manually removing $(extra_libs) from these two dependency lists fixes it. I do not recall having had this problem the last time I built ccache, but that was 3.2.1, so I don't know whether it was introduced in 3.2.2 or 3.2.3. regards, tom lane ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache interrupt handling bug
Nadav Har'El n...@cloudius-systems.com writes: First of all, thanks. This indeed fixes the bug, and is exactly the first patch I tried to fix it this problem. I want to explain, though, why I ended up sending a different patch for this problem - a 4-line patch instead of this one-liner. The explanation is not very important, and I should probably have done this earlier, but perhaps someone would find it interesting. When the user types ^C, the operating system sends SIGINT to all processes belonging to the terminal's process group - and in particular it sends SIGINT to *both* the child compiler and ccache, separately. Your patch makes ccache exit as soon as it gets the SIGINT - but the child compiler might still be running for a while longer. This will usually be fine, but I thought the user can be surprised if he sees ccache (which he considers to be the compiler) exit, but ps shows the compiler is still running. This would be especially annoying if some hypothetical compiler trapped SIGINT deliberately, and continued to run long after ccache exits. Actually, that's a bug not just a cosmetic problem, because it introduces a race condition. Consider this sequence of events: * make launches ccache gcc -o test.o test.c * user types control-C * ccache receives SIGINT and exits * make does unlink(test.o) to clean up after failed compile step, which it thinks is done * gcc creates test.o * gcc receives SIGINT and exits We're left with a probably-broken test.o in the filesystem, despite make's attempt to clean up. Now, the window for this is pretty narrow, but I think it could happen: I don't believe that sending signals to multiple processes is atomic in most kernels. You really don't want ccache to exit until after all externally-observable things have stopped happening. (I recently fixed a somewhat related bug in Postgres :-(, which is why this caught my eye.) regards, tom lane ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache interrupt handling bug
Nadav Har'El n...@cloudius-systems.com writes: On Mon, Aug 17, 2015 at 12:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: * make launches ccache gcc -o test.o test.c * user types control-C * ccache receives SIGINT and exits * make does unlink(test.o) to clean up after failed compile step, which it thinks is done * gcc creates test.o Stupid question: Does ccache let gcc write to test.o directly, or does it write to some temporary file and then ccache copies it? Dunno, but that just moves the problem somewhere else no? For instance, in the code fragment you mentioned, I wonder whether it's sane to do clean_up_pending_tmp_files(); when gcc is still on the loose. regards, tom lane ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
[ccache] Compiler warnings in ccache 3.2.3
I got a few warnings about assignments of pointers to bool values.
I think these are legitimate gripes, because on platforms where bool
is only a byte wide, the net effect will be to assign the pointer's
low-order byte to the bool. There's at least a 1-in-256 chance of
a non-null pointer erroneously converting to a false boolean value
(maybe more than that, if the pointer is likely to be aligned ...).
I propose the attached patches to fix this.
regards, tom lane
diff -c -r orig/ccache-3.2.3/ccache.c ccache-3.2.3/ccache.c
*** orig/ccache-3.2.3/ccache.c Sun Aug 16 08:12:05 2015
--- ccache-3.2.3/ccache.c Sun Aug 16 14:59:46 2015
***
*** 1263,1269
compiler_is_clang(struct args *args)
{
char *name = basename(args-argv[0]);
! bool is = strstr(name, clang);
free(name);
return is;
}
--- 1263,1269
compiler_is_clang(struct args *args)
{
char *name = basename(args-argv[0]);
! bool is = strstr(name, clang) != NULL;
free(name);
return is;
}
diff -c -r orig/ccache-3.2.3/conf.c ccache-3.2.3/conf.c
*** orig/ccache-3.2.3/conf.c Sun Aug 16 08:12:05 2015
--- ccache-3.2.3/conf.c Sun Aug 16 15:01:12 2015
***
*** 58,64
char **value = (char **)result;
free(*value);
*value = subst_env_in_string(str, errmsg);
! return *value;
}
static bool
--- 58,64
char **value = (char **)result;
free(*value);
*value = subst_env_in_string(str, errmsg);
! return *value != NULL;
}
static bool
diff -c -r orig/ccache-3.2.3/language.c ccache-3.2.3/language.c
*** orig/ccache-3.2.3/language.c Sun Aug 16 08:12:05 2015
--- ccache-3.2.3/language.c Sun Aug 16 15:01:01 2015
***
*** 149,155
bool
language_is_supported(const char *language)
{
! return p_language_for_language(language);
}
bool
--- 149,155
bool
language_is_supported(const char *language)
{
! return p_language_for_language(language) != NULL;
}
bool
diff -c -r orig/ccache-3.2.3/util.c ccache-3.2.3/util.c
*** orig/ccache-3.2.3/util.c Sun Aug 16 08:12:05 2015
--- ccache-3.2.3/util.c Sun Aug 16 15:00:41 2015
***
*** 1638,1644
if (curly) {
if (*q != '}') {
*errmsg = format(syntax error: missing '}' after \%s\, p);
! return NULL;
}
}
--- 1638,1644
if (curly) {
if (*q != '}') {
*errmsg = format(syntax error: missing '}' after \%s\, p);
! return false;
}
}
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache interrupt handling bug
On Sun, Aug 9, 2015 at 11:23 PM, Joel Rosdahl j...@rosdahl.net wrote:
Hi Nadav,
(Sorry for the delayed reply.)
Hi, I found a bug in ccache, which makes it impossible to
correctly interrupt a compilation with a control-C (I tried this on Linux).
[...]
Thanks for the bug report and analysis. This will be fixed in ccache 3.2.3.
Regards,
-- Joel
Thanks. I see now the fix in 3.2.3, which if I understand correctly, is
simply:
--- a/ccache.c
+++ b/ccache.c
@@ -341,6 +341,7 @@ signal_handler(int signo)
{
(void)signo;
clean_up_pending_tmp_files();
+ _exit(1);
}
First of all, thanks. This indeed fixes the bug, and is exactly the first
patch I tried to fix it this problem.
I want to explain, though, why I ended up sending a different patch for
this problem - a 4-line patch instead of this one-liner. The explanation is
not very important, and I should probably have done this earlier, but
perhaps someone would find it interesting.
When the user types ^C, the operating system sends SIGINT to all processes
belonging to the terminal's process group - and in particular it sends
SIGINT to *both* the child compiler and ccache, separately.
Your patch makes ccache exit as soon as it gets the SIGINT - but the child
compiler might still be running for a while longer. This will usually be
fine, but I thought the user can be surprised if he sees ccache (which he
considers to be the compiler) exit, but ps shows the compiler is still
running. This would be especially annoying if some hypothetical compiler
trapped SIGINT deliberately, and continued to run long after ccache exits.
My approach, was to let ccache continue after the SIGINT, and continue to
wait (as it always does) for the child compiler to finish. The child, who
received a SIGINT too, willl indeed finish quickly. The code already
recognizes this case of the child killed by a signal, and returns status ==
-1. So all my 4-line patch did was to not restart the compiler if it died
with a status = -1 (i.e., a signal) and not an ordinary failure. But ccache
does not exit immediately on a signal - just after the child compiler exits.
Again, I guess this destinction is not very important, but I just wanted to
point it out in case anybody thinks it is important.
--
Nadav Har'El
n...@cloudius-systems.com
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] ccache version 3.2.3 has been released
On 16 Aug 2015 13:39, Tom Lane wrote: $ grep extra_libs Makefile extra_libs = -lz ccache$(EXEEXT): $(ccache_objs) $(extra_libs) $(CC) $(all_cflags) -o $@ $(ccache_objs) $(all_ldflags) $(extra_libs) $(LIBS) test/main$(EXEEXT): $(base_objs) $(test_objs) $(extra_libs) $(CC) $(all_cflags) -o $@ $(base_objs) $(test_objs) $(all_ldflags) $(extra_libs) $(LIBS) and of course -lz isn't a valid dependency. that's not really true. make will expand it internally into paths like /usr/lib/libz.so. that said, there's no value anymore in putting -l flags into the deps list and should just omit it from builds. -mike signature.asc Description: Digital signature ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
