Re: [ccache] ccache has issues with current Apple compilers

2014-06-01 Thread Joel Rosdahl

 I tried to install ccache 3.1.9 on a current Mac system (OS X 10.9.3,
 Xcode 5.1.1).  Configure and compile went cleanly, with the exception
 of two probably-harmless warnings: [...]

Thanks for the patch. The build warning has however already been fixed in
another way in ccache (not released yet, though). You can get hold of the
latest development source code here:

 I did some further experiments and concluded that ccache is in fact kind
 of broken: when I try to compile a large project with it, I get thousands
 of weird warnings that do not appear when using gcc directly (many of them
 the same kind of unused -I argument bleat shown above, but others are
 warnings about C constructs that I don't normally see any warnings about).

As mentioned in the ccache manual: Only works with GCC and compilers that
behave similar enough. You are not using GCC but clang, which does not
behave similar enough...

As a workaround, you can set CCACHE_CPP2, as mentioned by Lubos Lunak. In
the yet unreleased ccache 3.2 version, that shouldn't be necessary.

 In any case, surely it's unexpected that ccache would change
 the compiler's warning behavior?

It's expected if the compiler emits different warnings when compiling from
source code compared with compiling from preprocessed source code since
that's how ccache operates.

-- Joel

On 31 May 2014 20:24, Tom Lane wrote:

 I tried to install ccache 3.1.9 on a current Mac system (OS X 10.9.3,
 Xcode 5.1.1).  Configure and compile went cleanly, with the exception
 of two probably-harmless warnings:

 manifest.c:223:2: warning: shift count = width of type
 READ_INT(1, version);
 manifest.c:150:10: note: expanded from macro 'READ_INT'
 (var) = 8; \
   ^   ~
 manifest.c:230:2: warning: shift count = width of type
 READ_INT(1, mf-hash_size);
 manifest.c:150:10: note: expanded from macro 'READ_INT'
 (var) = 8; \
   ^   ~
 2 warnings generated.

 I made this change to silence that, since I think that the C standard
 allows compilers to consider the case undefined behavior:

 $ diff -c manifest.c~ manifest.c
 *** manifest.c~ Sun Jan  6 11:57:59 2013
 --- manifest.c  Sat May 31 13:42:53 2014
 *** 147,154 
 if (ch_ == EOF) { \
 goto error; \
 } \
 !   (var) = 8; \
 !   (var) |= ch_  0xFF; \
 } \
 } while (0)

 --- 147,153 
 if (ch_ == EOF) { \
 goto error; \
 } \
 !   (var) = (((unsigned int) (var))  8) | (ch_ 
 0xFF); \
 } \
 } while (0)

 but it's probably not really an issue.  However, make test refuses
 to do anything much:

 PASSED: 178 assertions, 64 tests, 9 suites
 CC='gcc' ./
 WARNING: Compiler gcc not supported (version: Configured with:
 -- not running tests

 I looked at and concluded that it simply didn't recognize the
 output of gcc --version, which looks like this:

 $ gcc --version
 Configured with: --prefix=/Applications/
 Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
 Target: x86_64-apple-darwin13.2.0
 Thread model: posix

 So I diked out that check from the test script and tried again.
 The first half dozen test suites went OK, but in the basedir
 testsuite I got a lot of chatter:

 clang: warning: argument unused during compilation: '-I
 clang: warning: argument unused during compilation: '-I
 clang: warning: argument unused during compilation: '-I include'
 clang: warning: argument unused during compilation: '-I include'
 clang: warning: argument unused during compilation: '-I include'
 [ much more in the same vein... ]

 and then the pch testsuite failed altogether:

 starting testsuite pch
 SUITE: pch, TEST: no -fpch-preprocess, -include - Expected cache
 miss to be 1, got 0
 cache directory
 cache hit (direct) 0
 cache hit (preprocessed)   0
 cache miss 0
 preprocessor error 1
 files in cache 0
 cache size 

Re: [ccache] Support for color diagnostics

2014-06-01 Thread Lubos Lunak
On Friday 29 of November 2013, Lubos Lunak wrote:

  the attached patch adds ccache support for compiler color diagnostics
 (also reported by somebody as #10075).

 Ping? Any official comments on the patch? I've been using the patch for 
half a year by now without problems.

  Clang automatically uses colors for output automatically if used in
 terminal. Ccache's redirecting to a file disables this. GCC 4.8 has got a
 similar support, except that it apparently requires also $GCC_COLORS or an
 explicit option.

  The patch detects if the compiler would use colors if used without ccache
 and explicitly forces an option to achieve this. Note that I do not have
 GCC 4.8 here, so I tested with Clang's alias and the GCC_COLORS support is
 done based on documentation.


 - GCC developers decided to roll their own name for the option when
 introducing it. Clang has an alias for the GCC way, but versions predating
 that obviously can't support it, so it's necessary to detect the compiler.
 As ccache doesn't do that (and I don't find it worth much effort, as it
 can't be 100% reliable anyway), the code merely guesses from the binary
 name. If the compiler used will be e.g. the 'cc' symlink, there'll be no
 colors. No big deal.

 - Since the stderr is different, obviously compiling with and without
 colors has different results as well. That means that such a compile
 is duplicated. It's hopefully not such a common case, although it's
 perfectly possible. I don't know if it's worth the effort to try to be
 smart here. A possibly simple improvement could be to search the cache with
 and without the option set and if stderr is empty, reuse the result
 regardless of the option. I'm not quite sure where exactly this should
 happen in the code.

  I expect it'd make sense to add $CCACHE_NOCOLORS to disable this support?

  I can also create manpage section for this color support, but I first
 wanted to check here with the code.

 Lubos Lunak
From cacb14929748ae93eacefcfa194aa93689d217eb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lubo=C5=A1=20Lu=C5=88=C3=A1k?=
Date: Fri, 29 Nov 2013 12:14:03 +0100
Subject: [PATCH] support compiler color diagnostics if possible

Clang and GCC (starting with 4.9) support color output if possible,
but since ccache redirects stderr to a file, they detect the output
is not a terminal and do not enable colors. Try to detect if colors
would be used and force colors explicitly.

- Compiles with and without colors are considered different from each
  other (so they are duplicated).
- GCC decided to roll its own name for the option, so it's necessary
  to guess which compiler is actually used.
 ccache.c | 77 
 1 file changed, 77 insertions(+)

diff --git a/ccache.c b/ccache.c
index c395fad..e122c50 100644
--- a/ccache.c
+++ b/ccache.c
@@ -1065,6 +1065,24 @@ hash_compiler(struct mdfour *hash, struct stat *st, const char *path,
+ * Note that these compiler checks are unreliable, so nothing should hard-depend on them.
+ */
+static bool compiler_is_clang()
+	const char* name = strrchr( orig_args-argv[ 0 ], '/' );
+	name = name ? name + 1 : orig_args-argv[ 0 ];
+	return strstr( name, clang ) != NULL;
+static bool compiler_is_gcc()
+	const char* name = strrchr(orig_args-argv[ 0 ], '/' );
+	name = name ? name + 1 : orig_args-argv[ 0 ];
+	return strstr(name, gcc) != NULL || strstr(name, g++) != NULL;
  * Update a hash sum with information common for the direct and preprocessor
  * modes.
@@ -1128,6 +1146,15 @@ calculate_common_hash(struct args *args, struct mdfour *hash)
+	/* Possibly hash GCC_COLORS (for color diagnostics). */
+	if (compiler_is_gcc()) {
+		const char* gcc_colors = getenv(GCC_COLORS);
+		if (gcc_colors != NULL) {
+			hash_delimiter(hash,gcccolors);
+			hash_string(hash, gcc_colors);
+		}
+	}
@@ -1633,6 +1660,13 @@ is_precompiled_header(const char *path)
 	   || str_eq(get_extension(path), .pth);
+static bool color_output_possible()
+	const char* term_env = getenv(TERM);
+	return isatty(STDERR_FILENO)  term_env  strcasecmp(term_env, DUMB) != 0;
  * Process the compiler options into options suitable for passing to the
  * preprocessor and the real compiler. The preprocessor options don't include
@@ -1661,6 +1695,7 @@ cc_process_args(struct args *args, struct args **preprocessor_args,
 	int argc;
 	char **argv;
 	bool result = true;
+	bool found_color_diagnostics = false;
 	expanded_args = args_copy(args);
 	stripped_args = args_init(0, NULL);
@@ -2017,6 +2052,26 @@ cc_process_args(struct args *args, struct args **preprocessor_args,
+		if (str_eq(argv[i], -fcolor-diagnostics)
+		|| str_eq(argv[i], -fno-color-diagnostics)
+		|| str_eq(argv[i], -fdiagnostics-color)
+		|| str_eq(argv[i], -fdiagnostics-color=always)
+		|| str_eq(argv[i],