Re: [ccache] [PATCH] Some patches for Xcode support
On Sun, Jun 13, 2010 at 22:32, Joel Rosdahl j...@rosdahl.net wrote: I see. distcc 2.18.3 (the oldest release I can find on distcc.org) seems to know about .mi/.mii. Do you think it's acceptable to upgrade to a newer distcc to fix this? It would be nice to avoid having a workaround for older distcc versions. The bug is actually more subtle: distcc has global variables indicating whether whether it has seen an -x option and the last value of its argument, and forgets to clear them between running multiple jobs on the server, so it assumes an incorrect language override when no -x option is present. This bug is in the Apple-patched distcc version that's shipped with the 10.5 Developer Tools (source at http://opensource.apple.com/tarballs/distcc/distcc-1201.tar.gz), so it's -- sadly -- not a really old version. With ccache eating the -x option, I don't see an easy workaround for this bug, and upgrading distcc would mean having to upgrade it on all machines at once for compatibility reasons :-( Bernhard. ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
On 2010-06-14 19:13, Bernhard Bauer wrote: The bug is actually more subtle: distcc has global variables indicating whether whether it has seen an -x option and the last value of its argument, and forgets to clear them between running multiple jobs on the server, so it assumes an incorrect language override when no -x option is present. This bug is in the Apple-patched distcc version that's shipped with the 10.5 Developer Tools (source at http://opensource.apple.com/tarballs/distcc/distcc-1201.tar.gz), so it's -- sadly -- not a really old version. I see. Thanks for the investigation. In the light of this, I think that the work-around is acceptable, so I've committed a variant of your patch now. Please try it out! -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
On 2010-06-11 16:33, Bernhard Bauer wrote: Also, I added the -x option for compiling preprocessed code again if there was an -x option on the original command line. Otherwise, when compiling Objective-C code on a different machine, distcc wouldn't recognize the code as preprocessed Objective-C. Is this because distcc doesn't know about .mi/.mii being precompiled code? Why is this a problem? And in that case, don't you need to add -x unconditionally (not only if explicit_language has been set)? -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
On Sun, Jun 13, 2010 at 18:38, Joel Rosdahl j...@rosdahl.net wrote: On 2010-06-11 15:18, Bernhard Bauer wrote: I finally got around to rewriting the third patch. I added a special case to process_args that matches distcc's behavior (if the first argument to distcc is neither a source/object file nor an option, it's treated as the compiler). Thanks. One problem with the patch is that if ccache is invoked as distcc, ccache will hash the mtime (or content) of distcc instead of the compiler when trying to detect compiler changes. (See CCACHE_COMPILERCHECK in calculate_common_hash().) Also, it feels a bit strange to add a special case for distcc, so I would like to understand more about the problem to solve. You said earlier that distcc is called directly by xcodebuild and that you're masquerading ccache as distcc via a symlink. Is the following a correct guess of the setup? 1. /X/distcc is a symlink to ccache. 2. xcodebuild runs /X/distcc compiler arguments. 3. ccache determines (with your patch) that it's invoked as distcc. 4. ccache uses distcc compiler arguments as the compiler command and the real distcc is found in PATH. Yes, that's the setup. A small additional detail is that the compiler is also called with an absolute path, so without the patch ccache determines that it's a valid file and assumes it's an input file. If so, wouldn't an alternative solution be to replace /X/distcc with a script like this: #!/bin/sh CCACHE_PREFIX=distcc exec ccache $@ That should work, provided PATH is modified to exclude /X, otherwise ccache would call this script again when calling distcc (because it's not a symlink to ccache). As a bonus, this will make ccache call the preprocessor as compiler -E instead of distcc compiler -E, which avoids some extra processing. I think the overhead from distcc alone is negligible, however running the compiler directly frees up distcc slots on localhost, of which there is a limited number (which could increase throughput because more jobs are being preprocessed, or decrease it because of thrashing). I'll try to find out if it helps performance, and come back complaining to you if it doesn't ;-) Also, I added the -x option for compiling preprocessed code again if there was an -x option on the original command line. Otherwise, when compiling Objective-C code on a different machine, distcc wouldn't recognize the code as preprocessed Objective-C. Is this because distcc doesn't know about .mi/.mii being precompiled code? Why is this a problem? And in that case, don't you need to add -x unconditionally (not only if explicit_language has been set)? I had distcc failing complaining about a compiler option that's specific to Objective-C, but only when running the job on another host. My guess is that distcc (or at least the old version Apple is shipping) doesn't recognize .mi/.mii as being Objective-C files, so on the other host it writes them to a temp file ending in .i/.ii before calling gcc, then gcc assumes they're plain c/c++ files. But yeah, in that case I probably need -x always, I just didn't want to muck around with the command line options any more than necessary. Bernhard. ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
On 2010-06-13 20:22, Bernhard Bauer wrote: On Sun, Jun 13, 2010 at 18:38, Joel Rosdahl j...@rosdahl.net wrote: [...] #!/bin/sh CCACHE_PREFIX=distcc exec ccache $@ That should work, provided PATH is modified to exclude /X, otherwise ccache would call this script again when calling distcc (because it's not a symlink to ccache). Right. Or use an absolute path to the real distcc in CCACHE_PREFIX. [...] I'll try to find out if it helps performance, and come back complaining to you if it doesn't ;-) :-) Yes, it could be hard to measure any difference. I had distcc failing complaining about a compiler option that's specific to Objective-C, but only when running the job on another host. My guess is that distcc (or at least the old version Apple is shipping) doesn't recognize .mi/.mii as being Objective-C files, so on the other host it writes them to a temp file ending in .i/.ii before calling gcc, then gcc assumes they're plain c/c++ files. But yeah, in that case I probably need -x always, I just didn't want to muck around with the command line options any more than necessary. I see. distcc 2.18.3 (the oldest release I can find on distcc.org) seems to know about .mi/.mii. Do you think it's acceptable to upgrade to a newer distcc to fix this? It would be nice to avoid having a workaround for older distcc versions. -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
Also, I added the -x option for compiling preprocessed code again if
there was an -x option on the original command line. Otherwise, when
compiling Objective-C code on a different machine, distcc wouldn't
recognize the code as preprocessed Objective-C.
Bernhard.
On Fri, Jun 11, 2010 at 15:18, Bernhard Bauer bau...@google.com wrote:
Hi Joel,
I finally got around to rewriting the third patch. I added a special
case to process_args that matches distcc's behavior (if the first
argument to distcc is neither a source/object file nor an option, it's
treated as the compiler).
You can find the patch at http://github.com/sheepmaster/ccache .
Bernhard.
On Mon, May 17, 2010 at 14:27, Bernhard Bauer bau...@google.com wrote:
On Sunday, May 16, 2010, Joel Rosdahl wrote:
On 2010-05-12 15:52, Bernhard Bauer wrote:
attached are some patches needed to make compiling under Mac OS X with
Xcode/distcc work.
Thanks!
0001-Add-Objective-C-to-the-mapping-from-source-files-to-.patch
Sure. Applied.
0002-Add-fpreprocessed-option-for-compiling-preprocessed-.patch
Is this needed because GCC doesn't add -fpreprocessed implicitly like it
does for .i/.ii/.mi files?
The patch looks OK, except that it seems that -fpreprocessed was added
in GCC 3.0, so GCC 2.95 wouldn't be supported anymore if we add the flag
unconditionally. I don't know if anyone still uses GCC 2.95, but maybe
we could do it like this instead:
--- a/ccache.c
+++ b/ccache.c
@@ -497,6 +497,14 @@ static void to_cache(ARGS *args)
putenv(DEPENDENCIES_OUTPUT);
if (compile_preprocessed_source_code) {
+ char *p = strrchr(i_tmpfile, '.');
+ if (p strcmp(p, .mii) == 0) {
+ /*
+ * GCC doesn't add -fpreprocessed implicitly for
.mii
+ * files, so we have to add it ourselves.
+ */
+ args_add(args, -fpreprocessed);
+ }
args_add(args, i_tmpfile);
} else {
args_add(args, input_file);
What do you think?
Or maybe it got confused by the -x option added by xcodebuild (for
which I used Dirk Best's preliminary patch, so I didn't repeat it
here). I'll find out which one is the case.
0003-Fix-a-bug-where-a-compiler-invocation-with-distcc-di.patch
I'm don't think that's a good idea, since the executable flag can be set
on source code files as well, either by mistake or because the file
system (for instance vfat) may display all files as executable.
Maybe you set CCACHE_PREFIX to distcc instead? See
.
The problem is that distcc is called directly by xcodebuild, so I'm
masquerading ccache as distcc via a symlink. Maybe the proper way here
is to change the command line parsing so that it can cope with an
additional argument when called as distcc.
Bernhard.
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
On 2010-05-25 00:58, Bernhard Bauer wrote: Turns out, the -x option was overriding the language auto-detection based on the filename. I ended up rewriting the -x patch similar to the way you suggested to Dirk, so now it supports properly changing the language for preprocessed files, using a table mapping from file extension to language, and one mapping from language to preprocessed language (and it incorporates the first patch from my original mail). You can find the patch at http://github.com/sheepmaster/ccache/commits/gcc_x . Thanks! Applied with some additional polishing. -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
On Mon, May 17, 2010 at 05:27, Bernhard Bauer bau...@google.com wrote:
On Sunday, May 16, 2010, Joel Rosdahl wrote:
On 2010-05-12 15:52, Bernhard Bauer wrote:
attached are some patches needed to make compiling under Mac OS X with
Xcode/distcc work.
Thanks!
0001-Add-Objective-C-to-the-mapping-from-source-files-to-.patch
Sure. Applied.
0002-Add-fpreprocessed-option-for-compiling-preprocessed-.patch
Is this needed because GCC doesn't add -fpreprocessed implicitly like it
does for .i/.ii/.mi files?
The patch looks OK, except that it seems that -fpreprocessed was added
in GCC 3.0, so GCC 2.95 wouldn't be supported anymore if we add the flag
unconditionally. I don't know if anyone still uses GCC 2.95, but maybe
we could do it like this instead:
--- a/ccache.c
+++ b/ccache.c
@@ -497,6 +497,14 @@ static void to_cache(ARGS *args)
putenv(DEPENDENCIES_OUTPUT);
if (compile_preprocessed_source_code) {
+ char *p = strrchr(i_tmpfile, '.');
+ if (p strcmp(p, .mii) == 0) {
+ /*
+ * GCC doesn't add -fpreprocessed implicitly for .mii
+ * files, so we have to add it ourselves.
+ */
+ args_add(args, -fpreprocessed);
+ }
args_add(args, i_tmpfile);
} else {
args_add(args, input_file);
What do you think?
Or maybe it got confused by the -x option added by xcodebuild (for
which I used Dirk Best's preliminary patch, so I didn't repeat it
here). I'll find out which one is the case.
Turns out, the -x option was overriding the language auto-detection
based on the filename. I ended up rewriting the -x patch similar to
the way you suggested to Dirk, so now it supports properly changing
the language for preprocessed files, using a table mapping from file
extension to language, and one mapping from language to preprocessed
language (and it incorporates the first patch from my original mail).
You can find the patch at http://github.com/sheepmaster/ccache/commits/gcc_x .
Bernhard.
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
On Sunday, May 16, 2010, Joel Rosdahl wrote:
On 2010-05-12 15:52, Bernhard Bauer wrote:
attached are some patches needed to make compiling under Mac OS X with
Xcode/distcc work.
Thanks!
0001-Add-Objective-C-to-the-mapping-from-source-files-to-.patch
Sure. Applied.
0002-Add-fpreprocessed-option-for-compiling-preprocessed-.patch
Is this needed because GCC doesn't add -fpreprocessed implicitly like it
does for .i/.ii/.mi files?
The patch looks OK, except that it seems that -fpreprocessed was added
in GCC 3.0, so GCC 2.95 wouldn't be supported anymore if we add the flag
unconditionally. I don't know if anyone still uses GCC 2.95, but maybe
we could do it like this instead:
--- a/ccache.c
+++ b/ccache.c
@@ -497,6 +497,14 @@ static void to_cache(ARGS *args)
putenv(DEPENDENCIES_OUTPUT);
if (compile_preprocessed_source_code) {
+ char *p = strrchr(i_tmpfile, '.');
+ if (p strcmp(p, .mii) == 0) {
+ /*
+ * GCC doesn't add -fpreprocessed implicitly for .mii
+ * files, so we have to add it ourselves.
+ */
+ args_add(args, -fpreprocessed);
+ }
args_add(args, i_tmpfile);
} else {
args_add(args, input_file);
What do you think?
Or maybe it got confused by the -x option added by xcodebuild (for
which I used Dirk Best's preliminary patch, so I didn't repeat it
here). I'll find out which one is the case.
0003-Fix-a-bug-where-a-compiler-invocation-with-distcc-di.patch
I'm don't think that's a good idea, since the executable flag can be set
on source code files as well, either by mistake or because the file
system (for instance vfat) may display all files as executable.
Maybe you set CCACHE_PREFIX to distcc instead? See
.
The problem is that distcc is called directly by xcodebuild, so I'm
masquerading ccache as distcc via a symlink. Maybe the proper way here
is to change the command line parsing so that it can cope with an
additional argument when called as distcc.
Bernhard.
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] [PATCH] Some patches for Xcode support
On 05/16/2010 12:40 PM, Joel Rosdahl wrote: The patch looks OK, except that it seems that -fpreprocessed was added in GCC 3.0, so GCC 2.95 wouldn't be supported anymore if we add the flag unconditionally. I don't know if anyone still uses GCC 2.95, but maybe we could do it like this instead: Hi, Just to say that we still use gcc 2.95 :-) - Samuel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
