Hi Fergus! The PCH patch I made is (as I see it) incompatible with the DistCC3 pump mode - it requires a local preprocessor run to locate the PCH file. The proxy-file-mode (create a local dummy include for the PCH in case client and server run incompatble GCC binaries) seams a bit odd if we have pump mode, which eliminates the local preprocessor run completely.
A better approach (IMHO): 1. Add hash based file caching for _all_ files (similar to the way it's done in my PCH patch) 2. Pump the PCH file together with the source code - the prescan code could easily be extended to include PCH files if present. If done like this, the compiler still has a chance to fall back if the PCH file is not compatible with the command line options passed (maybe because some files are compiled with different options than others). This is a major flaw in my PCH patch. Due to the size of the PCH files, hash based file caching should be done first. I will do that, but that will take time because I am a bit busy at present. Kind regards, Sascha Fergus Henderson wrote: > Hi Sascha, > > Thanks for the patch. There are going to be some non-trivial issues > merging this patch in with the other changes in distcc 3.0, unfortunately. > For example, both distcc 3.0 and this patch add protocol version 3. > > I'm hoping that the speed benefits of distcc 3.0 will help persuade > you that it's worth your effort to merge this into distcc 3.0 :) > > Cheers, > Fergus. > > On Wed, Jun 4, 2008 at 12:20 PM, Sascha Demetrio <[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> wrote: > > Hi! > > The attached patch adds support for using precompiled headers (PCHs) > with distcc distributed builds. The patch is not ready for end-user > consumption, as it includes no documentation on how it works > (except for > comments in the code). > > > IMPORTANT: If you plan to use this patch, you'll also need the > dcc_unlock() patch I posted to this list a few hours ago. > > > The protocol has been changed to version 3. With this change, the > client > sends an additional word containing a set of flags. In these > flags, the > client specifies if file transfer compression should be enabled and if > the preprocessed file refers to a PCH. If both flags are clear, the > protocol after that remains unchanged, > > PCH support for the server is enabled by adding the ,pch option to the > host specification in DISTCC_HOSTS. This option may be combined > with the > ,lzo option. > > The client PCH support is enabled either if the compiler command line > contains the option '-fpch-preprocess' or if the DISTCC_PCH > environment > variable is set to non-zero. > > If server and client have the same compiler binary installed, then the > PCH patch works like this: > > The preprocessed output file is scanned for '#pragma GCC > pch_preprocess > "filename"'. In this line, the quoted file name is substituted by the > hex representation of the PCH file's MD5 hash. The client keeps a > pchfile+mtime to MD5-hash mapping in the ~/.distcc/state/pch file. > > After receiving the preprocessed file, the server has a chance to > query > the PCH file. It will do so if the PCH file is not already in a > PCH file > cache. In case of a cache hit, the server simply sends a 'no thanks' > message to the client and starts the compilation. The server looks for > the PCH #pragma and replaces the MD5 hash with the path to the locally > cached PCH file. > > > Things get a bit more complicated if client and server run > incompatible > GCC binaries. This is handled by compiling the PCH files on the server > and creating a local PCH proxy file. Here's the details: > > If the DISTCC_PCH_REMOTE variable is set to non-zero, then the client > will recognize the header file as an input file and send it to the > server for remote pre-compilation. A PCH file contains information > about > the preprocessor state, so the file sent to the server must > include this > state information. This is done by setting the -dD option when the > header is being preprocessed. All #define's and #undef's are moved to > the end of the preprocessed file (except for built-in macros and > macros > defined on the command line). This is required to make sure that the > macros do not interfere with the preprocessed file contents (which > will > have all macros expanded) - double expansion would lead to incorrect > results in some cases. > > The DISTCC_PCH_REMOTE option is useful even if server and client share > the same compiler binary, because it offloads the PCH pre-compilation > from the client to the server. That's the rationale for providing a > separate distcc option for that. > > Limitation: The header file is recognized as an input file only if the > DISTCC_PCH_REMOTE option is set _and_ the header file immediately > follows the -c switch. > > For PCHs to work, the local preprocessor must be able to extract the > preprocessor state from the PCH - which is not possible if the PCH > file > has been created by a different compiler binary. To solve this, a > local > proxy file is created by setting DISTCC_PCH_PROXY to non-zero. > > The proxy file is created at the same location as the PCH file, > with the > .gch suffix stripped. To prevent accidental clobbering of a real > source > file, this is done only if that file does not exist, so a make > rule for > creating the PCH file should remove the proxy file before starting the > remote PCH pre-compilation. > > The proxy file contains the '#pragma GCC pch_preprocess "file"' line > followed by all #define's and #undef's captured from the PCH > preprocessing. The build environment should be set up in such a > way that > the directory containing the proxy file is scanned before the > directory > containing the real file. Obviously, the compiler should be caller > _without_ the -fpch-preprocess option, because the preprocessor should > leave the local PCH file alone. > > Finally, the DISTCC_PCH option must be set to non-zero when > compiling - > there's no -fpch-preprocess option in the command line, so distcc can > not automatically recognize that we're doing PCHs. > > > This PCH patch is in everyday use here on site and works pretty > well so > far. BUT: The change is very intrusive and probably (certainly) > contains > bugs. There has been little review of the code, so expect things to be > broken. > > > Kind regards, > Sascha Demetrio > > ___ > This message contains confidential information and is intended > only for the individual named. If you are not the named addressee > you should not disseminate, distribute or copy this e-mail. Please > notify the sender immediately by e-mail if you have received this > e-mail by mistake and delete this e-mail from your system. E-mail > transmission cannot be guaranteed to be secure or error-free as > information could be intercepted, corrupted, lost, destroyed, > arrive late or incomplete, or contain viruses. The sender > therefore does not accept liability for any errors or omissions in > the contents of this message, which arise as a result of e-mail > transmission. If verification is required please request a > hard-copy version. Crytek GmbH - http://www.crytek.com > > > > > -- > Fergus Henderson <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> __ distcc mailing list http://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
