Thanks Jeff for the proposal! Faster builds for Mesos have been a long awaited feature, so great to see some real traction here.
Regarding benchmarks, would it be possible to have benchmarks (for clean build and incremental build) with 1) PCH only change 2) stout non-header-only and 3) 1+2 ? Not sure how easy it is to prototype these to get benchmarks, but I think having those numbers would clearly show us which one which we should prioritize? Thanks, On Wed, Feb 15, 2017 at 11:34 AM, Neil Conway <neil.con...@gmail.com> wrote: > Hi Jeff, > > Gotcha -- I just wanted to understand the tradeoffs here. > > I'd definitely prefer an approach in which we include "<xyz>" in both > "mesos_common.hpp" and each individual file that logically depends on > "<xyz>". This makes clear the dependencies between modules and also > makes it easy to disable building with PCH (see also the > recommendations in [1]). If the only reason avoid this is the cost of > the repeated include, it would be important to see benchmarks that > justify this. > > BTW, I think it's important that we script/automate this as far as > possible, e.g., using a script to decide which headers are included > often enough to justify being included in the PCH. This should avoid > the PCH getting out of date, as well as innumerable arguments down the > road about whether header X warrants being added to the PCH :) > > Overall, sounds cool to me! Faster builds would be fantastic. > > Neil > > [1] http://gamesfromwithin.com/the-care-and-feeding-of-pre- > compiled-headers > > On Wed, Feb 15, 2017 at 11:26 AM, Jeff Coffler > <jeff.coff...@microsoft.com.invalid> wrote: > > Ni Neil, > > > > What you're saying is essentially correct. If mesos_common.h includes a > bunch of, well, "common" stuff, and everybody includes mesos_common.h, then > those files will, by definition, have a least some number of items that > they didn't need. > > > > Since PCH works on both Windows and Linux, I don't think this is a "bad > thing". It's a trade-off. Is a (what I believe to be) very significant > speed-up in compile speed "worth it"? (Obviously, since I submitted the > proposal, I think so. But this is a very valid point). > > > > Yes, header guards will help, but header guards are not free. I would > rather not include a really large set of headers (say, windows.h, or stout) > multiple times, expecting header guards to make them fast. I'd rather just > include them once, in mesos_common.h. And this would also yield the > greatest performance enhancement as well. > > > > I'm working on getting some hard numbers for a subset of Mesos. Once we > have some hard comparisons with compiler performance (with and without > PCH), we can address this much more practically. > > > > /Jeff > > > > > > -----Original Message----- > > From: Neil Conway [mailto:neil.con...@gmail.com] > > Sent: Wednesday, February 15, 2017 11:13 AM > > To: dev <dev@mesos.apache.org> > > Subject: Re: Proposal for Mesos Build Improvements > > > > On Tue, Feb 14, 2017 at 11:28 AM, Jeff Coffler < > jeff.coff...@microsoft.com.invalid> wrote: > >> For efficiency purposes, if a header file is included by 50% or more of > the source files, it should be included in the precompiled header. If a > header is included in fewer than 50% of the source files, then it can be > separately included (and thus would not benefit from precompiled headers). > Note that this is a guideline; even if a header is used by less than 50% of > source files, if it's very large, we still may decide to throw it in the > precompiled header. > > > > It seems like this would have the effect of creating many false > > dependencies: if file X doesn't currently include header Y but Y is > included in the precompiled header, the symbols in Y will now be visible > when X is compiled. It would also mean that X would need to be recompiled > when Y changes. > > > > Related: the current policy is that headers and implementation files > should try to include all of their dependencies, without relying on > transitive includes. For example, if foo.cpp includes bar.hpp, which > includes <vector>, but foo.cpp also uses <vector>, both foo.cpp and bar.hpp > should "#include <vector>". Adopting precompiled headers would mean making > an exception to this policy, right? > > > > I wonder if we should instead use headers like: > > > > <- mesos_common.h -> > > #include <a> > > #include <b> > > #include <c> > > > > <- xyz.cpp, which needs headers "b" and "d" -> #include "mesos_common.h> > > > > #include <b> > > #include <d> > > > > That way, the fact that "xyz.cpp" logically depends on <b> (but not <a> > or <c>) is not obscured (in other words, Mesos should continue to compile > if 'mesos_common.h' is replaced with an empty file). Does anyone know > whether the header guard in <b> _should_ make the repeated inclusion of <b> > relatively cheap? > > > > Neil >
