29.08.2017, 21:19, "Olmstead, Don" <don.olmst...@sony.com>: > Did you happen to look at any sort of distributed build system? We use > distributed builds here for WebKit and a full rebuild is pretty reasonable. > That would negate the need to change how everything is done to get a unity > build going.
This isn't helpful for contributors who have only one machine at their disposal > > From: webkit-dev > [mailto:webkit-dev-bounces+don.olmstead=am.sony....@lists.webkit.org] On > Behalf Of Keith Miller > Sent: Monday, August 28, 2017 5:37 PM > To: WebKit Development Mailing List <webkit-dev@lists.webkit.org> > Subject: [webkit-dev] Growing tired of long build times? Check out this > awesome new way to speed up your build... soon (HINT: It's not buying a new > computer) > > Hello fellow Webkittens, > > Are you growing tired of long cat naps while waiting 45 minutes for WebKit to > build? (I’ve definitely never done this 🤐!) > > Do you want WebKit to build up to 3-4x faster on a clean build?* > > Does seeing your incremental build… stay the same sound good? > > You’ll be purring with excitement after you switch to unified source builds!** > > * Building JSC with CMake, including CMake configuration time, went from > ~8m15s to ~2m30s on my MBP using unified sources only on cpp files in > JavaScriptCore. Final results on WebKit may vary. Using unified sources may > cause sudden feelings of euphoria and productivity (or grumpiness from lack > of naps…). Not responsible for any bugs in extra patches produced by using > unified source builds (excluding build system related bugs) except where > required by law (pi...@apple.com). > > ** Soon. Unified source builds haven’t been landed yet. > > How are things going to change? Great question! Let’s find out! > > When you fire off the build script it will automagically paw your source > files into bundles. The size of each bundle is speculatively going to be 8 > .cpp files but that’s up for experimentation. From my testing so far, the > compile time for a bundle is at most 20% (~6s vs ~7s for the Air directory) > longer than the longest file it includes. Given that most of the build time > in incremental builds is scanning dependencies, this change is probably only > barely noticeable. > > Bundling happens as part of the CMake configuration process, in the CMake > build. In the Xcode build it’s more complicated since we cannot dynamically > choose the files we are going to compile. The only solution I know of is to > pre-list a bunch of files for the build to dump files into. Occasionally, > we’ll need to add more files to the build list. > > When you add or remove a file in the project, the cost is higher. The current > plan to bundle source files is by directory. This means that when a .cpp file > is added or removed you will rebalance the bundles in its directory, and you > will have to rebuild up to the full directory that file is in. > > My goal is to make all of this invisible to developers as much as possible. I > believe, for the most part, things should operate mostly as they have before. > As the details on unified source builds get finalized, I’m sending out a > separate email documenting any workflow changes. I’ll also do my best to > answer any questions or concerns. > > “But Keith, am I going to have to make major changes to the way I develop > WebKit?” Yes and, hopefully, no. There is one major common workflow change > that comes with unified sources, the naming of static variables and > functions. Since C++ does not have a way to only declare a static function or > variable in some restricted top level scope we are going to need to create > some work around. The choice I’m thinking we’ll go with is to have an > auto-generated macro value FILENAME, which is similar to __FILE__ except that > it doesn’t have file extension (We can’t use FILE since that’s actually a > type in C 🙁). This macro would be redefined in the unified source directly > above the file that is about to be included. Then in each cpp file that has > static functions/variables developers would have: > > namespace FILENAME { > > static int myVariable { 0 }; > > static int myFunction() { return myVariable; } > > } > > The advantage of using a macro over textually writing the filename is that it > makes code more portable between files. Also, once people get used to the > concept of FILENAME in the code base it will, hopefully become obvious what > it means. The main downside to this is using FILENAME in a header would give > you whatever cpp file first included you (if only we got the compiler magic > of __FILE__...). We would probably need to prohibit using FILENAME in .h > files. Unfortunately, you cannot "using FILENAME;” because it will export all > the values in each subsequent namespace. e.g. > > namespace JSC { > > namespace Foo { > > static int myVar { 0 }; > > } > > using Foo; > > } > > namespace JSC { > > namespace Bar { > > static int myVar { 0 }; > > } > > using Bar; > > } > > does not compile. > > Does anyone have other name recommendations for the macro name other than > FILENAME? I went with file name since it describes what the substituted value > will be the best, which is helpful for debugging. The other options I > considered were FILE_LOCAL and PER_FILE. > > Are there other issues that people think might occur with unified sources? > Note, It’s also likely that there are some source files that need to be > excluded from the unified source builds for various reasons. > > “Is there anything else I need to do with unified sources?”, Yes! Enjoy the > faster builds! > > To help visualize how much of a difference unified sources can make for you, > I’ve illustrated this helpful graph on the WebKit developer experience: > > * Due to size constraints this graph is not to scale. > > Cheers, > > Keith > > P.S. > > For those interested there was a long process in deciding how to improve > builds: > > The first thing I looked at was using C++ modules. > > There were a couple of problems with C++ modules. First, different compliers > support modules in different ways. This was not a major concern since we > probably would have gone with Clang’s version of modules. As far as I know, > almost all developers either use clang or could likely switch to clang for > development. The bigger problem with Clang modules is that it seemed like we > would have a rough early adopter problem. Additionally, since a lot of > Apple’s internal SDK’s don’t seem to support modules it was likely to be a > major slog to get everything working correctly. In a few years I think it > will probably be worth looking at C++ modules again as, in theory, C++ > modules should be better than unified sources. > > Since we decided against C++ modules that really only leaves us with some > form of unified sources. Obviously, doing all in one builds would be out of > the question. While all in ones would have been great for clean builds, it > also means incremental builds are essentially the same as clean builds. So, I > concluded that we should go with some kind of automated unified build file > solution. An automated unified build file solution means that most of the > time the build system is opaque to developers (well as much as is possible > with a unified source build). > > Since the static variable naming problem was likely to be the biggest issue > with unified sources, it was important to create a system that made this a > pain free as possible. There were other options we could go with. We could > have a script that parses .cpp files and looks for static variables with the > same name and yells at you. This is pretty annoying and wouldn’t help actual > development when people hit an error. For example, I know I never run the > check-webkit-style script until I’m putting a patch up for review. > , > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev -- Regards, Konstantin _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
