Sure but in a corporate environment it can be a really big win. All of our 
developers have their machines hooked up to the distributed build system and we 
also have servers hanging out that can be used in the build. This sort of thing 
doesn't require anyone to change how they write their code which is something a 
unity build ends up requiring.

In terms of WebKit development the only places it falls apart are in the source 
code generation and the link.

Really I was just curious if anything like that was discussed because all of 
this requires pretty big changes to source code. Additionally there's other fun 
things that can happen around includes when doing a unity build where a file 
will stop being able to be built in isolation. The same thing has happened with 
WebCorePrefix where a number of files can't be built without it.

-----Original Message-----
From: Konstantin Tokarev [mailto:annu...@yandex.ru] 
Sent: Tuesday, August 29, 2017 11:22 AM
To: Olmstead, Don <don.olmst...@sony.com>; Keith Miller 
<keith_mil...@apple.com>; WebKit Development Mailing List 
<webkit-dev@lists.webkit.org>
Subject: Re: [webkit-dev] Growing tired of long build times? Check out this 
awesome new way to speed up your build... soon (HINT: It's not buying a new 
computer)



29.08.2017, 21:19, "Olmstead, Don" <don.olmst...@sony.com>:
> Did you happen to look at any sort of distributed build system? We use 
> distributed builds here for WebKit and a full rebuild is pretty reasonable. 
> That would negate the need to change how everything is done to get a unity 
> build going.

This isn't helpful for contributors who have only one machine at their disposal

>
> From: webkit-dev 
> [mailto:webkit-dev-bounces+don.olmstead=am.sony....@lists.webkit.org] On 
> Behalf Of Keith Miller
> Sent: Monday, August 28, 2017 5:37 PM
> To: WebKit Development Mailing List <webkit-dev@lists.webkit.org>
> Subject: [webkit-dev] Growing tired of long build times? Check out this 
> awesome new way to speed up your build... soon (HINT: It's not buying a new 
> computer)
>
> Hello fellow Webkittens,
>
> Are you growing tired of long cat naps while waiting 45 minutes for WebKit to 
> build? (I’ve definitely never done this 🤐!)
>
> Do you want WebKit to build up to 3-4x faster on a clean build?*
>
> Does seeing your incremental build… stay the same sound good?
>
> You’ll be purring with excitement after you switch to unified source builds!**
>
> * Building JSC with CMake, including CMake configuration time, went from 
> ~8m15s to ~2m30s on my MBP using unified sources only on cpp files in 
> JavaScriptCore. Final results on WebKit may vary. Using unified sources may 
> cause sudden feelings of euphoria and productivity (or grumpiness from lack 
> of naps…). Not responsible for any bugs in extra patches produced by using 
> unified source builds (excluding build system related bugs) except where 
> required by law (pi...@apple.com).
>
> ** Soon. Unified source builds haven’t been landed yet.
>
> How are things going to change? Great question! Let’s find out!
>
> When you fire off the build script it will automagically paw your source 
> files into bundles. The size of each bundle is speculatively going to be 8 
> .cpp files but that’s up for experimentation. From my testing so far, the 
> compile time for a bundle is at most 20% (~6s vs ~7s for the Air directory) 
> longer than the longest file it includes. Given that most of the build time 
> in incremental builds is scanning dependencies, this change is probably only 
> barely noticeable.
>
> Bundling happens as part of the CMake configuration process, in the CMake 
> build. In the Xcode build it’s more complicated since we cannot dynamically 
> choose the files we are going to compile. The only solution I know of is to 
> pre-list a bunch of files for the build to dump files into. Occasionally, 
> we’ll need to add more files to the build list.
>
> When you add or remove a file in the project, the cost is higher. The current 
> plan to bundle source files is by directory. This means that when a .cpp file 
> is added or removed you will rebalance the bundles in its directory, and you 
> will have to rebuild up to the full directory that file is in.
>
> My goal is to make all of this invisible to developers as much as possible. I 
> believe, for the most part, things should operate mostly as they have before. 
> As the details on unified source builds get finalized, I’m sending out a 
> separate email documenting any workflow changes. I’ll also do my best to 
> answer any questions or concerns.
>
> “But Keith, am I going to have to make major changes to the way I develop 
> WebKit?” Yes and, hopefully, no. There is one major common workflow change 
> that comes with unified sources, the naming of static variables and 
> functions. Since C++ does not have a way to only declare a static function or 
> variable in some restricted top level scope we are going to need to create 
> some work around. The choice I’m thinking we’ll go with is to have an 
> auto-generated macro value FILENAME, which is similar to __FILE__ except that 
> it doesn’t have file extension (We can’t use FILE since that’s actually a 
> type in C 🙁). This macro would be redefined in the unified source directly 
> above the file that is about to be included. Then in each cpp file that has 
> static functions/variables developers would have:
>
> namespace FILENAME {
>
>             static int myVariable { 0 };
>
>             static int myFunction() { return myVariable; }
>
> }
>
> The advantage of using a macro over textually writing the filename is that it 
> makes code more portable between files. Also, once people get used to the 
> concept of FILENAME in the code base it will, hopefully become obvious what 
> it means. The main downside to this is using FILENAME in a header would give 
> you whatever cpp file first included you (if only we got the compiler magic 
> of __FILE__...). We would probably need to prohibit using FILENAME in .h 
> files. Unfortunately, you cannot "using FILENAME;” because it will export all 
> the values in each subsequent namespace. e.g.
>
> namespace JSC {
>
>     namespace Foo {
>
>             static int myVar { 0 };
>
>     }
>
>     using Foo;
>
> }
>
> namespace JSC {
>
>     namespace Bar {
>
>             static int myVar { 0 };
>
>     }
>
>     using Bar;
>
> }
>
> does not compile.
>
> Does anyone have other name recommendations for the macro name other than 
> FILENAME? I went with file name since it describes what the substituted value 
> will be the best, which is helpful for debugging. The other options I 
> considered were FILE_LOCAL and PER_FILE.
>
> Are there other issues that people think might occur with unified sources? 
> Note, It’s also likely that there are some source files that need to be 
> excluded from the unified source builds for various reasons.
>
> “Is there anything else I need to do with unified sources?”, Yes! Enjoy the 
> faster builds!
>
> To help visualize how much of a difference unified sources can make for you, 
> I’ve illustrated this helpful graph on the WebKit developer experience:
>
> * Due to size constraints this graph is not to scale.
>
> Cheers,
>
> Keith
>
> P.S.
>
> For those interested there was a long process in deciding how to improve 
> builds:
>
> The first thing I looked at was using C++ modules.
>
> There were a couple of problems with C++ modules. First, different compliers 
> support modules in different ways. This was not a major concern since we 
> probably would have gone with Clang’s version of modules. As far as I know, 
> almost all developers either use clang or could likely switch to clang for 
> development. The bigger problem with Clang modules is that it seemed like we 
> would have a rough early adopter problem. Additionally, since a lot of 
> Apple’s internal SDK’s don’t seem to support modules it was likely to be a 
> major slog to get everything working correctly. In a few years I think it 
> will probably be worth looking at C++ modules again as, in theory, C++ 
> modules should be better than unified sources.
>
> Since we decided against C++ modules that really only leaves us with some 
> form of unified sources. Obviously, doing all in one builds would be out of 
> the question. While all in ones would have been great for clean builds, it 
> also means incremental builds are essentially the same as clean builds. So, I 
> concluded that we should go with some kind of automated unified build file 
> solution. An automated unified build file solution means that most of the 
> time the build system is opaque to developers (well as much as is possible 
> with a unified source build).
>
> Since the static variable naming problem was likely to be the biggest issue 
> with unified sources, it was important to create a system that made this a 
> pain free as possible. There were other options we could go with. We could 
> have a script that parses .cpp files and looks for static variables with the 
> same name and yells at you. This is pretty annoying and wouldn’t help actual 
> development when people hit an error. For example, I know I never run the 
> check-webkit-style script until I’m putting a patch up for review.
> ,
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev


-- 
Regards,
Konstantin
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Reply via email to