> On Aug 28, 2017, at 9:20 PM, Daniel Bates <dba...@webkit.org> wrote:
>
> Thank you for looking into speeding up the build.
>
> How does the speed gain with your proposed "unified source" approach compare
> to using CMake + Ninja to build currently (I think builder Apple El Capitan
> CMake Debug does this)?
I think unified sources help in a way that's orthogonal to what CMake+Ninja do,
so we'd ideally want to apply both to a build of the whole WebKit stack.
Ideally this combination would become the default for engineering builds of the
macOS and iOS ports.
> Do we know what is the cause(es) for the slow clean builds? I am assuming
> that much of the speed up from the "unified source" comes from clang being
> able to use an in-memory #include cache to avoid disk I/O and re-parsing of a
> seen header. Have we exhausted all efforts (or have we given up?) removing
> extraneous #includes? Do we think this pursuing this effort would be more
> time consuming or have results that pale in comparison to the "unified
> source" approach?
My 2c:
Probably the better long-term solution would be C++ modules once they are ready
for prime time. In the short term, unified sources seem like a much faster way
to get a big win than include cleanup. There's also some reason to believe
unified sources can result in a higher potential win than include pruning,
since we in fact have many headers that legitimately need to be included in
lots of files.
- Maciej
>
>
>> On Aug 28, 2017, at 5:37 PM, Keith Miller <keith_mil...@apple.com> wrote:
>>
>> Hello fellow Webkittens,
>>
>> Are you growing tired of long cat naps while waiting 45 minutes for WebKit
>> to build? (I’ve definitely never done this 🤐!)
>> Do you want WebKit to build up to 3-4x faster on a clean build?*
>> Does seeing your incremental build… stay the same sound good?
>>
>> You’ll be purring with excitement after you switch to unified source
>> builds!**
>>
>> * Building JSC with CMake, including CMake configuration time, went from
>> ~8m15s to ~2m30s on my MBP using unified sources only on cpp files in
>> JavaScriptCore. Final results on WebKit may vary. Using unified sources may
>> cause sudden feelings of euphoria and productivity (or grumpiness from lack
>> of naps…). Not responsible for any bugs in extra patches produced by using
>> unified source builds (excluding build system related bugs) except where
>> required by law (pi...@apple.com).
>>
>> ** Soon. Unified source builds haven’t been landed yet.
>>
>> How are things going to change? Great question! Let’s find out!
>>
>> When you fire off the build script it will automagically paw your source
>> files into bundles.
>
> How does this work if you build from Xcode? I am assuming these bundles won't
> interfere with debugging in Xcode. When debugging in Xcode/lldb/gdb will
> Xcode/lldb/gdb resolve line information with respect to individual files or
> the bundles?
>
>> The size of each bundle is speculatively going to be 8 .cpp files but that’s
>> up for experimentation. From my testing so far, the compile time for a
>> bundle is at most 20% (~6s vs ~7s for the Air directory) longer than the
>> longest file it includes. Given that most of the build time in incremental
>> builds is scanning dependencies, this change is probably only barely
>> noticeable.
>>
>> Bundling happens as part of the CMake configuration process, in the CMake
>> build. In the Xcode build it’s more complicated since we cannot dynamically
>> choose the files we are going to compile. The only solution I know of is to
>> pre-list a bunch of files for the build to dump files into. Occasionally,
>> we’ll need to add more files to the build list.
>>
>> When you add or remove a file in the project, the cost is higher. The
>> current plan to bundle source files is by directory. This means that when a
>> .cpp file is added or removed you will rebalance the bundles in its
>> directory, and you will have to rebuild up to the full directory that file
>> is in.
>>
>> My goal is to make all of this invisible to developers as much as possible.
>> I believe, for the most part, things should operate mostly as they have
>> before. As the details on unified source builds get finalized, I’m sending
>> out a separate email documenting any workflow changes. I’ll also do my best
>> to answer any questions or concerns.
>>
>> “But Keith, am I going to have to make major changes to the way I develop
>> WebKit?” Yes and, hopefully, no. There is one major common workflow change
>> that comes with unified sources, the naming of static variables and
>> functions.
>
> I take it you mean any file-level function or definition with internal
> linkage. That is, this would also affect file-level constants.
>
>> Since C++ does not have a way to only declare a static function or variable
>> in some restricted top level scope we are going to need to create some work
>> around. The choice I’m thinking we’ll go with is to have an auto-generated
>> macro value FILENAME, which is similar to __FILE__ except that it doesn’t
>> have file extension (We can’t use FILE since that’s actually a type in C 🙁).
>> This macro would be redefined in the unified source directly above the file
>> that is about to be included. Then in each cpp file that has static
>> functions/variables developers would have:
>>
>> namespace FILENAME {
>> static int myVariable { 0 };
>> static int myFunction() { return myVariable; }
>> }
>>
>> The advantage of using a macro over textually writing the filename is that
>> it makes code more portable between files. Also, once people get used to the
>> concept of FILENAME in the code base it will, hopefully become obvious what
>> it means. The main downside to this is using FILENAME in a header would give
>> you whatever cpp file first included you (if only we got the compiler magic
>> of __FILE__...). We would probably need to prohibit using FILENAME in .h
>> files. Unfortunately, you cannot "using FILENAME;” because it will export
>> all the values in each subsequent namespace. e.g.
>>
>> namespace JSC {
>> namespace Foo {
>> static int myVar { 0 };
>> }
>> using Foo;
>> }
>>
>> namespace JSC {
>> namespace Bar {
>> static int myVar { 0 };
>> }
>> using Bar;
>> }
>>
>> does not compile.
>>
>> Does anyone have other name recommendations for the macro name other than
>> FILENAME? I went with file name since it describes what the substituted
>> value will be the best, which is helpful for debugging. The other options I
>> considered were FILE_LOCAL and PER_FILE.
>>
>
> I prefer FILE_LOCAL if I have to swallow this pill.
>
>> Are there other issues that people think might occur with unified sources?
>
> Bad news bears for files with unnamed namespaces. We will need to fix these
> up. There are likely few uses of them in the code base.
>
> Dan
> _______________________________________________
> webkit-dev mailing list
> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
> https://lists.webkit.org/mailman/listinfo/webkit-dev
> <https://lists.webkit.org/mailman/listinfo/webkit-dev>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
