Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

Hi, O Lun, 04-09-2017 ás 10:13 +0200, Xabier Rodríguez Calvar escribiu: > I have another one. You can deploy IceCC distributed compilation and > CCache. This will work only for people in offices as they those are > the > ones who can share the build time. If any of you comes from

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

Hi, Gosh, I go on vacation for a week and I find this "controversial" thread about builds and another even more controversial about RefPtr :)) Generally speaking, I have the not formed impression that this will unleash hell and fire and will affect code readability. O Lun, 28-08-2017 ás

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

Hi, O Mar, 29-08-2017 ás 12:27 -0700, Sam Weinig escribiu: > This isn’t the scenario I find myself in most often.  A much more > common scenario is working on a change; touch one or two files, and > then compile and test/debug.  Rinse and repeat. When I have to change tasks, sometimes I

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> - have code completion and symbol navigation work very nicely in Xcode Unified builds will likely make this work better. Fun fact about Xcode: The way it does symbol indexing for symbols in a header is that it compiles the header with a random .cpp and indexes the result. The more stuff in

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 30, 2017, at 7:33 PM, Dean Jackson wrote: > > Dan, how did you gather the data below (i.e. how did you count the number of > files compiled)? Number of Compile* instances in the build log. ___ webkit-dev mailing list

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On 30 Aug 2017, at 8:16 am, Geoffrey Garen wrote: > > Interesting. > > The majority cases here are 7 or fewer files. I don’t see much difference > between these cases and our existing benchmark for one file, where Keith > described the build time delta as "barely

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 30, 2017, at 7:25 PM, Dean Jackson wrote: > > >> On 29 Aug 2017, at 10:37 am, Keith Miller > > wrote: >> >> Are you growing tired of long cat naps while waiting 45 minutes for WebKit >> to build? (I’ve

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On 29 Aug 2017, at 10:37 am, Keith Miller wrote: > > Are you growing tired of long cat naps while waiting 45 minutes for WebKit to > build? (I’ve definitely never done this 蘿!) Never. > Do you want WebKit to build up to 3-4x faster on a clean build?* Of course. >

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 5:54 PM, Sam Weinig wrote: > > > >> On Aug 29, 2017, at 12:46 PM, Geoffrey Garen > > wrote: >> >>> This isn’t the scenario I find myself in most often. A much more common >>> scenario is working on a

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 30, 2017, at 1:01 PM, Alexey Proskuryakov wrote: > > > >> 30 авг. 2017 г., в 11:55, Andy Estes > > написал(а): >> > In a completely other direction, what does this mean for use of Xcode? > Can we still build

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> 30 авг. 2017 г., в 11:55, Andy Estes написал(а): > In a completely other direction, what does this mean for use of Xcode? Can we still build from Xcode? Debug? >>> >>> CMake can generate Xcode files, so you can still develop and debug in Xcode. >> >> This

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 30, 2017, at 11:54 AM, Konstantin Tokarev wrote: > > > > 30.08.2017, 21:05, "Alex Christensen" : >>> On Aug 29, 2017, at 5:54 PM, Sam Weinig wrote: >>> In a completely other direction, what does this mean for use of

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

30.08.2017, 21:05, "Alex Christensen" : >>  On Aug 29, 2017, at 5:54 PM, Sam Weinig wrote: >>  In a completely other direction, what does this mean for use of Xcode? Can >> we still build from Xcode? Debug? > > CMake can generate Xcode files, so you

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 5:54 PM, Sam Weinig wrote: > In a completely other direction, what does this mean for use of Xcode? Can we > still build from Xcode? Debug? CMake can generate Xcode files, so you can still develop and debug in Xcode. > On Aug 29, 2017, at 5:37 PM, Carlos

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 30, 2017, at 2:15 AM, Konstantin Tokarev wrote: > > > > 30.08.2017, 03:01, "Michael Catanzaro" : >> On Tue, Aug 29, 2017 at 6:20 PM, Alicia Boya García >> wrote: >>> As long as the sets are generated consistently that

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

30.08.2017, 03:01, "Michael Catanzaro" : > On Tue, Aug 29, 2017 at 6:20 PM, Alicia Boya García > wrote: >>  As long as the sets are generated consistently that approach should be >>  equally fine and indeed it would reduce the burden for contributors. >>

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 6:40 PM, Sam Weinig wrote: > > > >> On Aug 29, 2017, at 6:36 PM, Maciej Stachowiak > > wrote: >> >> >> >>> On Aug 29, 2017, at 5:54 PM, Sam Weinig >> > wrote: >>>

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 6:36 PM, Maciej Stachowiak wrote: > > > >> On Aug 29, 2017, at 5:54 PM, Sam Weinig > > wrote: >> >> >> >>> On Aug 29, 2017, at 12:46 PM, Geoffrey Garen >> >

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 5:54 PM, Sam Weinig wrote: > > > >> On Aug 29, 2017, at 12:46 PM, Geoffrey Garen > > wrote: >> >>> This isn’t the scenario I find myself in most often. A much more common >>> scenario is working on a

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 12:46 PM, Geoffrey Garen wrote: > >> This isn’t the scenario I find myself in most often. A much more common >> scenario is working on a change; touch one or two files, and then compile >> and test/debug. Rinse and repeat. > > We’ve already tested

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On 29/08/17 02:37, Keith Miller wrote: > Bundling happens as part of the CMake configuration process, in the > CMake build. In the Xcode build it’s more complicated since we cannot > dynamically choose the files we are going to compile. The onlysolution > I know of is to pre-list a bunch of files

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On Tue, Aug 29, 2017 at 3:22 PM, Olmstead, Don wrote: Sure but in a corporate environment it can be a really big win. All of our developers have their machines hooked up to the distributed build system and we also have servers hanging out that can be used in the build.

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 4:20 PM, Alicia Boya García wrote: > > On 08/30/2017 12:00 AM, Keith Miller wrote: > >> The automated system should handle your concern about bundling similar .cpp >> files together. The proposed system bundles by directory and also sorts the >> files

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On 08/30/2017 12:00 AM, Keith Miller wrote: > The automated system should handle your concern about bundling similar .cpp > files together. The proposed system bundles by directory and also sorts the > files in those directories when dividing up the files. Is there any reason to > think that

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 3:16 PM, Geoffrey Garen wrote: > > Interesting. > > The majority cases here are 7 or fewer files. I don’t see much difference > between these cases and our existing benchmark for one file, where Keith > described the build time delta as "barely

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

Interesting. The majority cases here are 7 or fewer files. I don’t see much difference between these cases and our existing benchmark for one file, where Keith described the build time delta as "barely noticeable". For the minority cases that are 23 - 75 files, these challenge Keith’s

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

As far as the static renaming problem, Darin convinced me in the other thread that we probably don’t need such a heavy handed approach. At least in JSC, there are a number of variables that have generic names that should be put into a namespace but the majority probably don’t need to be

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

29.08.2017, 14:44, "Alicia Boya García" : > On 08/29/2017 06:20 AM, Daniel Bates wrote: > >>  Do we know what is the cause(es) for the slow clean builds? I am assuming >> that much of the speed up from the "unified source" comes from clang being >> able to use an in-memory

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 1:39 PM, Geoffrey Garen wrote: > >> I see. The right question to ask would have been how much change occurs in >> their working copy between consecutive incremental builds. > > If you want to help make our benchmark righter, please do share any data

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

29.08.2017, 23:54, "Alicia Boya García" : > I'm all for stable bundle .cpp files. Automatic generation can be nice > for the first time... and once in while we can try to modify the bundles > to try to make the build even faster, but really most of the time bundle > contents

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

I'm all for stable bundle .cpp files. Automatic generation can be nice for the first time... and once in while we can try to modify the bundles to try to make the build even faster, but really most of the time bundle contents should not change unless new files are added and everybody should get

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> I see. The right question to ask would have been how much change occurs in > their working copy between consecutive incremental builds. If you want to help make our benchmark righter, please do share any data you have about the average content of an incremental build that is distinct from a

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 1:22 PM, Olmstead, Don wrote: > > Sure but in a corporate environment it can be a really big win. All of our > developers have their machines hooked up to the distributed build system and > we also have servers hanging out that can be used in the

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

Sure but in a corporate environment it can be a really big win. All of our developers have their machines hooked up to the distributed build system and we also have servers hanging out that can be used in the build. This sort of thing doesn't require anyone to change how they write their code

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

29.08.2017, 22:48, "Geoffrey Garen" : >>> We have some preliminary data that says incremental builds will be OK, but >>> not a full benchmark. >>> >>> Here’s a full benchmark I propose to test incremental builds: >>> >>> Start 7 days ago in SVN history. Do a clean build. >>>

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 12:48 PM, Geoffrey Garen wrote: > >>> We have some preliminary data that says incremental builds will be OK, but >>> not a full benchmark. >>> >>> Here’s a full benchmark I propose to test incremental builds: >>> >>> Start 7 days ago in SVN

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

>> We have some preliminary data that says incremental builds will be OK, but >> not a full benchmark. >> >> Here’s a full benchmark I propose to test incremental builds: >> >> Start 7 days ago in SVN history. Do a clean build. >> >> SVN update forward by 24 hours. Do an incremental

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> This isn’t the scenario I find myself in most often. A much more common > scenario is working on a change; touch one or two files, and then compile and > test/debug. Rinse and repeat. We’ve already tested this case. The worst case slowdown, if you touch a small file that's in the same

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 11:03 AM, Geoffrey Garen wrote: > >> I worry about adopting unity build because while it makes clean builds >> faster, it also slows down incremental builds. As a developer, I rarely do >> clean builds, I mostly do incremental builds so this would

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

I'd like to propose a variation on Keith's proposal. I think we should manually determine which files should be built together, rather than writing CMake code to do that automatically. Let's consider an example of what use of the FILENAME namespace, as Keith proposes, might look like. I

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

  29.08.2017, 21:32, "Maciej Stachowiak" :  On Aug 29, 2017, at 9:10 AM, Chris Dumez wrote: I worry about adopting unity build because while it makes clean builds faster, it also slows down incremental builds. As a developer, I rarely do clean builds, I mostly do

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 9:10 AM, Chris Dumez wrote: > > I worry about adopting unity build because while it makes clean builds > faster, it also slows down incremental builds. As a developer, I rarely do > clean builds, I mostly do incremental builds so this would likely make

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 11:03 AM, Geoffrey Garen wrote: > >> I worry about adopting unity build because while it makes clean builds >> faster, it also slows down incremental builds. As a developer, I rarely do >> clean builds, I mostly do incremental builds so this would

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

29.08.2017, 21:19, "Olmstead, Don" : > Did you happen to look at any sort of distributed build system? We use > distributed builds here for WebKit and a full rebuild is pretty reasonable. > That would negate the need to change how everything is done to get a unity >

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 11:03 AM, Geoffrey Garen wrote: > >> I worry about adopting unity build because while it makes clean builds >> faster, it also slows down incremental builds. As a developer, I rarely do >> clean builds, I mostly do incremental builds so this would

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

Did you happen to look at any sort of distributed build system? We use distributed builds here for WebKit and a full rebuild is pretty reasonable. That would negate the need to change how everything is done to get a unity build going. From: webkit-dev

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> I worry about adopting unity build because while it makes clean builds > faster, it also slows down incremental builds. As a developer, I rarely do > clean builds, I mostly do incremental builds so this would likely make my > experience worse? We have some preliminary data that says

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 9:10 AM, Chris Dumez wrote: > > I worry about adopting unity build because while it makes clean builds > faster, it also slows down incremental builds. As a developer, I rarely do > clean builds, I mostly do incremental builds so this would likely make

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On Tue, Aug 29, 2017 at 12:10 PM, Chris Dumez wrote: > I worry about adopting unity build because while it makes clean builds > faster, it also slows down incremental builds. As a developer, I rarely do > clean builds, I mostly do incremental builds so this would likely make my

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 29, 2017, at 9:24 AM, Keith Miller wrote: > >> * We could not use any namespaces and just fix name clashes as they arise, >> but only if we have some more predictable way to determine which files will >> be built together regardless of port or build

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 28, 2017, at 8:10 PM, Michael Catanzaro wrote: > > On Mon, Aug 28, 2017 at 7:37 PM, Keith Miller wrote: >> Given that most of the build time in incremental builds is scanning >> dependencies, this change is probably only barely

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

I worry about adopting unity build because while it makes clean builds faster, it also slows down incremental builds. As a developer, I rarely do clean builds, I mostly do incremental builds so this would likely make my experience worse? Unity builds also require a lot of code churn to resolve

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> I wonder whether the unified sources builds would still allow us to > keep using compiler caches like ccache. Our plan is to use ccache. > Second question/caveat is, wouldn't the bundle build penalize too much > incremental builds? If your data is correct we would get a <20% build > time

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

I often use the “Show Preprocessed Source” option in Xcode. Will I be able to continue using this? > On Aug 29, 2017, at 8:10 AM, Sergio Villar Senin wrote: > > O Lun, 28-08-2017 ás 17:37 -0700, Keith Miller escribiu: >> Hello fellow Webkittens, >> >> Are you growing tired

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

O Lun, 28-08-2017 ás 17:37 -0700, Keith Miller escribiu: > Hello fellow Webkittens, > > Are you growing tired of long cat naps while waiting 45 minutes for > WebKit to build? (I’ve definitely never done this 蘿!) > Do you want WebKit to build up to 3-4x faster on a clean build?* > Does seeing your

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On Mon, Aug 28, 2017 at 11:46 PM, Geoffrey Garen wrote: Our plan is to use #include. That has worked in the past. Geoff Um yes, that would be better. :) ___ webkit-dev mailing list webkit-dev@lists.webkit.org

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On 08/29/2017 06:20 AM, Daniel Bates wrote: > Do we know what is the cause(es) for the slow clean builds? I am assuming > that much of the speed up from the "unified source" comes from clang being > able to use an in-memory #include cache to avoid disk I/O and re-parsing of a > seen header.

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> The line numbers and filenames will be total nonsense if we just concatenate > multiple source files together. But that's very easy to fix if the script > that concatenates the sources also adds a #line statement between "files" to > change the filename and reset the line number to 1. See >

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On Mon, Aug 28, 2017 at 11:20 PM, Daniel Bates wrote: How does the speed gain with your proposed "unified source" approach compare to using CMake + Ninja to build currently (I think builder Apple El Capitan CMake Debug does this)? I don't have any hard numbers, but

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

> On Aug 28, 2017, at 9:20 PM, Daniel Bates wrote: > > Thank you for looking into speeding up the build. > > How does the speed gain with your proposed "unified source" approach compare > to using CMake + Ninja to build currently (I think builder Apple El Capitan > CMake

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

Thank you for looking into speeding up the build. How does the speed gain with your proposed "unified source" approach compare to using CMake + Ninja to build currently (I think builder Apple El Capitan CMake Debug does this)? Do we know what is the cause(es) for the slow clean builds? I am

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On Mon, Aug 28, 2017 at 10:34 PM, Ryosuke Niwa wrote: I don't think we should make this change. It would mean that whenever the function signature changes, we'd have to modify the header file, which in turn triggers rebuild of every cpp file which includes that header file.

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On Mon, Aug 28, 2017 at 8:10 PM, Michael Catanzaro wrote: > On Mon, Aug 28, 2017 at 7:37 PM, Keith Miller > wrote: >> >> Given that most of the build time in incremental builds is scanning >> dependencies, this change is probably only barely

Re: [webkit-dev] Growing tired of long build times? Check out this awesome new way to speed up your build... soon (HINT: It's not buying a new computer)

On Mon, Aug 28, 2017 at 7:37 PM, Keith Miller wrote: Given that most of the build time in incremental builds is scanning dependencies, this change is probably only barely noticeable. What, with CMake? That's true of the make backend, but definitely not the ninja