Re: Is there a way to improve partial compilation times?

zbranie...@mozilla.com writes: > * I still have only 8GB of ram which is probably the ultimate > limiting factor You are right here. RAM is required not only for link time, but also when compiling several large unified files at a time (though perhaps this is not so significant with only 4

Re: Is there a way to improve partial compilation times?

Reporting first results. We got an icecream setup in SF office and I was able to plug myself into it and got a icecc+ccache+gcc combo with a fresh debug build in <30 min. On top of that, I had low load on my machine, which is nice as in the meantime I was able to work on other things. Now,

Re: Is there a way to improve partial compilation times?

On Tue, Mar 7, 2017 at 5:40 PM, Randell Jesup wrote: > >On 07/03/17 20:29, zbranie...@mozilla.com wrote: > > > >> I was just wondering if really two days of patches landing in Gecko > should result > >> in what seems like basically full rebuild. > >> > >> A clean build

Re: Is there a way to improve partial compilation times?

I'm pretty sure that by the time we're reaching that number of cores we'll be blocked on preprocessing, as preprocessing occurs on the local machine (where the header files are), and on network I/O. I imagine that peak efficiency is well below 2k machines. In addition, there's some unavoidable

Re: Is there a way to improve partial compilation times?

On Thu, Mar 09, 2017 at 07:45:13AM -0500, Ted Mielczarek wrote: > On Wed, Mar 8, 2017, at 05:43 PM, Ehsan Akhgari wrote: > > On 2017-03-08 11:31 AM, Simon Sapin wrote: > > > On 08/03/17 15:24, Ehsan Akhgari wrote: > > >> What we did in the Toronto office was walked to people who ran Linux on > >

Re: Is there a way to improve partial compilation times?

On Wed, Mar 8, 2017, at 05:43 PM, Ehsan Akhgari wrote: > On 2017-03-08 11:31 AM, Simon Sapin wrote: > > On 08/03/17 15:24, Ehsan Akhgari wrote: > >> What we did in the Toronto office was walked to people who ran Linux on > >> their desktop machines and installed the icecream server on their > >>

Re: Is there a way to improve partial compilation times?

On Thu, Mar 9, 2017, at 07:42 AM, Wei-Cheng Pan wrote: > We are using icecream in the Taipei office too, and it is a big enhance. > Sadly when we tried to use it on Mac OS, we always got wrong stack > information. > I've read the article on MDN, seems it's related to a compiler flag >

Re: Is there a way to improve partial compilation times?

On 08/03/2017 10:24 PM, Ehsan Akhgari wrote: > On 2017-03-08 6:40 AM, James Graham wrote: >> On 08/03/17 11:11, Frederik Braun wrote: >>> On 08.03.2017 01:17, Ralph Giles wrote: I second Jeff's point about building with icecream[1]. If you work in an office with a build farm, or near a

Re: Is there a way to improve partial compilation times?

On 03/08/2017 06:21 AM, Ehsan Akhgari wrote: On 2017-03-07 2:49 PM, Eric Rahm wrote: I often wonder if unified builds are making things slower for folks who use ccache (I assume one file changing would mean a rebuild for the entire unified chunk), I'm not sure if there's a solution to that but

Re: Is there a way to improve partial compilation times?

On Wednesday, March 8, 2017 at 8:57:57 AM UTC-8, James Graham wrote: > On 08/03/17 14:21, Ehsan Akhgari wrote: > At risk of stating the obvious, if you aren't touching C++ code (or > maybe jsm?), and aren't using any funky compile options, you should be > using an artifact build for best

Re: Is there a way to improve partial compilation times?

On 08/03/17 14:21, Ehsan Akhgari wrote: On 2017-03-07 2:49 PM, Eric Rahm wrote: I often wonder if unified builds are making things slower for folks who use ccache (I assume one file changing would mean a rebuild for the entire unified chunk), I'm not sure if there's a solution to that but it

Re: Is there a way to improve partial compilation times?

On 08/03/17 15:24, Ehsan Akhgari wrote: What we did in the Toronto office was walked to people who ran Linux on their desktop machines and installed the icecream server on their computer. I suggest you do the same in London. There is no need to wait for dedicated build machines. ;-) We’ve

Re: Is there a way to improve partial compilation times?

Gotcha. Problem for the Berlin office: There are only 3 people who have a desktop and run linux. Two of them are part of our "cluster" :) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform

Re: Is there a way to improve partial compilation times?

On 2017-03-08 6:40 AM, James Graham wrote: > On 08/03/17 11:11, Frederik Braun wrote: >> On 08.03.2017 01:17, Ralph Giles wrote: >>> I second Jeff's point about building with icecream[1]. If you work in >>> an office with a build farm, or near a fast desktop machine you can >>> pass jobs to, this

Re: Is there a way to improve partial compilation times?

On 2017-03-07 2:49 PM, Eric Rahm wrote: > I often wonder if unified builds are making things slower for folks who use > ccache (I assume one file changing would mean a rebuild for the entire > unified chunk), I'm not sure if there's a solution to that but it would be > interesting to see if

Re: Is there a way to improve partial compilation times?

On 08/03/17 11:11, Frederik Braun wrote: On 08.03.2017 01:17, Ralph Giles wrote: I second Jeff's point about building with icecream[1]. If you work in an office with a build farm, or near a fast desktop machine you can pass jobs to, this makes laptop builds much more tolerable. What do you

Re: Is there a way to improve partial compilation times?

On 08/03/2017 01:11, Mike Hommey wrote: > You probably want a desktop machine, not a new laptop. I second that, modern laptops are usually thermally limited. I actually drilled holes in the back of my Thinkpad to improve airflow (and it did improve build times). My main box is a not-so-recent

Re: Is there a way to improve partial compilation times?

On 08.03.2017 01:17, Ralph Giles wrote: > I second Jeff's point about building with icecream[1]. If you work in > an office with a build farm, or near a fast desktop machine you can > pass jobs to, this makes laptop builds much more tolerable. > What do you mean by build farm? Do some offices

Re: Is there a way to improve partial compilation times?

On 3/7/17 4:25 PM, Chris Peterson wrote: Can you just nice mach? I seem to recall trying that and it not helping enough (on MacOS) with the default "use -j8 on a 4-core machine" behavior. YMMV based on OS, ratio of RAM to cores, and whatnot. -Boris

Re: Is there a way to improve partial compilation times?

I second Jeff's point about building with icecream[1]. If you work in an office with a build farm, or near a fast desktop machine you can pass jobs to, this makes laptop builds much more tolerable. Despite the warnings on the mdn page, I do this over the wan as well. It's a lot slower than when

Re: Is there a way to improve partial compilation times?

On Tue, Mar 07, 2017 at 03:50:56PM -0800, zbranie...@mozilla.com wrote: > On Tuesday, March 7, 2017 at 3:24:33 PM UTC-8, Mike Hommey wrote: > > On what OS? I have a XPS 12 from 2013 and a XPS 13 9360, and both do > > clobber builds in 40 minutes (which is the sad surprise that laptop CPUs > >

Re: Is there a way to improve partial compilation times?

On Tuesday, March 7, 2017 at 3:24:33 PM UTC-8, Mike Hommey wrote: > On what OS? I have a XPS 12 from 2013 and a XPS 13 9360, and both do > clobber builds in 40 minutes (which is the sad surprise that laptop CPUs > performance have not improved in 3 years), on Linux. 70 minutes is way > too much.

Re: Is there a way to improve partial compilation times?

70 minutes is about what a clobber build takes on my Surface Book. And yes I agree, it is way too much! On Tue, Mar 7, 2017 at 3:24 PM, Mike Hommey wrote: > On Tue, Mar 07, 2017 at 11:29:00AM -0800, zbranie...@mozilla.com wrote: > > So, > > > > I'm on Dell XPS 13 (9350), and

Re: Is there a way to improve partial compilation times?

>On 07/03/17 20:29, zbranie...@mozilla.com wrote: > >> I was just wondering if really two days of patches landing in Gecko should >> result >> in what seems like basically full rebuild. >> >> A clean build takes 65-70, a rebuild after two days of patches takes >> 50-60min. > >That seems pretty

Re: Is there a way to improve partial compilation times?

On 3/7/2017 11:19 AM, Steve Fink wrote: I have at times spun off builds into their own cgroup. It seems to isolate the load pretty well, when I want to bother with remembering how to set it up again. Perhaps it'd be a good thing for mach to do automatically. Then again, if dropping the -j count

Re: Is there a way to improve partial compilation times?

On Tue, Mar 7, 2017 at 2:29 PM, wrote: > So, > > I'm on Dell XPS 13 (9350), and I don't think that toying with MOZ_MAKE_FLAGS > will help me here. "-j4" seems to be a bit high and a bit slowing down my > work while the compilation is going on, but bearable. > > I was

Re: Is there a way to improve partial compilation times?

So, I'm on Dell XPS 13 (9350), and I don't think that toying with MOZ_MAKE_FLAGS will help me here. "-j4" seems to be a bit high and a bit slowing down my work while the compilation is going on, but bearable. I was just wondering if really two days of patches landing in Gecko should result in

Re: Is there a way to improve partial compilation times?

On 03/07/2017 11:10 AM, Boris Zbarsky wrote: On 3/7/17 2:05 PM, Mike Conley wrote: FWIW, the MOZ_MAKE_FLAGS bit can probably be removed, as I believe mach will just choose the optimal number based on examining your processor cores. Except mach's definition of "optimal" is "maybe optimize for

Re: Is there a way to improve partial compilation times?

On 3/7/17 2:05 PM, Mike Conley wrote: FWIW, the MOZ_MAKE_FLAGS bit can probably be removed, as I believe mach will just choose the optimal number based on examining your processor cores. Except mach's definition of "optimal" is "maybe optimize for compile throughput", not "optimize for doing

Re: Is there a way to improve partial compilation times?

On Tue, Mar 7, 2017 at 8:05 PM, Mike Conley wrote: > FWIW, the MOZ_MAKE_FLAGS bit can probably be removed, as I believe mach > will just choose the optimal number based on examining your processor > cores. >From my experience the chosen value is too conservative, I think

Re: Is there a way to improve partial compilation times?

Perhaps you need a faster computer(s). Are you building on Windows? With icecream on Linux I can do a full clobber build in ~5 minutes. -Jeff On Tue, Mar 7, 2017 at 1:59 PM, wrote: > I'm on Linux (Arch), with ccache, and I work on mozilla-central, rebasing my >

Is there a way to improve partial compilation times?

I'm on Linux (Arch), with ccache, and I work on mozilla-central, rebasing my bookmarks on top of central every couple days. And every couple days the recompilation takes 50-65 minutes. Here's my mozconfig: ▶ cat mozconfig mk_add_options MOZ_MAKE_FLAGS="-j4" mk_add_options AUTOCLOBBER=1