Re: Building js/xul/css from Firefox faster

2015-10-05 Thread Benoit Girard 
This is great progress!

I had hope that something like this would also include the 'build binaries'
DAG. It might make it slightly slower but it should still be very fast and
lessen the cognitive load. I was under the impression that 'build binaries'
at some point was a single DAG but it doesn't seem to be the case anymore
(I see make entering/exiting directories now)?

It would be nice if we got to a state where we had a 'build faster' like
target that supported all the most frequently modified extensions
all-in-one (say cpp/h/js/jsm/idl/css/xul).

On Mon, Oct 5, 2015 at 12:30 PM, Gregory Szorc  wrote:

> Basically a set of build actions related to frontend development and known
> by moz.build files are assembled in a single make file that contains a
> single DAG and doesn't need to recurse into directories. See
> python/mozbuild/mozbuild/backend/fastermake.py and /faster/Makefile
> for the low-level details.
>
> On Fri, Oct 2, 2015 at 12:18 PM, Justin Dolske  wrote:
>
> > Nice! Out of curiosity, how does "faster" work? Does it just ignore build
> > targets/directories that involve native code?
> >
> > FWIW, I benchmarked various no-changes builds with yesterday's m-c, on my
> > low-end Surface Pro 3 (i3, 4GB)...
> >
> > mach build: 7:38
> > mach build browser: 0:43
> > mach build toolkit: 1:42
> > mach build faster: 0:22
> >
> > Big wins!
> >
> > Justin
> >
> > On Thu, Oct 1, 2015 at 5:23 PM, Mike Hommey  wrote:
> >
> >> Hi,
> >>
> >> I recently landed a new build backend that, if you opt-in to running it,
> >> will make your non-C++ changes to Firefox more readily available in
> >> local builds.
> >>
> >> After you built Firefox normally once, and assuming you only changed
> >> non-C++ code, you can now use the following command for a faster
> >> incremental build:
> >>   ./mach build faster
> >>
> >> Now, since this is fresh out of the oven, there are a few things to
> >> know:
> >> - it doesn't handle removing files
> >> - it doesn't handle files that would end up outside of dist/bin
> >> - it doesn't handle a few things like the files from the default profile
> >> - it currently only works for desktop Firefox
> >>
> >> Obviously, this is not an end state, so things will improve in the
> >> future, but it should work well enough for most day-to-day work for
> >> desktop Firefox, thus this message.
> >>
> >> On my machine, `mach build faster` completes in about 4 seconds for an
> >> incremental build. This should get even faster very soon.
> >>
> >> Additionally, while requiring some manual steps (which bug 1207888 and
> >> bug 1207890 will help with), it is also possible to use this to build
> >> desktop Firefox without actually building any C++. Here is how that
> >> goes:
> >> - Run `./mach configure` with a mozconfig containing:
> >>   ac_add_options --disable-compile-environment
> >> - Download and unpack a nightly
> >> - Use `./mach python toolkit/mozapps/installer/unpack.py `, where
> >>is the nightly's firefox/ directory.
> >> - Move that fully unpacked nightly to $objdir/dist/bin (for mac, that
> >>   involves more fiddling, because dist/bin is a somewhat flattened
> >>   version of the .app directory)
> >> - Ensure the files in $objdir/dist/bin are older than the source files.
> >> - Run `./mach build faster`.
> >> - On mac, you will need to run something like (untested)
> >> ./mach build browser/app/repackage
> >>
> >> After that $objdir/dist/bin should contain a bastardized Firefox, with
> >> xul, js, etc. coming from the source tree, and the remainder still being
> >> there from the original nightly.
> >>
> >> `mach run` should work with that.
> >>
> >> Cheers,
> >>
> >> Mike
> >> ___
> >> firefox-dev mailing list
> >> firefox-...@mozilla.org
> >> https://mail.mozilla.org/listinfo/firefox-dev
> >>
> >
> >
> > ___
> > firefox-dev mailing list
> > firefox-...@mozilla.org
> > https://mail.mozilla.org/listinfo/firefox-dev
> >
> >
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Building js/xul/css from Firefox faster

2015-10-05 Thread Benoit Girard 
This is great progress!

I had hope that something like this would also include the 'build binaries'
DAG. It might make it slightly slower but it should still be very fast and
lessen the cognitive load. I was under the impression that 'build binaries'
at some point was a single DAG but it doesn't seem to be the case anymore
(I see make entering/exiting directories now)?

It would be nice if we got to a state where we had a 'build faster' like
target that supported all the most frequently modified extensions
all-in-one (say cpp/h/js/jsm/idl/css/xul).


On Mon, Oct 5, 2015 at 12:30 PM, Gregory Szorc  wrote:

> Basically a set of build actions related to frontend development and known
> by moz.build files are assembled in a single make file that contains a
> single DAG and doesn't need to recurse into directories. See
> python/mozbuild/mozbuild/backend/fastermake.py and /faster/Makefile
> for the low-level details.
>
> On Fri, Oct 2, 2015 at 12:18 PM, Justin Dolske  wrote:
>
> > Nice! Out of curiosity, how does "faster" work? Does it just ignore build
> > targets/directories that involve native code?
> >
> > FWIW, I benchmarked various no-changes builds with yesterday's m-c, on my
> > low-end Surface Pro 3 (i3, 4GB)...
> >
> > mach build: 7:38
> > mach build browser: 0:43
> > mach build toolkit: 1:42
> > mach build faster: 0:22
> >
> > Big wins!
> >
> > Justin
> >
> > On Thu, Oct 1, 2015 at 5:23 PM, Mike Hommey  wrote:
> >
> >> Hi,
> >>
> >> I recently landed a new build backend that, if you opt-in to running it,
> >> will make your non-C++ changes to Firefox more readily available in
> >> local builds.
> >>
> >> After you built Firefox normally once, and assuming you only changed
> >> non-C++ code, you can now use the following command for a faster
> >> incremental build:
> >>   ./mach build faster
> >>
> >> Now, since this is fresh out of the oven, there are a few things to
> >> know:
> >> - it doesn't handle removing files
> >> - it doesn't handle files that would end up outside of dist/bin
> >> - it doesn't handle a few things like the files from the default profile
> >> - it currently only works for desktop Firefox
> >>
> >> Obviously, this is not an end state, so things will improve in the
> >> future, but it should work well enough for most day-to-day work for
> >> desktop Firefox, thus this message.
> >>
> >> On my machine, `mach build faster` completes in about 4 seconds for an
> >> incremental build. This should get even faster very soon.
> >>
> >> Additionally, while requiring some manual steps (which bug 1207888 and
> >> bug 1207890 will help with), it is also possible to use this to build
> >> desktop Firefox without actually building any C++. Here is how that
> >> goes:
> >> - Run `./mach configure` with a mozconfig containing:
> >>   ac_add_options --disable-compile-environment
> >> - Download and unpack a nightly
> >> - Use `./mach python toolkit/mozapps/installer/unpack.py `, where
> >>is the nightly's firefox/ directory.
> >> - Move that fully unpacked nightly to $objdir/dist/bin (for mac, that
> >>   involves more fiddling, because dist/bin is a somewhat flattened
> >>   version of the .app directory)
> >> - Ensure the files in $objdir/dist/bin are older than the source files.
> >> - Run `./mach build faster`.
> >> - On mac, you will need to run something like (untested)
> >> ./mach build browser/app/repackage
> >>
> >> After that $objdir/dist/bin should contain a bastardized Firefox, with
> >> xul, js, etc. coming from the source tree, and the remainder still being
> >> there from the original nightly.
> >>
> >> `mach run` should work with that.
> >>
> >> Cheers,
> >>
> >> Mike
> >> ___
> >> firefox-dev mailing list
> >> firefox-...@mozilla.org
> >> https://mail.mozilla.org/listinfo/firefox-dev
> >>
> >
> >
> > ___
> > firefox-dev mailing list
> > firefox-...@mozilla.org
> > https://mail.mozilla.org/listinfo/firefox-dev
> >
> >
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Building js/xul/css from Firefox faster

2015-10-05 Thread Mike Hommey 
On Mon, Oct 05, 2015 at 03:08:15PM -0400, Benoit Girard wrote:
> This is great progress!
> 
> I had hope that something like this would also include the 'build binaries'
> DAG. It might make it slightly slower but it should still be very fast and
> lessen the cognitive load. I was under the impression that 'build binaries'
> at some point was a single DAG but it doesn't seem to be the case anymore
> (I see make entering/exiting directories now)?

It never was a single DAG, so that never changed. It however used to
have a trick that avoided part of the recursion, but it was complex and
didn't work to make 'build binaries' part of the normal build.

> It would be nice if we got to a state where we had a 'build faster' like
> target that supported all the most frequently modified extensions
> all-in-one (say cpp/h/js/jsm/idl/css/xul).

The eventual goal is for the 'build faster' thing to be the default, but
it's a long way there. I definitely want to add binaries to the
equation, but the first goal was to provide something that works well
enough to help frontend developers.

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Building js/xul/css from Firefox faster

2015-10-05 Thread Mike Hommey 
On Fri, Oct 02, 2015 at 12:18:12PM -0700, Justin Dolske wrote:
> Nice! Out of curiosity, how does "faster" work? Does it just ignore build
> targets/directories that involve native code?

Greg answered this :)

> FWIW, I benchmarked various no-changes builds with yesterday's m-c, on my
> low-end Surface Pro 3 (i3, 4GB)...
> 
> mach build: 7:38
> mach build browser: 0:43
> mach build toolkit: 1:42
> mach build faster: 0:22
> 
> Big wins!

Could you compare with 'mach build faster' with bug 1210642 and bug 1210687
applied?

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Building js/xul/css from Firefox faster

2015-10-05 Thread Gregory Szorc 
Basically a set of build actions related to frontend development and known
by moz.build files are assembled in a single make file that contains a
single DAG and doesn't need to recurse into directories. See
python/mozbuild/mozbuild/backend/fastermake.py and /faster/Makefile
for the low-level details.

On Fri, Oct 2, 2015 at 12:18 PM, Justin Dolske  wrote:

> Nice! Out of curiosity, how does "faster" work? Does it just ignore build
> targets/directories that involve native code?
>
> FWIW, I benchmarked various no-changes builds with yesterday's m-c, on my
> low-end Surface Pro 3 (i3, 4GB)...
>
> mach build: 7:38
> mach build browser: 0:43
> mach build toolkit: 1:42
> mach build faster: 0:22
>
> Big wins!
>
> Justin
>
> On Thu, Oct 1, 2015 at 5:23 PM, Mike Hommey  wrote:
>
>> Hi,
>>
>> I recently landed a new build backend that, if you opt-in to running it,
>> will make your non-C++ changes to Firefox more readily available in
>> local builds.
>>
>> After you built Firefox normally once, and assuming you only changed
>> non-C++ code, you can now use the following command for a faster
>> incremental build:
>>   ./mach build faster
>>
>> Now, since this is fresh out of the oven, there are a few things to
>> know:
>> - it doesn't handle removing files
>> - it doesn't handle files that would end up outside of dist/bin
>> - it doesn't handle a few things like the files from the default profile
>> - it currently only works for desktop Firefox
>>
>> Obviously, this is not an end state, so things will improve in the
>> future, but it should work well enough for most day-to-day work for
>> desktop Firefox, thus this message.
>>
>> On my machine, `mach build faster` completes in about 4 seconds for an
>> incremental build. This should get even faster very soon.
>>
>> Additionally, while requiring some manual steps (which bug 1207888 and
>> bug 1207890 will help with), it is also possible to use this to build
>> desktop Firefox without actually building any C++. Here is how that
>> goes:
>> - Run `./mach configure` with a mozconfig containing:
>>   ac_add_options --disable-compile-environment
>> - Download and unpack a nightly
>> - Use `./mach python toolkit/mozapps/installer/unpack.py `, where
>>is the nightly's firefox/ directory.
>> - Move that fully unpacked nightly to $objdir/dist/bin (for mac, that
>>   involves more fiddling, because dist/bin is a somewhat flattened
>>   version of the .app directory)
>> - Ensure the files in $objdir/dist/bin are older than the source files.
>> - Run `./mach build faster`.
>> - On mac, you will need to run something like (untested)
>> ./mach build browser/app/repackage
>>
>> After that $objdir/dist/bin should contain a bastardized Firefox, with
>> xul, js, etc. coming from the source tree, and the remainder still being
>> there from the original nightly.
>>
>> `mach run` should work with that.
>>
>> Cheers,
>>
>> Mike
>> ___
>> firefox-dev mailing list
>> firefox-...@mozilla.org
>> https://mail.mozilla.org/listinfo/firefox-dev
>>
>
>
> ___
> firefox-dev mailing list
> firefox-...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Building js/xul/css from Firefox faster

2015-10-05 Thread Justin Dolske 
Nice! Out of curiosity, how does "faster" work? Does it just ignore build
targets/directories that involve native code?

FWIW, I benchmarked various no-changes builds with yesterday's m-c, on my
low-end Surface Pro 3 (i3, 4GB)...

mach build: 7:38
mach build browser: 0:43
mach build toolkit: 1:42
mach build faster: 0:22

Big wins!

Justin

On Thu, Oct 1, 2015 at 5:23 PM, Mike Hommey  wrote:

> Hi,
>
> I recently landed a new build backend that, if you opt-in to running it,
> will make your non-C++ changes to Firefox more readily available in
> local builds.
>
> After you built Firefox normally once, and assuming you only changed
> non-C++ code, you can now use the following command for a faster
> incremental build:
>   ./mach build faster
>
> Now, since this is fresh out of the oven, there are a few things to
> know:
> - it doesn't handle removing files
> - it doesn't handle files that would end up outside of dist/bin
> - it doesn't handle a few things like the files from the default profile
> - it currently only works for desktop Firefox
>
> Obviously, this is not an end state, so things will improve in the
> future, but it should work well enough for most day-to-day work for
> desktop Firefox, thus this message.
>
> On my machine, `mach build faster` completes in about 4 seconds for an
> incremental build. This should get even faster very soon.
>
> Additionally, while requiring some manual steps (which bug 1207888 and
> bug 1207890 will help with), it is also possible to use this to build
> desktop Firefox without actually building any C++. Here is how that
> goes:
> - Run `./mach configure` with a mozconfig containing:
>   ac_add_options --disable-compile-environment
> - Download and unpack a nightly
> - Use `./mach python toolkit/mozapps/installer/unpack.py `, where
>is the nightly's firefox/ directory.
> - Move that fully unpacked nightly to $objdir/dist/bin (for mac, that
>   involves more fiddling, because dist/bin is a somewhat flattened
>   version of the .app directory)
> - Ensure the files in $objdir/dist/bin are older than the source files.
> - Run `./mach build faster`.
> - On mac, you will need to run something like (untested)
> ./mach build browser/app/repackage
>
> After that $objdir/dist/bin should contain a bastardized Firefox, with
> xul, js, etc. coming from the source tree, and the remainder still being
> there from the original nightly.
>
> `mach run` should work with that.
>
> Cheers,
>
> Mike
> ___
> firefox-dev mailing list
> firefox-...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Building js/xul/css from Firefox faster

2015-10-02 Thread Neil 

Mike Hommey wrote:


 ./mach build faster


So is this the complete opposite of ./mach build binaries?

--
Warning: May contain traces of nuts.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Building js/xul/css from Firefox faster

2015-10-01 Thread Mike Hommey 
Hi,

I recently landed a new build backend that, if you opt-in to running it,
will make your non-C++ changes to Firefox more readily available in
local builds.

After you built Firefox normally once, and assuming you only changed
non-C++ code, you can now use the following command for a faster
incremental build:
  ./mach build faster

Now, since this is fresh out of the oven, there are a few things to
know:
- it doesn't handle removing files
- it doesn't handle files that would end up outside of dist/bin
- it doesn't handle a few things like the files from the default profile
- it currently only works for desktop Firefox

Obviously, this is not an end state, so things will improve in the
future, but it should work well enough for most day-to-day work for
desktop Firefox, thus this message.

On my machine, `mach build faster` completes in about 4 seconds for an
incremental build. This should get even faster very soon.

Additionally, while requiring some manual steps (which bug 1207888 and
bug 1207890 will help with), it is also possible to use this to build
desktop Firefox without actually building any C++. Here is how that
goes:
- Run `./mach configure` with a mozconfig containing:
  ac_add_options --disable-compile-environment
- Download and unpack a nightly
- Use `./mach python toolkit/mozapps/installer/unpack.py `, where
   is the nightly's firefox/ directory.
- Move that fully unpacked nightly to $objdir/dist/bin (for mac, that
  involves more fiddling, because dist/bin is a somewhat flattened
  version of the .app directory)
- Ensure the files in $objdir/dist/bin are older than the source files.
- Run `./mach build faster`.
- On mac, you will need to run something like (untested)
./mach build browser/app/repackage

After that $objdir/dist/bin should contain a bastardized Firefox, with
xul, js, etc. coming from the source tree, and the remainder still being
there from the original nightly.

`mach run` should work with that.

Cheers,

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform