Re: Intent to remove client.mk

[Nicholas Nethercote]

This is excellent news.

Relatedly, I want to particularly say that I think moz.build files are
great. The syntax and semantics are very clear. They're easy to modify.
They handle both simple cases and complex cases well. They pretty much
never get in the way, which is exactly what a developer wants from a build
system. The contrast with Makefiles is... significant.

Nick

On Sat, Oct 28, 2017 at 10:16 AM, Gregory Szorc  wrote:

> client.mk has existed since 1998 (https://hg.mozilla.org/
> experimental/mozilla-central-cvs/rev/0a5e1fa423bd). Before `mach`,
> client.mk was the recommended way to build Firefox and perform other
> build tasks. client.mk has rarely changed in recent years because the
> build system maintainers have been working around it and because not much
> has changed around how the very early parts of "invoke the build system"
> work.
>
> The build system maintainers are making a significant push towards
> supporting building Firefox without GNU make in this and future quarters.
>
> Since client.mk is a make file and we want to not require make to build
> Firefox, client.mk is incompatible with our vision for the future of the
> Firefox build system. Furthermore, client.mk represents an ancient piece
> of build system infrastructure and its convoluted content creates
> consternation and inhibits forward progress. For these reasons, I'm
> announcing the intent to remove client.mk.
>
> If you have tools, infrastructure, workflows, etc that use client.mk - or
> any direct use of `make` for that matter - please transition them to `mach`
> commands and/or check with a build peer regarding your use case. You can
> file bugs regarding client.mk dependencies against bug 1412398.
>
> Thank you for your understanding.
>
> ___
> dev-builds mailing list
> dev-bui...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-builds
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More ThinkStation P710 Nvidia tips (was Re: Faster gecko builds with IceCC on Mac and Linux)

[Robert O'Callahan]

BTW can someone forward this entire thread to their friends at AMD so AMD
will fix their CPUs to run rr? They're tantalizingly close :-/.

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to remove client.mk

[Kris Maglione]

On Fri, Oct 27, 2017 at 04:16:01PM -0700, Gregory Szorc wrote:

client.mk has existed since 1998 (
https://hg.mozilla.org/experimental/mozilla-central-cvs/rev/0a5e1fa423bd).
Before `mach`, client.mk was the recommended way to build Firefox and
perform other build tasks. client.mk has rarely changed in recent years
because the build system maintainers have been working around it and
because not much has changed around how the very early parts of "invoke the
build system" work.

The build system maintainers are making a significant push towards
supporting building Firefox without GNU make in this and future quarters.


I think I speak for all of us when I say: \o/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More ThinkStation P710 Nvidia tips (was Re: Faster gecko builds with IceCC on Mac and Linux)

[Gabriele Svelto]

On 28/10/2017 01:08, Sophana "Soap" Aik wrote:
> Thanks Gabriele, that poses a problem then for the system build we have
> in mind here as the i9's do not support ECC memory. That may have to be
> a separate system with a Xeon.

Xeon-W processors are identical to the i9 but come with more
workstation/server-oriented features such as ECC memory support; they
are also offered with slightly higher peak clock speed to equivalent
i9s. Here's a side-by-side comparison of the top 4 SKUs in both families:

https://ark.intel.com/compare/123589,126709,123767,126707,125042,123613,126793,126699

 Gabriele



signature.asc
Description: OpenPGP digital signature
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More ThinkStation P710 Nvidia tips (was Re: Faster gecko builds with IceCC on Mac and Linux)

[Gregory Szorc]

Yeah. Only the Xeons and ThreadRipper (as our potential high core count
machines) support ECC. rr, ECC, or reasonable costs: pick at most two :/

On Fri, Oct 27, 2017 at 4:08 PM, Sophana "Soap" Aik 
wrote:

> Thanks Gabriele, that poses a problem then for the system build we have in
> mind here as the i9's do not support ECC memory. That may have to be a
> separate system with a Xeon.
>
> On Fri, Oct 27, 2017 at 3:58 PM, Gabriele Svelto 
> wrote:
>
>> On 27/10/2017 01:02, Gregory Szorc wrote:
>> > Sophana (CCd) is working on a new system build right now. It will be
>> based
>> > on the i9's instead of dual socket Xeons and should be faster and
>> cheaper.
>>
>> ... and lacking ECC memory. Please whatever CPU is chosen make sure it
>> has ECC support and the machine comes loaded with ECC memory. Developer
>> boxes usually ship with plenty of memory, and they can stay on for days
>> without a reboot churning at builds and tests. Memory errors happen and
>> they can ruin days of work if they hit you at the wrong time.
>>
>>  Gabriele
>>
>>
>>
>
>
> --
> moz://a
> Sophana "Soap" Aik
> IT Vendor Management Analyst
> IRC/Slack: soap
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Intent to remove client.mk

[Gregory Szorc]

client.mk has existed since 1998 (
https://hg.mozilla.org/experimental/mozilla-central-cvs/rev/0a5e1fa423bd).
Before `mach`, client.mk was the recommended way to build Firefox and
perform other build tasks. client.mk has rarely changed in recent years
because the build system maintainers have been working around it and
because not much has changed around how the very early parts of "invoke the
build system" work.

The build system maintainers are making a significant push towards
supporting building Firefox without GNU make in this and future quarters.

Since client.mk is a make file and we want to not require make to build
Firefox, client.mk is incompatible with our vision for the future of the
Firefox build system. Furthermore, client.mk represents an ancient piece of
build system infrastructure and its convoluted content creates
consternation and inhibits forward progress. For these reasons, I'm
announcing the intent to remove client.mk.

If you have tools, infrastructure, workflows, etc that use client.mk - or
any direct use of `make` for that matter - please transition them to `mach`
commands and/or check with a build peer regarding your use case. You can
file bugs regarding client.mk dependencies against bug 1412398.

Thank you for your understanding.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More ThinkStation P710 Nvidia tips (was Re: Faster gecko builds with IceCC on Mac and Linux)

[Gabriele Svelto]

On 27/10/2017 01:02, Gregory Szorc wrote:
> Sophana (CCd) is working on a new system build right now. It will be based
> on the i9's instead of dual socket Xeons and should be faster and cheaper.

... and lacking ECC memory. Please whatever CPU is chosen make sure it
has ECC support and the machine comes loaded with ECC memory. Developer
boxes usually ship with plenty of memory, and they can stay on for days
without a reboot churning at builds and tests. Memory errors happen and
they can ruin days of work if they hit you at the wrong time.

 Gabriele




signature.asc
Description: OpenPGP digital signature
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More ThinkStation P710 Nvidia tips (was Re: Faster gecko builds with IceCC on Mac and Linux)

[Steve Fink]

Not necessarily relevant to this specific discussion, but I'm on a 
Lenovo P50 running Linux, and wanted to offer up my setup as a 
datapoint. (It's not quite either a recommendation or a word of warning. 
A combination.)


I use Linux (Fedora 25) as the host OS, with two external monitors plus 
the laptop screen. Windows is installed natively on a separate 
partition. For a while, I used VirtualBox to run the native Windows 
installation in a VM, rebooting into Windows on the rare occasions when 
I needed the extra performance or I wanted to diagnose whether something 
was virtualization-specific. The machine has an Intel HD Graphics P530 
and a Quadro M2000M. One external monitor is hooked up via DP, the other 
HDMI. I have a single desktop spread across all three (as in, I can drag 
windows between them). I use the nouveau driver. Videoconferencing works.


It all works well enough. There are large caveats and drawbacks. It took 
an insane amount of configuration attempts to get it to where it is, and 
again: there are large caveats and drawbacks.


Whichever monitor is on HDMI is at the wrong resolution (1920x1080 
instead of its native 1920x1200). I am running X11 because Wayland 
doesn't work. (Though I'm fine with that, because I'm old school and I 
run xfce4.) The laptop screen is HiDPI and when I disconnect from the 
external screens, I have to zoom everything in, which only partially 
works (eg Firefox's chrome is still small). I used to use xrandr with 
--scale 0.5x0.5 to expand everything, but that caused too many issues. 
When I turn my external monitors back on in the morning, one of them 
comes up fine and the other does not display anything until I do 
ctrl-alt-f3 alt-f2 to switch to VT3 then back to VT2. When I reconnect 
my monitors, it will often mirror a single image to all 3 displays, and 
I have to turn mirroring on and then back off again then drag my 
monitors back to the right relative positioning.


I use the nouveau driver now. I started out with nouveau and it was 
causing lots of random lockups, so I switched to the proprietary nvidia 
driver. It did not work well when I disconnected and reconnected the 
external monitors. Nor sometimes if I suspended and resumed. I have no 
idea why nouveau has magically become stable; probably some update or other.


My Windows setup broke when I switched from an HDD to an SSD. First, it 
stopped booting natively and I could only run it through the VM. Now it 
hangs on boot even with the VM unless I boot into safe mode. I have sunk 
more time than I'm willing to admit in trying to fix it and failed. My 
plan is to start over with a disk with Windows preinstalled and clone my 
Linux partitions over to it, but I can't muster the energy to dive back 
into the nightmare and I don't really need Windows very often anyway.



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: WebIDL consts now generate C++ definitions

[Boris Zbarsky]

On 10/27/17 12:33 PM, Steve Fink wrote:
Within Spidermonkey, our rule is to never #include windows.h 
directly, but always via jswin.h


Ah, I thought I'd seen this sort of thing somewhere; I had recalled a 
header that you could include to under "all the bad windows.h stuff"...


This approach doesn't work for Gecko out of the box, because there's 
third-party code Gecko uses whose headers include windows.h, as you note.


Now the Gecko approach is being extended to creating a WebIDL attribute 
for this. What are the objections to doing things the jswin.h way?


Expediency, in this case...

I understand there may be difficulties with third party code including 
windows.h, but that just means that you additionally need to include 
mozwin.h (or whatever we choose to call it) after including those header 
files


The hard part is finding where all "those header files" are included. 
They're everywhere, unfortunately, because the chromium-ipc headers 
include windows.h.  :(


And you can't easily use try to get your way around this, because msvc 
very helpfully doesn't give include paths for its errors.


Then if you need to define a webidl constant that conflicts with 
windows.h damage, you'd add it to mozwin.h with no magic attribute 
required.


Yes, this would be absolutely ideal from my point of view.  It's a much 
bigger project than I could justify asking Kyle to do for what was 
fundamentally a spare-time patch


-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: WebIDL consts now generate C++ definitions

[Steve Fink]

On 10/26/17 8:22 PM, Kyle Machulis wrote:

This addition also creates the new [NeedsWindowsUndef] extended attribute,
as some WebIDL constant names conflict with windows.h macros, and undef'ing
them in the binding generation is easier than tracking down include order
issues.


Gecko seems to handle windows.h in (what feels to me like) an ad hoc 
way, where individual files will #undef whatever macros interfere with 
them. Within Spidermonkey, our rule is to never #include windows.h 
directly, but always via jswin.h, which is just


#ifdef XP_WIN

# include 

# undef min

# undef max

# undef assert

.

.

.

#endif

Now the Gecko approach is being extended to creating a WebIDL attribute 
for this. What are the objections to doing things the jswin.h way? I 
understand there may be difficulties with third party code including 
windows.h, but that just means that you additionally need to include 
mozwin.h (or whatever we choose to call it) after including those header 
files, and before any code that might make use of those tokens -- no 
different than the decision of where to place an #undef.


Then if you need to define a webidl constant that conflicts with 
windows.h damage, you'd add it to mozwin.h with no magic attribute required.


I am not confident that the mozwin.h approach is fully workable, as 
there may be cases where the windows.h macros are used by other headers 
or something. That is why I'm asking for objections.



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Stylo is built as default even if Fennec/Android

[Nicholas Alexander]

On Fri, Oct 27, 2017 at 6:24 AM, Makoto Kato 
wrote:

> Hi, all.
>
> You know, stylo (Quantum CSS) is turned on Firefox Desktop only.
> Stylo team is working very hard for Android too, then all reftests and
> mochitests are passed now even if Fennec/Android.
>
> So I would like to turn on stylo build on Android of Nightly channel
> for feedback.  Although the preference still keeps off as default on
> 58 cycle, it will be turned on 59 cycle.
>

Great work, Makoto and team!  Excited to see new technology on Android.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Better triage for intermittent leaks in tests?

[Andrew McCreight]

I half-heartedly keep an eye on intermittent leaks, so I can more active in
looking at them. We'll need more logging for them to be actionable, but I
guess the high frequency ones are easier to catch.

Andrew


On Thu, Oct 26, 2017 at 10:20 AM, Geoffrey Brown  wrote:

> Some of our most troublesome intermittent test failures are leak bugs
> ("Intermittent LeakSanitizer | leak at ..." or "Intermittent leakcheck
> | default process:  bytes leaked ...") Even when they fail
> frequently, these bugs often seem to remain unresolved for many weeks.
> Leaks are sometimes not strongly associated with a particular test,
> making it difficult to assign to a useful bugzilla component, or find
> a motivated triage owner or assignee.
>
> I feel like these bugs are not being connected to the "right" people
> effectively. Could we do better?
>
> For instance, could we assign all leak bugs to a specific bugzilla
> component, with a "leak guru" as triage owner? Volunteers??
>
> - Geoff
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Stylo is built as default even if Fennec/Android

[Makoto Kato]

Hi, all.

You know, stylo (Quantum CSS) is turned on Firefox Desktop only.
Stylo team is working very hard for Android too, then all reftests and
mochitests are passed now even if Fennec/Android.

So I would like to turn on stylo build on Android of Nightly channel
for feedback.  Although the preference still keeps off as default on
58 cycle, it will be turned on 59 cycle.

After landing bug 1411802 [*1], you can enable stylo with
layout.css.servo.enabled=true via about:config even if Android.

Also, this change is nightly channel only.  Even if 58, beta and
release channel for Fennec don't build stylo due to package size
(incremented size is 1.6MB when using NDK11c's gcc).

And, developers don't require additional build config after this
change.  But you might require ./mach bootstrap if you don't install
clang (for bindgen) yet.  Of course, we can use --disable-stylo not to
build sytlo.

If you want to know current status for Android/stylo, please watch bug
1366049 [*2] that is meta bug.


-- Makoto Kato

[*1] https://bugzilla.mozilla.org/show_bug.cgi?id=1411802
[*2] https://bugzilla.mozilla.org/show_bug.cgi?id=1366049
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More ThinkStation P710 Nvidia tips (was Re: Faster gecko builds with IceCC on Mac and Linux)

[Robert O'Callahan]

On Fri, Oct 27, 2017 at 2:34 AM, Henri Sivonen  wrote:

> And the downsides don't even end there. rr didn't work. Plus other
> stuff not worth mentioning here.
>

Turns out that rr not working with Nvidia on Ubuntu 17.10 was actually an
rr issue triggered by the Ubuntu libc upgrade, not Nvidia's fault. I just
fixed it in rr master. We'll do an rr release soon, because the libc update
required a number of rr fixes.

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Visual Studio 2017 coming soon

[Kevin Brosnan]

On Fri, Oct 27, 2017 at 12:03 AM, Xidorn Quan  wrote:
> On Thu, Oct 26, 2017, at 12:24 PM, Ted Mielczarek wrote:
>> On the other hand, it's easier to
>> justify dropping support if VS is the last compiler holding us back from
>> being able to use new C++ features.
>
> FWIW, it's not at the moment. Currently we are really only blocked on
> GCC for new C++ features.
>
> From the language features table [1] we are still required to support
> GCC 4.9. IIRC it was because of Android build where GCC 4.9 is the
> highest version provided by NDK. Have we switched our Android build to
> Clang? If so, we should probably also bump our GCC requirement to
> unblock more C++ features. This is another topic, though.
>
> [1] https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code
>
>
> - Xidorn
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

Nathan Froyd is working on getting clang builds for Android running in
https://bugzilla.mozilla.org/show_bug.cgi?id=1163171


Kevin
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Visual Studio 2017 coming soon

[Xidorn Quan]

On Thu, Oct 26, 2017, at 12:24 PM, Ted Mielczarek wrote:
> On the other hand, it's easier to
> justify dropping support if VS is the last compiler holding us back from
> being able to use new C++ features.

FWIW, it's not at the moment. Currently we are really only blocked on
GCC for new C++ features.

>From the language features table [1] we are still required to support
GCC 4.9. IIRC it was because of Android build where GCC 4.9 is the
highest version provided by NDK. Have we switched our Android build to
Clang? If so, we should probably also bump our GCC requirement to
unblock more C++ features. This is another topic, though.

[1] https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code


- Xidorn
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More ThinkStation P710 Nvidia tips (was Re: Faster gecko builds with IceCC on Mac and Linux)

[Henri Sivonen]

On Fri, Oct 27, 2017 at 4:48 AM, Sophana "Soap" Aik  wrote:
> Hello everyone, great feedback that I will keep in mind and continue to work
> with our vendors to find the best solution with. One of the cards that I was
> looking at is fairly cheap and can at least drive multi-displays (even 4K
> 60hz) was the Nvidia Quadro P600.

Is that GPU known to be well-supported by Nouveau of Ubuntu 16.04 vintage?

I don't want to deny a single-GPU multi-monitor setup to anyone for
whom that's the priority, but considering how much damage the Quadro
M2000 has done to my productivity (and from what I've heard from other
people on the DOM team, I gather I'm not the only one who has had
trouble with it), the four DisplayPort connectors on it look like very
bad economics.

I suggest these two criteria be considered for developer workstations
in addition to build performance:
 1) The CPU is compatible with rr (at present, this means that the CPU
has to be from Intel and not from AMD)
 2) The GPU offered by default (again, I don't want to deny multiple
DisplayPort connectors on a single GPU to people who request them)
works well in OpenGL mode (i.e. without llvmpipe activating) without
freezes using the Open Source drivers included in Ubuntu LTS and
Fedora.

On Fri, Oct 27, 2017 at 2:36 AM, Gregory Szorc  wrote:
> Host OS matters for finding UI bugs and issues with add-ons (since lots of
> add-on developers are also on Linux or MacOS).

I think it's a bad tradeoff to trade off the productivity of
developers working on the cross-platform core of Firefox in order to
get them to report Windows-specific bugs. We have people in the
organization who aren't developing the cross-platform core and who are
running Windows anyway. I'd prefer the energy currently put into
getting developers of the cross-platform core to use Windows to be put
into getting the people who use Windows anyway to use Nightly. (It
saddens me to hear fear of Nightly from within Mozilla.)

> Unless you have requirements that prohibit using a VM, I encourage using this 
> setup.

For some three-four years, I developed in a Linux VM hosted on
Windows. I'm not too worried about the performance overhead of a VM.
However, rr is such an awesome tool that it justifies running Linux as
the host OS.

> I concede that performance testing on i9s and Xeons is not at all indicative
> of the typical user :)

Indeed. Still, we don't need Nvidia professional GPUs for build times,
so boring well-supported consumer-grade GPUs would also be in the
interest of "using what our users use" even if paired with a CPU that
isn't representative of typical users' computers.

On Fri, Oct 27, 2017 at 1:13 AM, Thomas Daede  wrote:
> I have a RX 460 in a desktop with F26 and can confirm that it works
> out-of-the-box at 4K with the open source drivers, and will happily run
> Pathfinder demos at <16ms frame time.* It also seems to run Servo's
> Webrender just fine.
>
> It's been superseded by the RX 560, which is a faster clock of the same
> chip. It should work just as well, but might need a slightly newer
> kernel than the 4xx to pick up the pci ids (maybe a problem with LTS
> ubuntu?) The RX 570 and 580 should be fine too, but require power
> connectors. The Vega models are waiting on a kernel-side driver rewrite
> (by AMD) that will land in 4.15 (hopefully with new features and
> regressions to the RX 5xx series...)

Thank you. I placed an order for an RX 460.

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform