Re: Update on rustc/clang goodness

[Sylvestre Ledru]

Le 04/06/2018 à 08:50, Henri Sivonen a écrit :
> On Wed, May 30, 2018 at 5:16 PM, Mike Hommey  wrote:
>> On Wed, May 30, 2018 at 02:40:01PM +0300, Henri Sivonen wrote:
>>> The Linux distro case is
>>> trickier than Mozilla's compiler choice. For CPUs that are tier-3 for
>>> Mozilla, we already tolerate less great performance attributes in
>>> order to enable availability, so distros keeping using GCC for tier-3
>>> probably isn't a problem. x86_64 could be a problem, though. If
>>> Firefox's performance becomes significantly dependent on having
>>> cross-language inlining, and I expect it will, having a substantial
>>> portion of the user base run without it while thinking they have a
>>> top-tier build could be bad. I hope we can get x86_64 Linux distros to
>>> track our compiler configuration closely.
>> That part might end up more difficult than one could expect.
>> Cross-language inlining is going to require rustc and clang having a
>> compatible llvm ir, and that's pretty much guaranteed to be a problem,
>> even for Mozilla.
> I thought the rustc codebase supported building with unpatched LLVM in
> order to let distros maintain one copy of LLVM source (if not .so). Is
> that not the case?
It is indeed the case. Debian/Ubuntu remove LLVM sources in a repackaging
phase and use the libraries provided by the distribution.

We (Mozilla) are also already applying patches on our local clang
builds. Example:

https://dxr.mozilla.org/mozilla-central/source/build/build-clang/clang-6-pre-linux64.json?q=clang-6-pre-linux64.json_type=direct#17

S

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Henri Sivonen]

On Wed, May 30, 2018 at 5:16 PM, Mike Hommey  wrote:
> On Wed, May 30, 2018 at 02:40:01PM +0300, Henri Sivonen wrote:
>> The Linux distro case is
>> trickier than Mozilla's compiler choice. For CPUs that are tier-3 for
>> Mozilla, we already tolerate less great performance attributes in
>> order to enable availability, so distros keeping using GCC for tier-3
>> probably isn't a problem. x86_64 could be a problem, though. If
>> Firefox's performance becomes significantly dependent on having
>> cross-language inlining, and I expect it will, having a substantial
>> portion of the user base run without it while thinking they have a
>> top-tier build could be bad. I hope we can get x86_64 Linux distros to
>> track our compiler configuration closely.
>
> That part might end up more difficult than one could expect.
> Cross-language inlining is going to require rustc and clang having a
> compatible llvm ir, and that's pretty much guaranteed to be a problem,
> even for Mozilla.

I thought the rustc codebase supported building with unpatched LLVM in
order to let distros maintain one copy of LLVM source (if not .so). Is
that not the case?

Why couldn't Mozilla build clang with Rust's LLVM fork and use that
for building releases? (And move Rust's fork forward as needed.)

(Also, what Ehsan said about IR compat suggests these might not even
need to be as closely synced.)

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Ehsan Akhgari]

On Wed, May 30, 2018 at 5:02 PM, Anthony Jones  wrote:

> On Thursday, 31 May 2018 02:44:51 UTC+12, Tom Ritter  wrote:
> > Oh.  Are we doing this rustc inlining development on a particular old
> > version of clang?  I'm not even close to getting CFI ready but I'm
> > basically working off llvm trunk as I'm finding and filing llvm bugs
> > and working with llvm devs to get them fixed
>
> We don't have experience with cross language LTO. It is difficult to
> predict but relatively easy to try. Here's what we do know:
>
> In order for the linker to do LTO, it needs to be able to understand the
> llvm-ir from both clang and rustc. For the most part this means having an
> lld verison more recent than the compilers. In theory the newer linker can
> read old versions of llvm-ir. Unfortunately llvm-ir changes over time and
> while some effort goes into making it backward compatible, it seems
> reasonable to expect a large gap between clang and rustc versions may fall
> short of excellence.
>
> We may be able to get away with heterogenous clang/rustc/lld versions, but
> it may be more trouble than it is worth. Alternatively we may need to keep
> rustc/clang/lld in lock step with all that it entails. It is possible that
> in the end we decide the whole thing isn't worth the trouble. Time will
> tell.
>

LLVM has this policy on IR backwards compatibility:
https://llvm.org/docs/DeveloperPolicy.html#ir-backwards-compatibility

In short, newer compiler versions can be expected to read and compile older
IRs going back to LLVM 3.0 correctly, while fewer guarantees exist to keep
debug info working.  All backwards incompatible changes are required to be
recorded in
https://github.com/llvm-mirror/llvm/blob/master/test/Bitcode/compatibility.ll
(and its versioned counterparts in the same directory) per each release.

Per this policy, as long as our LLVM toolchain comes from the same major
release we should be mostly fine, except for occasional bugs.

Cheers,
-- 
Ehsan
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Anthony Jones]

On Thursday, 31 May 2018 02:44:51 UTC+12, Tom Ritter  wrote:
> Oh.  Are we doing this rustc inlining development on a particular old
> version of clang?  I'm not even close to getting CFI ready but I'm
> basically working off llvm trunk as I'm finding and filing llvm bugs
> and working with llvm devs to get them fixed

We don't have experience with cross language LTO. It is difficult to predict 
but relatively easy to try. Here's what we do know:

In order for the linker to do LTO, it needs to be able to understand the 
llvm-ir from both clang and rustc. For the most part this means having an lld 
verison more recent than the compilers. In theory the newer linker can read old 
versions of llvm-ir. Unfortunately llvm-ir changes over time and while some 
effort goes into making it backward compatible, it seems reasonable to expect a 
large gap between clang and rustc versions may fall short of excellence.

We may be able to get away with heterogenous clang/rustc/lld versions, but it 
may be more trouble than it is worth. Alternatively we may need to keep 
rustc/clang/lld in lock step with all that it entails. It is possible that in 
the end we decide the whole thing isn't worth the trouble. Time will tell.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Tom Ritter]

On Wed, May 30, 2018 at 2:16 PM, Mike Hommey  wrote:
> I'm sure the day we'll have to choose between not
> doing cross-language inlining or upgrading clang for e.g. security
> features is relatively close.

Oh.  Are we doing this rustc inlining development on a particular old
version of clang?  I'm not even close to getting CFI ready but I'm
basically working off llvm trunk as I'm finding and filing llvm bugs
and working with llvm devs to get them fixed

-tom
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Mike Hommey]

On Wed, May 30, 2018 at 02:40:01PM +0300, Henri Sivonen wrote:
> On Wed, May 30, 2018 at 8:06 AM, Dave Townsend  wrote:
> > On Tue, May 29, 2018 at 10:03 PM Jeff Gilbert  wrote:
> >> I get that, but it reminds me of the reasons people give for "our
> >> website works best in $browser".
> >
> > I was concerned by this too but found myself swayed by the arguments in
> > https://blog.mozilla.org/nfroyd/2018/05/29/when-implementation-monoculture-right-thing/and
> > in particular the first comment there.
> 
> Indeed, the first comment there (by roc) gets to the point.
> 
> Additionally, the reasons for not supporting multiple browsers tend to
> be closer to the "didn't bother" kind whereas we're looking to get a
> substantial benefit from clang that MSVC and GCC don't offer to us but
> clang likely will: Cross-language inlining across code compiled with
> clang and code compiled with rustc.
> 
> To the extent Mozilla runs the compiler, it makes sense to go for the
> Open Source choice that allows us to deliver better on "performance as
> a feature". We still have at least one static analysis running on GCC,
> so I wouldn't expect GCC-compatibility to be dropped even if the app
> wouldn't be "best compiled with" GCC. The Linux distro case is
> trickier than Mozilla's compiler choice. For CPUs that are tier-3 for
> Mozilla, we already tolerate less great performance attributes in
> order to enable availability, so distros keeping using GCC for tier-3
> probably isn't a problem. x86_64 could be a problem, though. If
> Firefox's performance becomes significantly dependent on having
> cross-language inlining, and I expect it will, having a substantial
> portion of the user base run without it while thinking they have a
> top-tier build could be bad. I hope we can get x86_64 Linux distros to
> track our compiler configuration closely.

That part might end up more difficult than one could expect.
Cross-language inlining is going to require rustc and clang having a
compatible llvm ir, and that's pretty much guaranteed to be a problem,
even for Mozilla. I'm sure the day we'll have to choose between not
doing cross-language inlining or upgrading clang for e.g. security
features is relatively close.

Anyways, Mozilla builds today are built with PGO, and many downstreams
don't even do that...

Mike 
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[mcastelluccio]

We've also been running Windows code coverage builds with clang on CI for a 
while (since December), with almost all tests.

- Marco.


On Friday, May 11, 2018 at 2:35:57 AM UTC+2, Anthony Jones wrote:
> You may already know that the Low-Level Tools team support important tools 
> and code infrastructure. Lately we’ve also been improving our rustc/clang 
> (LLVM) story and I’d like bring everyone up to date.
> 
> There are a lot of important and interesting things going on:
> 
> * Michael Woerister and Nathan Froyd recently (mid-March) enabled Rust 
> incremental compilation for Firefox developers
> * Michael is experimenting with cross language inlining[1] (rustc/clang) 
> using LTO
> * Preliminary results show compiling LLVM with clang and using LTO on 
> rustc improves stylo compilation time by around 15% [on-my-machine 
> measurements]
> * LTO requires more work to support Firefox
> * Nick Nethercote has started speeding up rustc[2] and has already had a 
> number of wins
> * David Major has got Windows clang builds working with green tests
> * LTO is still buggy (but works well enough to run benchmarks) and 
> PGO won’t run on Windows
> * Win32 - clang with LTO w/o PGO is marginally faster than MSVC with 
> LTO/PGO
> * Win64 - clang with LTO w/o PGO is ~5-10% slower than MSVC with 
> LTO/PGO
> * Windows users can build locally with clang[3]
> * Mike Hommey is tracking improvements and regressions in the Rust 
> compiler (performance regression[4], crash reporting[5] and more crash 
> reporting[6])
> * Tom Tromey is continuing to work on first class Rust support in GDB and 
> LLDB
> 
> Note: this a summary of things rustc/clang stuff that is happening only.
> 
> Ultimately, I’d like to see us standardise on clang on all platforms because 
> it makes Rust/C++ integration better as well as making things simpler for 
> developers and build system maintainers. We’ll get more done if we make our 
> own lives simpler.
> 
> I have some specific requests for you:
> 
> Let me know if you have specific Firefox related cases where Rust is 
> slowing you down (thanks Jeff [7])
> Cross language inlining is coming - avoid duplication between Rust and 
> C++ in the name of performance 
> Do developer builds with clang
> 
> Anthony
> 
> [1] https://github.com/rust-lang/rust/issues/49879
> [2] 
> https://blog.mozilla.org/nnethercote/2018/04/30/how-to-speed-up-the-rust-compiler-in-2018/
> [3] 
> https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Building_Firefox_on_Windows_with_clang-cl
> [4] https://github.com/rust-lang/rust/issues/49873
> [5] https://bugzilla.mozilla.org/show_bug.cgi?id=1456150
> [6] https://bugzilla.mozilla.org/show_bug.cgi?id=1448868
> [7] https://github.com/rust-lang/rust/issues/50584

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Henri Sivonen]

On Wed, May 30, 2018 at 8:06 AM, Dave Townsend  wrote:
> On Tue, May 29, 2018 at 10:03 PM Jeff Gilbert  wrote:
>> I get that, but it reminds me of the reasons people give for "our
>> website works best in $browser".
>
> I was concerned by this too but found myself swayed by the arguments in
> https://blog.mozilla.org/nfroyd/2018/05/29/when-implementation-monoculture-right-thing/and
> in particular the first comment there.

Indeed, the first comment there (by roc) gets to the point.

Additionally, the reasons for not supporting multiple browsers tend to
be closer to the "didn't bother" kind whereas we're looking to get a
substantial benefit from clang that MSVC and GCC don't offer to us but
clang likely will: Cross-language inlining across code compiled with
clang and code compiled with rustc.

To the extent Mozilla runs the compiler, it makes sense to go for the
Open Source choice that allows us to deliver better on "performance as
a feature". We still have at least one static analysis running on GCC,
so I wouldn't expect GCC-compatibility to be dropped even if the app
wouldn't be "best compiled with" GCC. The Linux distro case is
trickier than Mozilla's compiler choice. For CPUs that are tier-3 for
Mozilla, we already tolerate less great performance attributes in
order to enable availability, so distros keeping using GCC for tier-3
probably isn't a problem. x86_64 could be a problem, though. If
Firefox's performance becomes significantly dependent on having
cross-language inlining, and I expect it will, having a substantial
portion of the user base run without it while thinking they have a
top-tier build could be bad. I hope we can get x86_64 Linux distros to
track our compiler configuration closely.

I do feel bad for the GCC devs, but it's worth noting that this is a
result of a deliberate decision not to modularize GCC for licensing
strategy reasons while LLVM has been designed as a module, demand for
which has had solid technical reasons. The modularity meant it made
more sense to build rustc on LLVM than on GCC and now that technical
design leads to better synergy with clang.

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Dave Townsend]

On Tue, May 29, 2018 at 10:03 PM Jeff Gilbert  wrote:

> I get that, but it reminds me of the reasons people give for "our
> website works best in $browser".
>

I was concerned by this too but found myself swayed by the arguments in
https://blog.mozilla.org/nfroyd/2018/05/29/when-implementation-monoculture-right-thing/and
in particular the first comment there.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Jeff Gilbert]

I get that, but it reminds me of the reasons people give for "our
website works best in $browser".

There are also other less-obvious benefits where having multiple
backends can illuminate bugs and deviations from standards, as well as
having another set of warnings and static analysis passes. Once we go
monoculture, it's also really hard to go back, since non-portable
dependencies (both deliberate and accidental) start to pile up.

Now we could deliberately retain these advantages by keeping non-clang
compilers at at least tier-2, while leveraging clang where useful.
(looks like not win64 yet perf-wise, anyways) It would be nice to
establish our plan here.

On Tue, May 29, 2018 at 9:10 PM, Anthony Jones  wrote:
> On Wednesday, 30 May 2018 08:48:12 UTC+12, Jeff Gilbert  wrote:
>> It would be sad to see us standardize on a clang monoculture.
>
> It pays not to be sentimental about tools. We get better practicality and 
> productivity by focusing on clang. Using the same compiler across all 
> platforms means that we can do more with less effort.
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Anthony Jones]

On Wednesday, 30 May 2018 08:48:12 UTC+12, Jeff Gilbert  wrote:
> It would be sad to see us standardize on a clang monoculture.

It pays not to be sentimental about tools. We get better practicality and 
productivity by focusing on clang. Using the same compiler across all platforms 
means that we can do more with less effort.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Mike Hommey]

On Wed, May 30, 2018 at 09:19:29AM +1000, Nicholas Nethercote wrote:
> On Wed, May 30, 2018 at 7:58 AM, Gregory Szorc  wrote:
> 
> >
> > MSVC is a separate beast. It is a great compiler.
> >
> 
> FWIW: numerous times I have been stymied by MSVC's bugs or lack of
> features. Bug 1449787 is a recent example.

Also:
 https://bugzilla.mozilla.org/show_bug.cgi?id=1460341#c22
 https://bugzilla.mozilla.org/show_bug.cgi?id=1460341#c24
 https://bugzilla.mozilla.org/show_bug.cgi?id=1464036#c17

MSVC generates static initializers for constexpr constructors!

(And we're tracking their number as of bug 1464522) 

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Nicholas Nethercote]

On Wed, May 30, 2018 at 7:58 AM, Gregory Szorc  wrote:

>
> MSVC is a separate beast. It is a great compiler.
>

FWIW: numerous times I have been stymied by MSVC's bugs or lack of
features. Bug 1449787 is a recent example.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Gregory Szorc]

On Tue, May 29, 2018 at 1:48 PM, Jeff Gilbert  wrote:

> It would be sad to see us standardize on a clang monoculture.
>

I am sympathetic to that concern. (I have similar monoculture fears that
the open source world is over-pivoting towards non-OSS platforms like
GitHub and Slack as well.)

We will still support GCC on at least Linux because many downstream
distributions rely on it. I'm not sure what tier it will be supported at.
But I suspect it will be tier 1 for the foreseeable future. Although we may
not be able to fully test every revision with GCC due to capacity/cost
reasons. We can certainly get build coverage in CI and periodic test
coverage. So maybe that means tier 2 by definition. But nobody on the build
side is seriously talking about dropping GCC support completely.

MSVC is a separate beast. It is a great compiler. But it is also closed
source and Windows only. Visual Studio is a great IDE and getting better
with every release. So that would be a compelling reason to keep MSVC
support. But Visual Studio supports Clang natively these days. And you can
always configure solutions to "shell out" to an alternate build backend
(that's how the autogenerated Visual Studio files work today). So I'm less
confident that we will retain MSVC support longer than required to safely
complete the transition to Clang.

Does that assuage your concerns?


>
> On Tue, May 1, 2018 at 7:56 PM, Anthony Jones  wrote:
> > You may already know that the Low-Level Tools team support important
> tools
> > and code infrastructure. Lately we’ve also been improving our rustc/clang
> > (LLVM) story and I’d like bring everyone up to date.
> >
> >
> > There are a lot of important and interesting things going on:
> >
> >
> > Michael Woerister and Nathan Froyd recently (mid-March) enabled Rust
> > incremental compilation for Firefox developers
> >
> > Michael is experimenting with cross language inlining (rustc/clang) using
> > LTO
> >
> > Preliminary results show compiling LLVM with clang and using LTO on rustc
> > improves stylo compilation time by around 15%
> >
> > LTO requires more work to support Firefox
> >
> > Nick Nethercote has started speeding up rustc and has already had a
> number
> > of wins
> >
> > David Major has got Windows clang builds working with green tests
> >
> > LTO is still buggy (but works well enough to run benchmarks) and PGO
> won’t
> > run on Windows
> >
> > Win32 - clang with LTO w/o PGO is marginally faster than MSVC with
> LTO/PGO
> >
> > Win64 - clang with LTO w/o PGO is ~5-10% slower than MSVC with LTO/PGO
> >
> > Windows users can build locally with clang
> >
> > Mike Hommey is tracking improvements and regressions in the Rust compiler
> > (performance regression, crash reporting and more crash reporting)
> >
> > Tom Tromey is continuing to work on first class Rust support in GDB and
> LLDB
> >
> >
> > Ultimately, I’d like to see us standardise on clang on all platforms
> because
> > it makes Rust/C++ integration better as well as making things simpler for
> > developers and build system maintainers. We’ll get more done if we make
> our
> > own lives simpler.
> >
> >
> > I have some specific requests:
> >
> >
> > Let me know if you have specific Firefox cases where Rust is slowing you
> > down
> >
> > Cross language inlining is coming - avoid duplication between Rust and
> C++
> >
> > Start doing your developer builds with clang
> >
> > Anthony
> >
> >
> >
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
> >
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Tom Ritter]

On Tue, May 29, 2018 at 8:48 PM, Jeff Gilbert  wrote:
> It would be sad to see us standardize on a clang monoculture.

I wouldn't want us to abandon msvc and gcc as well-supported
compilers; but from just one perspective (security) it would be very
advantageous to have a single open source toolchain on all platforms
so that security improvements like Control Flow Integrity and other,
much newer, techniques coming out of academia are applicable to all
platforms immediately. Currently, for Control Flow Integrity we have
separate implementations on Mac and Windows (necessitating duplicate
effort) and no good option on Linux. For academic vtable protections
I'm curious about, they exist as clang plugins and nothing else.

-tom
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Jeff Gilbert]

It would be sad to see us standardize on a clang monoculture.

On Tue, May 1, 2018 at 7:56 PM, Anthony Jones  wrote:
> You may already know that the Low-Level Tools team support important tools
> and code infrastructure. Lately we’ve also been improving our rustc/clang
> (LLVM) story and I’d like bring everyone up to date.
>
>
> There are a lot of important and interesting things going on:
>
>
> Michael Woerister and Nathan Froyd recently (mid-March) enabled Rust
> incremental compilation for Firefox developers
>
> Michael is experimenting with cross language inlining (rustc/clang) using
> LTO
>
> Preliminary results show compiling LLVM with clang and using LTO on rustc
> improves stylo compilation time by around 15%
>
> LTO requires more work to support Firefox
>
> Nick Nethercote has started speeding up rustc and has already had a number
> of wins
>
> David Major has got Windows clang builds working with green tests
>
> LTO is still buggy (but works well enough to run benchmarks) and PGO won’t
> run on Windows
>
> Win32 - clang with LTO w/o PGO is marginally faster than MSVC with LTO/PGO
>
> Win64 - clang with LTO w/o PGO is ~5-10% slower than MSVC with LTO/PGO
>
> Windows users can build locally with clang
>
> Mike Hommey is tracking improvements and regressions in the Rust compiler
> (performance regression, crash reporting and more crash reporting)
>
> Tom Tromey is continuing to work on first class Rust support in GDB and LLDB
>
>
> Ultimately, I’d like to see us standardise on clang on all platforms because
> it makes Rust/C++ integration better as well as making things simpler for
> developers and build system maintainers. We’ll get more done if we make our
> own lives simpler.
>
>
> I have some specific requests:
>
>
> Let me know if you have specific Firefox cases where Rust is slowing you
> down
>
> Cross language inlining is coming - avoid duplication between Rust and C++
>
> Start doing your developer builds with clang
>
> Anthony
>
>
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Chris Hutten-Czapski]

Recently I found myself using the mozbuild-supplied clang for compiling
Gecko on Linux (https://bugzilla.mozilla.org/show_bug.cgi?id=1451312). It
works just peachily for me, and I find myself liking their error message
decorators better than gcc's.

Since we supply a compiler during bootstrap, it seems a little odd we don't
default to using it.

:chutten


On Mon, May 14, 2018 at 4:58 PM David Major  wrote:

> We've confirmed that this issue with debug symbols comes from lld-link and
> not from clang-cl. This will likely need a fix from the LLVM side, but in
> the meantime I'd like to encourage people not to be deterred from using
> clang-cl as your compiler.
>
> On Thu, May 10, 2018 at 9:12 PM Xidorn Quan  wrote:
>
> > On Fri, May 11, 2018, at 10:35 AM, Anthony Jones wrote:
> > > I have some specific requests for you:
> > >
> > > Let me know if you have specific Firefox related cases where Rust
> is
> > > slowing you down (thanks Jeff [7])
> > > Cross language inlining is coming - avoid duplication between Rust
> > > and C++ in the name of performance
> > > Do developer builds with clang
>
> > Regarding the last item about building with clang on Windows, I'd not
> recommend people who use Visual Studio for debugging Windows build to build
> with clang at this moment.
>
> > I've tried using lld-link as linker (while continuing using cl rather
> than clang-cl) for my local Windows build, and it seems to cause problems
> when debugging with Visual Studio. Specifically, you may not be able to
> invoke debugging functions like DumpJSStack, DumpFrameTree in Immediate
> Windows, and variable value watching doesn't seem to work well either.
>
> > I've filed a bug[1] for the debugging function issue (and probably should
> file another for the watching issue as well).
>
> > [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1458109
>
>
> > - Xidorn
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[David Major]

We've confirmed that this issue with debug symbols comes from lld-link and
not from clang-cl. This will likely need a fix from the LLVM side, but in
the meantime I'd like to encourage people not to be deterred from using
clang-cl as your compiler.

On Thu, May 10, 2018 at 9:12 PM Xidorn Quan  wrote:

> On Fri, May 11, 2018, at 10:35 AM, Anthony Jones wrote:
> > I have some specific requests for you:
> >
> > Let me know if you have specific Firefox related cases where Rust is
> > slowing you down (thanks Jeff [7])
> > Cross language inlining is coming - avoid duplication between Rust
> > and C++ in the name of performance
> > Do developer builds with clang

> Regarding the last item about building with clang on Windows, I'd not
recommend people who use Visual Studio for debugging Windows build to build
with clang at this moment.

> I've tried using lld-link as linker (while continuing using cl rather
than clang-cl) for my local Windows build, and it seems to cause problems
when debugging with Visual Studio. Specifically, you may not be able to
invoke debugging functions like DumpJSStack, DumpFrameTree in Immediate
Windows, and variable value watching doesn't seem to work well either.

> I've filed a bug[1] for the debugging function issue (and probably should
file another for the watching issue as well).

> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1458109


> - Xidorn
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Gregory Szorc]

On Thu, May 10, 2018 at 5:35 PM, Anthony Jones  wrote:

> You may already know that the Low-Level Tools team support important tools
> and code infrastructure. Lately we’ve also been improving our rustc/clang
> (LLVM) story and I’d like bring everyone up to date.
>
> There are a lot of important and interesting things going on:
>
> * Michael Woerister and Nathan Froyd recently (mid-March) enabled Rust
> incremental compilation for Firefox developers
> * Michael is experimenting with cross language inlining[1]
> (rustc/clang) using LTO
> * Preliminary results show compiling LLVM with clang and using LTO
> on rustc improves stylo compilation time by around 15% [on-my-machine
> measurements]
> * LTO requires more work to support Firefox
> * Nick Nethercote has started speeding up rustc[2] and has already had
> a number of wins
> * David Major has got Windows clang builds working with green tests
> * LTO is still buggy (but works well enough to run benchmarks) and
> PGO won’t run on Windows
> * Win32 - clang with LTO w/o PGO is marginally faster than MSVC
> with LTO/PGO
> * Win64 - clang with LTO w/o PGO is ~5-10% slower than MSVC with
> LTO/PGO
> * Windows users can build locally with clang[3]
> * Mike Hommey is tracking improvements and regressions in the Rust
> compiler (performance regression[4], crash reporting[5] and more crash
> reporting[6])
> * Tom Tromey is continuing to work on first class Rust support in GDB
> and LLDB
>
> Note: this a summary of things rustc/clang stuff that is happening only.
>
> Ultimately, I’d like to see us standardise on clang on all platforms
> because it makes Rust/C++ integration better as well as making things
> simpler for developers and build system maintainers. We’ll get more done if
> we make our own lives simpler.
>
> I have some specific requests for you:
>
> Let me know if you have specific Firefox related cases where Rust is
> slowing you down (thanks Jeff [7])
> Cross language inlining is coming - avoid duplication between Rust and
> C++ in the name of performance
> Do developer builds with clang
>

This is all fantastic work!

Speaking as a build system maintainer, I would also like to see us
standardize on Clang on all platform for various reasons, especially
simplicity. Speaking as an open source advocate, I can't wait to be using
an open source toolchain on Windows. Speaking as someone who cares about
developer ergonomics, I can't wait until we reduce the number of toolchains
we support as tier 1 platforms so we all don't have to spend so much effort
making things work on N+1 platforms. Reducing the number of toolchains we
need to worry about is very much analogous to the security benefits of
reducing attack surface area.

I do have some concerns about dropping support for non-Clang toolchains and
what that will mean for various people who rely on them. But I think
"switching toolchains" is a separate discussion from "dropping toolchains"
and we can largely defer those discussions. As far as switching to Clang
for all builds we ship to users is concerned, I'm all aboard from the build
system side.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Steve Fink]

On 5/11/18 12:16 PM, Anthony Jones wrote:

On Friday, 11 May 2018 21:14:21 UTC+12, Sylvestre Ledru  wrote:

Do we have a list of blockers (or meta bug) to make the switch on Linux?

We're already using clang for ASAN builds on Linux and passing tests. We'll 
want to switch to clang on Linux pretty soon after getting it running on 
Windows, it not at the same time. Once we have cross language inlining (through 
LTO), we can save a lot of complexity by relying on it for performance.

We're making good progress on Windows but I don't have a timeline because of 
dependencies on third parties.


Note that we have at least one dependency on keeping gcc working: the 
JavaScript GC rooting hazard static analysis. The first layer uses a gcc 
plugin and is very tied to the internal gcc data structures. Rewriting 
it for clang should be possible, but is not currently scheduled and 
would likely be a multiple month project.



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Sylvestre Ledru]

Hello,

Bravo for all this work. Having a single opensource toolchain would be a great 
gain for us!

On 11/05/2018 02:35, Anthony Jones wrote:
> You may already know that the Low-Level Tools team support important tools 
> and code infrastructure. Lately we’ve also been improving our rustc/clang 
> (LLVM) story and I’d like bring everyone up to date.
>
> There are a lot of important and interesting things going on:
[...]
> Note: this a summary of things rustc/clang stuff that is happening only.
>
> Ultimately, I’d like to see us standardise on clang on all platforms because 
> it makes Rust/C++ integration better as well as making things simpler for 
> developers and build system maintainers. We’ll get more done if we make our 
> own lives simpler.
Do we have a list of blockers (or meta bug) to make the switch on Linux?

Thanks,
Sylvestre
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Update on rustc/clang goodness

[Xidorn Quan]

On Fri, May 11, 2018, at 10:35 AM, Anthony Jones wrote:
> I have some specific requests for you:
> 
> Let me know if you have specific Firefox related cases where Rust is 
> slowing you down (thanks Jeff [7])
> Cross language inlining is coming - avoid duplication between Rust 
> and C++ in the name of performance 
> Do developer builds with clang

Regarding the last item about building with clang on Windows, I'd not recommend 
people who use Visual Studio for debugging Windows build to build with clang at 
this moment.

I've tried using lld-link as linker (while continuing using cl rather than 
clang-cl) for my local Windows build, and it seems to cause problems when 
debugging with Visual Studio. Specifically, you may not be able to invoke 
debugging functions like DumpJSStack, DumpFrameTree in Immediate Windows, and 
variable value watching doesn't seem to work well either.

I've filed a bug[1] for the debugging function issue (and probably should file 
another for the watching issue as well).

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1458109


- Xidorn
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform