Re: More Rust code

[Mike Hommey]

On Tue, Aug 08, 2017 at 09:09:09AM +0900, Mike Hommey wrote:
> On Tue, Aug 08, 2017 at 07:12:13AM +0900, Mike Hommey wrote:
> > On Fri, Aug 04, 2017 at 08:45:14AM +0300, Henri Sivonen wrote:
> > > I guess I buried my questions in too long a post, so extracting them:
> > > 
> > > On Mon, Jul 31, 2017 at 1:02 PM, Henri Sivonen  
> > > wrote:
> > > > Naïvely, one would think that it should be possible to do that with
> > > > clang producing "object files" holding LLVM IR and rustc producing
> > > > "object files" holding LLVM IR and the "link" step involving mashing
> > > > those together, running LLVM optimizations again and then producing
> > > > machine code from a massive collection of mashed-together LLVM IR.
> > > ...
> > > > If we were compiling C++ using clang on all platforms, how far off
> > > > would such cross-language inlining be?
> > > >
> > > > If we could have the cross-language inlining benefit from compiling
> > > > C++ using clang on all platforms, how far off would we be from being
> > > > able to switch to clang on all platforms?
> > 
> > Here's a bunch of data why "let's switch compilers" is not necessarily
> > easy (I happen to have gathered that recently):
> > 
> > Windows MSVC (PGO) vs. clang-cl:
> > https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=581eab2aa162=try=c963d0cd6565=1=0
> > 
> > Windows MSVC (PGO) vs. clang-cl -O2 (the build system currently uses -O1 for
> > msvc builds, which flags we use for clang-cl builds):
> > https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=581eab2aa162=try=c963d0cd6565=1=0
> 
> It was pointed to me that both urls were the same. The second is
> actually:
> https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=581eab2aa162=try=232f977d9739=1=0

And now that the clang trunk bug that made it crash when building
Firefox is fixed, here's another go at MSVC(PGO) vs. clang-cl -O2:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=f8ff2c492d2e=try=868f0ef87a7d=1=0

(tl;dr: not much change from the version we were using before)

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Ehsan Akhgari]

On 08/08/2017 11:32 AM, Jeff Muizelaar wrote:

On Mon, Aug 7, 2017 at 6:12 PM, Mike Hommey  wrote:

   Note that the tp5n main_startup_fileio reflects the resulting size of
   xul.dll, which also impacts the installer size:
  32-bits   64-bits
   MSVC (PGO):   37904383  40803170
   clang-cl: 39537860  40561849
   clang-cl -O2: 41976097  43338891

FWIW, https://bugs.llvm.org//show_bug.cgi?id=26299 is the metabug for
tracking improvements to x86-32 code size.
And the equivalent bug on the Chromium side: 
https://bugs.chromium.org/p/chromium/issues/detail?id=457078

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Gregory Szorc]

On Mon, Aug 7, 2017 at 11:01 PM, Henri Sivonen  wrote:

> On Tue, Aug 8, 2017 at 1:12 AM, Mike Hommey  wrote:
> > Here's a bunch of data why "let's switch compilers" is not necessarily
> > easy (I happen to have gathered that recently):
>
> Thank you.
>
> > Newer versions of clang-cl might generate faster code, but they crash
> > during the build: https://bugs.llvm.org/show_bug.cgi?id=33997
>
> I'm guessing using a very new clang was what allowed Chrome to switch
> from MSVC to clang? (Chrome accepted a binary size increase on 32-bit
> Windows as a result of switching to clang.)
>

I also found references to performance hits in their issue tracker.
Specifically, it looked like process startup and new tab times regressed.
But I could find no numbers quantifying it.

We should view their decision as having to do more with using a unified and
open source toolchain [that they can patch, improve, and deploy in days
instead of months] instead of purely performance. It's a huge win for
Chrome's developers (target 1 toolchain instead of N) and it will allow
them to move faster since they're not beholden to any lagging one-off
toolchain. They still have to worry about packaging for Chromium, of
course. Hopefully this will result in faster distro packaging for newer
LLVM/Clang releases (not unlike how Rust is forcing the world to come to
terms with a toolchain that is released every 6 weeks).
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Jeff Muizelaar]

On Mon, Aug 7, 2017 at 6:12 PM, Mike Hommey  wrote:
>   Note that the tp5n main_startup_fileio reflects the resulting size of
>   xul.dll, which also impacts the installer size:
>  32-bits   64-bits
>   MSVC (PGO):   37904383  40803170
>   clang-cl: 39537860  40561849
>   clang-cl -O2: 41976097  43338891

FWIW, https://bugs.llvm.org//show_bug.cgi?id=26299 is the metabug for
tracking improvements to x86-32 code size.

-Jeff
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Henri Sivonen]

On Tue, Aug 8, 2017 at 1:12 AM, Mike Hommey  wrote:
> Here's a bunch of data why "let's switch compilers" is not necessarily
> easy (I happen to have gathered that recently):

Thank you.

> Newer versions of clang-cl might generate faster code, but they crash
> during the build: https://bugs.llvm.org/show_bug.cgi?id=33997

I'm guessing using a very new clang was what allowed Chrome to switch
from MSVC to clang? (Chrome accepted a binary size increase on 32-bit
Windows as a result of switching to clang.)

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Mike Hommey]

On Tue, Aug 08, 2017 at 07:12:13AM +0900, Mike Hommey wrote:
> On Fri, Aug 04, 2017 at 08:45:14AM +0300, Henri Sivonen wrote:
> > I guess I buried my questions in too long a post, so extracting them:
> > 
> > On Mon, Jul 31, 2017 at 1:02 PM, Henri Sivonen  wrote:
> > > Naïvely, one would think that it should be possible to do that with
> > > clang producing "object files" holding LLVM IR and rustc producing
> > > "object files" holding LLVM IR and the "link" step involving mashing
> > > those together, running LLVM optimizations again and then producing
> > > machine code from a massive collection of mashed-together LLVM IR.
> > ...
> > > If we were compiling C++ using clang on all platforms, how far off
> > > would such cross-language inlining be?
> > >
> > > If we could have the cross-language inlining benefit from compiling
> > > C++ using clang on all platforms, how far off would we be from being
> > > able to switch to clang on all platforms?
> 
> Here's a bunch of data why "let's switch compilers" is not necessarily
> easy (I happen to have gathered that recently):
> 
> Windows MSVC (PGO) vs. clang-cl:
> https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=581eab2aa162=try=c963d0cd6565=1=0
> 
> Windows MSVC (PGO) vs. clang-cl -O2 (the build system currently uses -O1 for
> msvc builds, which flags we use for clang-cl builds):
> https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=581eab2aa162=try=c963d0cd6565=1=0

It was pointed to me that both urls were the same. The second is
actually:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=581eab2aa162=try=232f977d9739=1=0

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Mike Hommey]

On Fri, Aug 04, 2017 at 08:45:14AM +0300, Henri Sivonen wrote:
> I guess I buried my questions in too long a post, so extracting them:
> 
> On Mon, Jul 31, 2017 at 1:02 PM, Henri Sivonen  wrote:
> > Naïvely, one would think that it should be possible to do that with
> > clang producing "object files" holding LLVM IR and rustc producing
> > "object files" holding LLVM IR and the "link" step involving mashing
> > those together, running LLVM optimizations again and then producing
> > machine code from a massive collection of mashed-together LLVM IR.
> ...
> > If we were compiling C++ using clang on all platforms, how far off
> > would such cross-language inlining be?
> >
> > If we could have the cross-language inlining benefit from compiling
> > C++ using clang on all platforms, how far off would we be from being
> > able to switch to clang on all platforms?

Here's a bunch of data why "let's switch compilers" is not necessarily
easy (I happen to have gathered that recently):

Windows MSVC (PGO) vs. clang-cl:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=581eab2aa162=try=c963d0cd6565=1=0

Windows MSVC (PGO) vs. clang-cl -O2 (the build system currently uses -O1 for
msvc builds, which flags we use for clang-cl builds):
https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=581eab2aa162=try=c963d0cd6565=1=0

Newer versions of clang-cl might generate faster code, but they crash
during the build: https://bugs.llvm.org/show_bug.cgi?id=33997

  Note that the tp5n main_startup_fileio reflects the resulting size of
  xul.dll, which also impacts the installer size:
 32-bits   64-bits
  MSVC (PGO):   37904383  40803170
  clang-cl: 39537860  40561849
  clang-cl -O2: 41976097  43338891

  64-bits clang-cl is miraculously smaller, but the zip is larger, and so
  is the on-disk xul.dll.


Linux gcc 4.9 PGO vs clang trunk (r309511):
https://treeherder.mozilla.org/perf.html#/compare?originalProject=try=3c83f65009d6=try=5d22234f2596=1=0

This might look unfair (PGO vs. not-PGO), but for some reason, trying
PGO with clang breaks the build... while invoking cargo...
https://treeherder.mozilla.org/#/jobs?repo=try=8aeb26f7b22b7c8a03602229caa79d20bdadc94d
if anyone wants to take a look.

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Henri Sivonen]

On Mon, Jul 31, 2017 at 4:53 PM, smaug  wrote:

>> And on the topic of memory management:
>>
>> DOM nodes themselves obviously have to be able to deal with multiple
>> references to them, but otherwise we have a lot of useless use of
>> refcounting attributable to the 1998/1999 mindset of making everything
>> an nsIFoo. In cases where mozilla::UniquePtr would suffice and
>> nsCOMPtr isn't truly needed considering the practical ownership
>> pattern, making the Rust objects holdable by mozilla::UniquePtr is
>> rather easy: mozilla::Decoder and mozilla::Encoder are real-world
>> examples.
>
> Reference counting is needed always if both JS and C++ can have a pointer to
> the object.

Being able to have JS point to a lot of things is part of the original
"everything is an nsIFoo" XPCOM mindset. For example, the types that
mozilla::Decoder and mozilla::Encoder replaced were not scriptable but
had XPCOM reference counting. They were always used in a manner where
mozilla::UniquePtr would have sufficed in C++ and didn't work from JS
anyway.

> And UniquePtr makes it harder to ensure some object stays alive during a
> method call, say, if a member variable is
> UniquePtr. With refcounting it is always easy to have local strong
> reference.

When UniquePtr points to a Rust object, there's the mitigation that
introducing a call from Rust to C++ involves more intentionality than adding
another C++ to C++ call, so it's easier to notice when a call to C++
that might release the C++ object that holds the UniquePtr is
introduced. For example, mozilla::Decoder and mozilla::Encoder never
call to C++ code, so it's easy to reason that such leaf code can't
accidentally delete what `self` points to.

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Henri Sivonen]

I guess I buried my questions in too long a post, so extracting them:

On Mon, Jul 31, 2017 at 1:02 PM, Henri Sivonen  wrote:
> Naïvely, one would think that it should be possible to do that with
> clang producing "object files" holding LLVM IR and rustc producing
> "object files" holding LLVM IR and the "link" step involving mashing
> those together, running LLVM optimizations again and then producing
> machine code from a massive collection of mashed-together LLVM IR.
...
> If we were compiling C++ using clang on all platforms, how far off
> would such cross-language inlining be?
>
> If we could have the cross-language inlining benefit from compiling
> C++ using clang on all platforms, how far off would we be from being
> able to switch to clang on all platforms?
...
>  2) Rust code reading from the DOM when the DOM is guaranteed not to change.
...
> I've been talking about wishing to rewrite our DOM serializers (likely
> excluding the one we use for innerHTML in the document is in the HTML
> mode) in Rust. I have been assuming that such work could reuse the
> code that Stylo of has for viewing the DOM from Rust in a read-only
> stop-the-world  fashion.
>
> I haven't actually examined how reusable that Stylo code is for
> non-Stylo purposes. Is it usable for non-Stylo purposes?

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[smaug]

On 07/31/2017 01:02 PM, Henri Sivonen wrote:

On Tue, Jul 18, 2017 at 7:01 AM, Jim Blandy  wrote:

BTW, speaking of training: Jason's and my book, "Programming Rust" will be
available on paper from O'Reilly on August 29th!


And already available on Safari Books Online (access available via
Service Now request subject to managerial approval).

On Mon, Jul 17, 2017 at 10:43 PM, Ted Mielczarek  wrote:

From my perspective Rust is our
single-biggest competitive advantage for shipping Firefox, and every
time we choose C++ over Rust we throw that away.


I agree.


but we are quickly
going to hit a point where "I don't feel like learning Rust" is not
going to cut it anymore.


Indeed.

On Tue, Jul 11, 2017 at 4:37 PM, smaug  wrote:

How is the performance when crossing Rust <-> C++ boundary? We need to make
everything faster, not slower.
If I understood emilio's explanation on IRC correctly having the performance
of an inlined (C++) function requires
handwritten rust bindings to access member variables of some C++ object.
That doesn't sound too good - hard to maintain and possibly easy to forget
to optimize.

I don't claim to understand anything about the current setup, but
has anyone written down what would be needed to have fast and easy to
maintain Rust <-> C++ boundary in
such way that also memory handling is easy to manage (aka, how to deal with
CC/GC).
I think it would be better to sort out this kind of low level issues rather
soon before we have too much
Rust code in tree, or perhaps we won't see much Rust usage before those
issues are sorted out.

(I'm looking this all from DOM point of view, where pretty much all the
objects need to be cycle collectable JS holders, but perhaps Rust would fit
better in code outside DOM)


Rust indeed is, at least at present, a better fit for code outside the
area of cycle-collectable DOM objects.

The performance issue you mention applies if the usage scenario is
that Rust code needs to get or set a lot of fields on a C++ object.
While we do have code that, if implemented in Rust, would have to do
performance-sensitive field access on C++ objects, we also have areas
for which that would not be a concern. For example, in the case of
encoding_rs, the data that crosses the FFI boundary is structurally
simple (mozilla::Span / Rust slices decomposing to pointer to an array
of primitives and a length for FFI crossing) and the amount of work
done on the Rust side is substantial compared to the frequency of
crossing the FFI boundary.

In the absence of the Stylo-like optimization of frequent
performance-sensitive access of fields of a foreign-language object,
the FFI story that one can expect involves three functions/methods per
logical method.

Either (for C++ caller and Rust callee)
 1) C++ method wrapping the C function to hide the unsafety and bad
ergonomics of raw C.
 2) C function declared in C++ and implemented in Rust.
 3) Rust method: the actual callee that does something useful.
Or (for Rust caller and C++ callee)
 1) Rust method wrapping the C function to hide the unsafety and bad
ergonomics of raw C.
 2) C function declared in Rust and implemented in C++.
 3) C++ method: the actual callee that does something useful.

So there's the real callee method, there's a small C function that
wraps that method in a C ABI-compatible way and then there is a
wrapper for the C function that provides the ergonomics that one would
expect in the calling language.

The caller-side wrapper around the C function is trivial to make
inline and as a matter of code size is likely harmless or even
strictly beneficial to make inline.

The compilers don't have visibility across the declaration definition
of the C function, since that's where the cross-language linkage
happens, so currently one needs to assume that the C function always
has the cost of an actual function call.

As for inlining the actual callee method of interest in the language
being called into the implementation of the C function, it may or may
not happen automatically and when it doesn't happen automatically,
forcing it to happen manually might be a problem in terms of code
size.

So when the callee that actually does the work we care about doesn't
get inlined into its C wrapper, one should approximate a call from
Rust to C++ or from C++ to Rust have the cost of two *non-virtual*
function calls instead of one. (It would be interesting to contrast
this to the cost of over-use of virtual calls due to nsIFoo
interfaces.)

- -

Ideally, both the caller-language-side wrapper around the C function
and the C function itself would get inlined so that the cross-language
call would on the machine code level look just like a normal (if not
also inlined!) call to a method of the callee language. For that to
happen, we'd need link-time inlining across object files produced by
different-language compilers.

Naïvely, one would think that it should be possible to do that with

Re: More Rust code

[Henri Sivonen]

On Tue, Jul 18, 2017 at 7:01 AM, Jim Blandy  wrote:
> BTW, speaking of training: Jason's and my book, "Programming Rust" will be
> available on paper from O'Reilly on August 29th!

And already available on Safari Books Online (access available via
Service Now request subject to managerial approval).

On Mon, Jul 17, 2017 at 10:43 PM, Ted Mielczarek  wrote:
> From my perspective Rust is our
> single-biggest competitive advantage for shipping Firefox, and every
> time we choose C++ over Rust we throw that away.

I agree.

> but we are quickly
> going to hit a point where "I don't feel like learning Rust" is not
> going to cut it anymore.

Indeed.

On Tue, Jul 11, 2017 at 4:37 PM, smaug  wrote:
> How is the performance when crossing Rust <-> C++ boundary? We need to make
> everything faster, not slower.
> If I understood emilio's explanation on IRC correctly having the performance
> of an inlined (C++) function requires
> handwritten rust bindings to access member variables of some C++ object.
> That doesn't sound too good - hard to maintain and possibly easy to forget
> to optimize.
>
> I don't claim to understand anything about the current setup, but
> has anyone written down what would be needed to have fast and easy to
> maintain Rust <-> C++ boundary in
> such way that also memory handling is easy to manage (aka, how to deal with
> CC/GC).
> I think it would be better to sort out this kind of low level issues rather
> soon before we have too much
> Rust code in tree, or perhaps we won't see much Rust usage before those
> issues are sorted out.
>
> (I'm looking this all from DOM point of view, where pretty much all the
> objects need to be cycle collectable JS holders, but perhaps Rust would fit
> better in code outside DOM)

Rust indeed is, at least at present, a better fit for code outside the
area of cycle-collectable DOM objects.

The performance issue you mention applies if the usage scenario is
that Rust code needs to get or set a lot of fields on a C++ object.
While we do have code that, if implemented in Rust, would have to do
performance-sensitive field access on C++ objects, we also have areas
for which that would not be a concern. For example, in the case of
encoding_rs, the data that crosses the FFI boundary is structurally
simple (mozilla::Span / Rust slices decomposing to pointer to an array
of primitives and a length for FFI crossing) and the amount of work
done on the Rust side is substantial compared to the frequency of
crossing the FFI boundary.

In the absence of the Stylo-like optimization of frequent
performance-sensitive access of fields of a foreign-language object,
the FFI story that one can expect involves three functions/methods per
logical method.

Either (for C++ caller and Rust callee)
 1) C++ method wrapping the C function to hide the unsafety and bad
ergonomics of raw C.
 2) C function declared in C++ and implemented in Rust.
 3) Rust method: the actual callee that does something useful.
Or (for Rust caller and C++ callee)
 1) Rust method wrapping the C function to hide the unsafety and bad
ergonomics of raw C.
 2) C function declared in Rust and implemented in C++.
 3) C++ method: the actual callee that does something useful.

So there's the real callee method, there's a small C function that
wraps that method in a C ABI-compatible way and then there is a
wrapper for the C function that provides the ergonomics that one would
expect in the calling language.

The caller-side wrapper around the C function is trivial to make
inline and as a matter of code size is likely harmless or even
strictly beneficial to make inline.

The compilers don't have visibility across the declaration definition
of the C function, since that's where the cross-language linkage
happens, so currently one needs to assume that the C function always
has the cost of an actual function call.

As for inlining the actual callee method of interest in the language
being called into the implementation of the C function, it may or may
not happen automatically and when it doesn't happen automatically,
forcing it to happen manually might be a problem in terms of code
size.

So when the callee that actually does the work we care about doesn't
get inlined into its C wrapper, one should approximate a call from
Rust to C++ or from C++ to Rust have the cost of two *non-virtual*
function calls instead of one. (It would be interesting to contrast
this to the cost of over-use of virtual calls due to nsIFoo
interfaces.)

- -

Ideally, both the caller-language-side wrapper around the C function
and the C function itself would get inlined so that the cross-language
call would on the machine code level look just like a normal (if not
also inlined!) call to a method of the callee language. For that to
happen, we'd need link-time inlining across object files produced by
different-language compilers.

Naïvely, one would think that it should be possible to do that with

Re: More Rust code

[David Teller]

Thanks for starting this conversation. I'd love to be able to use more
Rust in Firefox.

In SpiderMonkey, the main blocker I encounter is interaction with all
the nice utility classes we have in C++, in particular templatized ones.

Also, for the rest of Gecko, my main blocker was the lack of support for
Rust-implemented webidl in m-c, which meant that roughly 50% of the code
I would be writing would have been bug-prone adapters.

Cheers,
 David

On 10/07/17 12:29, Nicholas Nethercote wrote:
> Hi,
> 
> Firefox now has multiple Rust components, and it's on track to get a
> bunch more. See https://wiki.mozilla.org/Oxidation for details.
> 
> I think this is an excellent trend, and I've been thinking about how to
> accelerate it. Here's a provocative goal worth considering: "when
> writing a new compiled-code component, or majorly rewriting an existing
> one, Rust should be considered / preferred / mandated."
> 
> What are the obstacles? Here are some that I've heard.
> 
> - Lack of Rust expertise for both writing and reviewing code. We have
> some pockets of expertise, but these need to be expanded greatly. I've
> heard that there has been some Rust training in the Paris and Toronto
> offices. Would training in other offices (esp. MV and SF, given their
> size) be a good idea? What about remoties?
> 
> - ARM/Android is not yet a Tier-1 platform for Rust. See
> https://forge.rust-lang.org/platform-support.html and
> https://internals.rust-lang.org/t/arm-android-to-tier-1/5227 for some
> details.
> 
> - Interop with existing components can be difficult. IPDL codegen rust
> bindings could be a big help.
> 
> - Compile times are high, especially for optimized builds.
> 
> Anything else?
> 
> Nick
> 
> 
> ___
> firefox-dev mailing list
> firefox-...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
> 
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Axel Hecht]

Am 17.07.17 um 21:43 schrieb Ted Mielczarek:

Nick,

Thanks for kicking off this discussion! I felt like a broken record
talking to people about this in SF. From my perspective Rust is our
single-biggest competitive advantage for shipping Firefox, and every
time we choose C++ over Rust we throw that away. We know the costs of
shipping complicated C++ code: countless hours of engineering time spent
chasing down hard-to-reproduce crashes, exploitable security holes, and
threading issues. Organizationally we need to get to a point where every
engineer has the tools and training they need to make Rust their first
choice for writing code that ships with Firefox.

On Mon, Jul 10, 2017, at 09:15 PM, Bobby Holley wrote:

I think this is pretty uncontroversial. The high-level strategic decision
to bet on Rust has already been made, and the cost of depending on the
language is already sunk. Now that we're past that point, I haven't heard
anyone arguing why we shouldn't opt for memory safety when writing new
standalone code. If there are people out there who disagree and think
they
have the arguments/clout/allies to make the case, please speak up.



From my anecdotal experiences, I've heard two similar refrains:

1) "I haven't learned Rust well enough to feel confident choosing it for
this code."
2) "I don't want to spend time learning Rust that I could be spending
just writing the code in C++."

I believe that every developer that writes C++ at Mozilla should be
given access to enough Rust training and work hours to spend learning it
beyond the training so that we can eliminate case #1. With the Rust
training sessions at prior All-Hands and self-motivated learning, I
think we've pretty well saturated the group of early adopters. These
people are actively writing new Rust code. We need to at least get the
people that want to learn Rust but don't feel like they've had time to
that same place.


I've been at (maybe half) a rust training at an allhands, and recently 
found myself looking at writing some code in rust. The experience was 
more about understanding other people's code, and re-using parts of it. 
Given that experience, I'd like to ask for a few more things:


Readable Rust. We spent half-n-hour on 3 lines of code, and that 
shouldn't be like that. I'm not sure if that was because the code was 
written badly, or because reading rust code requires dedicated training.


Copy-n-paste Rust code. That, to my experience, doesn't work like in any 
other language we frequently use. I'm used to copy, cut out the thing 
that it did originally, and the incrementally fill in my stuff. That 
works in most languages, but in Rust, it seems to break really really bad.


Documentation improvements. I've hit quite a few documentation pieces 
that stated the existence of the thing I was looking for. My place of 
failure was rc, which seems to have gotten quite a bunch of doc updates 
in the meantime, which is good.


I guess what I'm asking for is training on how to deal with rust code 
that other people wrote, maybe more so than writing rust code from 
scratch, starting with hello-world.


Axel



For case #2, there will always be people that don't want to learn new
languages, and I'm sympathetic to their perspective. Learning Rust well
does take a large investment of time. I don't know that I would go down
the road of making Rust training mandatory (yet), but we are quickly
going to hit a point where "I don't feel like learning Rust" is not
going to cut it anymore. I would hope that by that point we will have
trained everyone well enough that case #2 no longer exists, but if not
we will have to make harder choices.

  

The tradeoffs come when the code is less standalone, and we need to weigh
the integration costs. This gets into questions like whether/how Rust
code
should integrate into the cycle collector or into JS object reflection,
which is very much a technical decision that should be made by experts. I
have a decent sense of who some of those experts might be, and would like
to find the most lightweight mechanism for them to steer this ship.


We definitely need to figure out an ergonomic solution for writing core
DOM components in Rust, but I agree that this needs a fair bit of work
to be feasible. Most of the situations I've seen recently were not that
tightly integrated into Gecko.

-Ted



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Frederik Braun]

On 18.07.2017 06:01, Jim Blandy wrote:
> BTW, speaking of training: Jason's and my book, "Programming Rust" will be
> available on paper from O'Reilly on August 29th! Steve Klabnik's book with
> No Starch Press is coming out soon as well, but I don't know the details
> there.
> 

Steve's book is the paper version of 
AFAIU.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Jim Blandy]

BTW, speaking of training: Jason's and my book, "Programming Rust" will be
available on paper from O'Reilly on August 29th! Steve Klabnik's book with
No Starch Press is coming out soon as well, but I don't know the details
there.

On Mon, Jul 17, 2017 at 12:43 PM, Ted Mielczarek  wrote:

> Nick,
>
> Thanks for kicking off this discussion! I felt like a broken record
> talking to people about this in SF. From my perspective Rust is our
> single-biggest competitive advantage for shipping Firefox, and every
> time we choose C++ over Rust we throw that away. We know the costs of
> shipping complicated C++ code: countless hours of engineering time spent
> chasing down hard-to-reproduce crashes, exploitable security holes, and
> threading issues. Organizationally we need to get to a point where every
> engineer has the tools and training they need to make Rust their first
> choice for writing code that ships with Firefox.
>
> On Mon, Jul 10, 2017, at 09:15 PM, Bobby Holley wrote:
> > I think this is pretty uncontroversial. The high-level strategic decision
> > to bet on Rust has already been made, and the cost of depending on the
> > language is already sunk. Now that we're past that point, I haven't heard
> > anyone arguing why we shouldn't opt for memory safety when writing new
> > standalone code. If there are people out there who disagree and think
> > they
> > have the arguments/clout/allies to make the case, please speak up.
>
> From my anecdotal experiences, I've heard two similar refrains:
> 1) "I haven't learned Rust well enough to feel confident choosing it for
> this code."
> 2) "I don't want to spend time learning Rust that I could be spending
> just writing the code in C++."
>
> I believe that every developer that writes C++ at Mozilla should be
> given access to enough Rust training and work hours to spend learning it
> beyond the training so that we can eliminate case #1. With the Rust
> training sessions at prior All-Hands and self-motivated learning, I
> think we've pretty well saturated the group of early adopters. These
> people are actively writing new Rust code. We need to at least get the
> people that want to learn Rust but don't feel like they've had time to
> that same place.
>
> For case #2, there will always be people that don't want to learn new
> languages, and I'm sympathetic to their perspective. Learning Rust well
> does take a large investment of time. I don't know that I would go down
> the road of making Rust training mandatory (yet), but we are quickly
> going to hit a point where "I don't feel like learning Rust" is not
> going to cut it anymore. I would hope that by that point we will have
> trained everyone well enough that case #2 no longer exists, but if not
> we will have to make harder choices.
>
>
> > The tradeoffs come when the code is less standalone, and we need to weigh
> > the integration costs. This gets into questions like whether/how Rust
> > code
> > should integrate into the cycle collector or into JS object reflection,
> > which is very much a technical decision that should be made by experts. I
> > have a decent sense of who some of those experts might be, and would like
> > to find the most lightweight mechanism for them to steer this ship.
>
> We definitely need to figure out an ergonomic solution for writing core
> DOM components in Rust, but I agree that this needs a fair bit of work
> to be feasible. Most of the situations I've seen recently were not that
> tightly integrated into Gecko.
>
> -Ted
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Ted Mielczarek]

Nick,

Thanks for kicking off this discussion! I felt like a broken record
talking to people about this in SF. From my perspective Rust is our
single-biggest competitive advantage for shipping Firefox, and every
time we choose C++ over Rust we throw that away. We know the costs of
shipping complicated C++ code: countless hours of engineering time spent
chasing down hard-to-reproduce crashes, exploitable security holes, and 
threading issues. Organizationally we need to get to a point where every
engineer has the tools and training they need to make Rust their first
choice for writing code that ships with Firefox.

On Mon, Jul 10, 2017, at 09:15 PM, Bobby Holley wrote:
> I think this is pretty uncontroversial. The high-level strategic decision
> to bet on Rust has already been made, and the cost of depending on the
> language is already sunk. Now that we're past that point, I haven't heard
> anyone arguing why we shouldn't opt for memory safety when writing new
> standalone code. If there are people out there who disagree and think
> they
> have the arguments/clout/allies to make the case, please speak up.

>From my anecdotal experiences, I've heard two similar refrains:
1) "I haven't learned Rust well enough to feel confident choosing it for
this code."
2) "I don't want to spend time learning Rust that I could be spending
just writing the code in C++."

I believe that every developer that writes C++ at Mozilla should be
given access to enough Rust training and work hours to spend learning it
beyond the training so that we can eliminate case #1. With the Rust
training sessions at prior All-Hands and self-motivated learning, I
think we've pretty well saturated the group of early adopters. These
people are actively writing new Rust code. We need to at least get the
people that want to learn Rust but don't feel like they've had time to
that same place.

For case #2, there will always be people that don't want to learn new
languages, and I'm sympathetic to their perspective. Learning Rust well
does take a large investment of time. I don't know that I would go down
the road of making Rust training mandatory (yet), but we are quickly
going to hit a point where "I don't feel like learning Rust" is not
going to cut it anymore. I would hope that by that point we will have
trained everyone well enough that case #2 no longer exists, but if not
we will have to make harder choices.

 
> The tradeoffs come when the code is less standalone, and we need to weigh
> the integration costs. This gets into questions like whether/how Rust
> code
> should integrate into the cycle collector or into JS object reflection,
> which is very much a technical decision that should be made by experts. I
> have a decent sense of who some of those experts might be, and would like
> to find the most lightweight mechanism for them to steer this ship.

We definitely need to figure out an ergonomic solution for writing core
DOM components in Rust, but I agree that this needs a fair bit of work
to be feasible. Most of the situations I've seen recently were not that
tightly integrated into Gecko.

-Ted
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Ted Mielczarek]

On Wed, Jul 12, 2017, at 07:41 PM, Mike Hommey wrote:
> On Wed, Jul 12, 2017 at 04:06:39PM -0700, Eric Rahm wrote:
> > Interesting points.
> > 
> >- *using breakpad* - was the problem that creating wrappers to access
> >the c/c++ code was too tedious? Could bindgen help with that, if not it
> >would be interesting gather some details about why it wouldn't work and
> >file bugs against it.
> >- *pingsender* - was something like https://hyper.rs/ not around when
> >you were working on it or is this a case of finding the things you want 
> > can
> >be difficult in rust-land? Either way it might be a good idea for us to 
> > put
> >together a list of "sanctioned" crates for various tasks.
> 
> Note that pingsender is a small self-contained binary. I'm afraid writing
> in
> in rust would make it very much larger.

While this is probably true, it's also a best-case scenario for writing
a component in Rust: it doesn't have to call into *any* Gecko C++ code.
We also have zero visibility into failures in helper programs like this,
so the stability gains we get from using Rust instead of C++ are
outsized in this case.

-Ted
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Gabriele Svelto]

On 13/07/2017 01:06, Eric Rahm wrote:
>   * *using breakpad* - was the problem that creating wrappers to access
> the c/c++ code was too tedious? Could bindgen help with that, if not
> it would be interesting gather some details about why it wouldn't
> work and file bugs against it.

There were multiple reasons: I had no experience in writing Rust
bindings and we wanted to have it ready quickly; additionally we're in
the process of rewriting this part of Breakpad functionality in Rust so
we decided that it would all become Rust at a later point and writing
bindings wouldn't have been a good use of the time.

>   * *pingsender* - was something like https://hyper.rs/ not around when
> you were working on it or is this a case of finding the things you
> want can be difficult in rust-land? Either way it might be a good
> idea for us to put together a list of "sanctioned" crates for
> various tasks.

hyper.rs was already available but it's a big dependency and I didn't
feel like pulling it into mozilla-central just for the pingsender. I
believe we'll have HTTP functionality available from Rust code at some
point in the future so when it's ready I might as well port the
pingsender to it. As Mike said though the program is very small and
self-contained at this point so it would be nice to keep it that way
even after a Rust rewrite if possible.

 Gabriele



signature.asc
Description: OpenPGP digital signature
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Nicholas Nethercote]

On Tue, Jul 11, 2017 at 11:03 AM, Nicholas Nethercote <
n.netherc...@gmail.com> wrote:

>
>
>> The first thing comes to my mind is crash reports. It currently doesn't
>> always include useful panic message from Rust, see for example [1] and [2].
>>
>
> I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1379857 for this.
> It's blocking https://bugzilla.mozilla.org/show_bug.cgi?id=1348896, which
> is the Rust crash tracking bug.
>

jryans just landed a fix for bug 1379857. Turns out that Rust panic
messages weren't being captured by crash reports in content processes.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Mike Hommey]

On Wed, Jul 12, 2017 at 04:06:39PM -0700, Eric Rahm wrote:
> Interesting points.
> 
>- *using breakpad* - was the problem that creating wrappers to access
>the c/c++ code was too tedious? Could bindgen help with that, if not it
>would be interesting gather some details about why it wouldn't work and
>file bugs against it.
>- *pingsender* - was something like https://hyper.rs/ not around when
>you were working on it or is this a case of finding the things you want can
>be difficult in rust-land? Either way it might be a good idea for us to put
>together a list of "sanctioned" crates for various tasks.

Note that pingsender is a small self-contained binary. I'm afraid writing in
in rust would make it very much larger.

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Selena Deckelmann]

[moving dev-platform and firefox-dev to Bcc]

What I've heard in this thread is that we have a few blockers, some pain
points and bugs to file regarding more Rust integration with Firefox.

Nick -- please ensure those bugs get filed, and a meta-bug entitled "Make
the developer experience for Firefox + Rust great". There are some issues
that Stylo needs resolved before we ship, and those bugs should probably
block a Stylo tracking bug. Once we have that in place, we can work
together to prioritize them. Whatever is blocking Stylo would be high
priority Rust/tooling work. Also, the Stylo team is considering what
performance related work will be the highest priority.

We also have significant technical problems to solve regarding integration
between C++/Rust/JS components.

Bobby raised the issue of our decision making process around where to focus
efforts around Rust integration. Ehsan is on vacation this week, so let’s
pause this discussion and have a meeting next week between the module peers
and those who have been working on the Stylo integration, because they have
to most relevant experience. I’ll put that together and report out.

-selena
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Enrico Weigelt, metux IT consult]

On 10.07.2017 12:29, Nicholas Nethercote wrote:

Hi folks,

Firefox now has multiple Rust components, and it's on track to get a 
bunch more. See https://wiki.mozilla.org/Oxidation for details.


I wonder which of the pressing problems (eg. massing resource wasting,
bad performance, horribly complicated build systems, hard to maintain
source tree, too much in one application, etc), are actually improved
by yet another language. No idea, which fancy features of rust you're
so keen on, but most of what I've seen so far, has already been done
in ADA (hmm, does rust have domain types and compile-time constraints?)

By the way, just in case nobody noticed: recent Debian stable doesn't
have rustc (for good reasons, as it's still unstable), so pretty much
no chance of recent moz in Debian world (I'd guess the same w/ other
stable distros) - limited to bleeding-edge distros for the next years.
Rustc is so alpha, that it doesn't even compile (eg. wrong guesses
on the target architecture, neeed bleeding-edge cmake, ...)

I'd rather concentrate on consequent cleanups, starting with dropping
bundled 3rdparty libraries, dropping various misfeatures, esoteric
build systems, move audio/video stuff to separate tools that are
really made for that (eg. ffmpeg or gst - yes, that gives you hw
acceleration for free), move out mail protocol handling to separate
service (mailfs is your friend) - same for address books, dictionaries,
bookmarks finally an lightweight interface
to run extensions as separate programs.

When I look back the past decade, I don't recall any major improvement
(okay, don't remember whether session management did exist back then),
but it just got slower and slower, even w/ magnitudes faster machines
since back then.

Will moving to rust improve anything of these areas ?

- Lack of Rust expertise for both writing and reviewing code. We have 
some pockets of expertise, but these need to be expanded greatly.


Wouldn't that be a yet another big argument for keeping rust stuff
optional, until that problem is settled (and rust is matured enough
so become an option for production systems ?)

- ARM/Android is not yet a Tier-1 platform for Rust. See 
https://forge.rust-lang.org/platform-support.html and 
https://internals.rust-lang.org/t/arm-android-to-tier-1/5227 for some 
details.


So, is android also dropped out of the list of supported platforms,
just like all recent stable gnu/linux distros already are ?


- Compile times are high, especially for optimized builds.


Compile times for C++ are already very high (c++ in general is very
expensive to compile, and also very complicated to write good code).
How much worse is that for rust ?


--mtx
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Kyle Machulis]

So, I've been kinda working toward the "more rust in gecko" goal, in my own
space that just happened to be a good sandbox for it.

Last year I was brought over to help on WebAuthn, which needs access to USB
HID for things like Yubikeys. I decided to write the lowest level cross
platform parts of this in Rust, because it would give us safety in a place
we desperately need it (the syscall/external bindings level). Tim Taubert
and JC Jones are now leading this, and we're starting to look at landing
the rust parts of this project soon. I had originally planned on taking
this strategy for WebMIDI, but that project has unfortunately stalled due
to priorities.

My goal is doing this is being able to bring in rust from the bottom-up, on
a part of the system that won't be part of any tight loops, just so we can
start to get an idea of what this will look like for some (not all)
systems. Right now, for webauthn, we go IPDL -> C++ -> C Bindings -> Rust.
Once we have this solid, the plan is to start chipping our way up the chain
to the IPDL boundary, though there are definitely obstacles to that, like
our calls to NSS.

Note that this project is certainly not reflective of the wide scope of
bringing rust everywhere in the browser:

- This specifically touches system components that are in C, so we don't
have to deal with C++ binding issues yet.
- It's not a particularly performance sensitive API, since human activity
is in the loop
- It doesn't really touch many other parts of the system at the moment.
This could change depending on other factors in the project, movement in
the spec, etc.

WebAuthn isn't the only sandbox we could do this in. There are other
portions, related to hardware or the HAL, either already written or still
waiting to be implemented (gamepad, webmidi, etc), that we can use to try
out some simple ideas while work is happening on other parts. There's also
been talking of seeing if we could possibly share code with Servo this way,
which is something I've been planning on trying via the Gamepad API.

Once this project was over I was planning on talking to the rest of the
team about doing a writeup, and would still love to see a blog post happen
when it's all done and shipped, but it seems like making it part of this
conversation would also be helpful.

qDot


On Wed, Jul 12, 2017 at 4:06 PM, Eric Rahm  wrote:

> Interesting points.
>
>- *using breakpad* - was the problem that creating wrappers to access
>the c/c++ code was too tedious? Could bindgen help with that, if not it
>would be interesting gather some details about why it wouldn't work and
>file bugs against it.
>- *pingsender* - was something like https://hyper.rs/ not around when
>you were working on it or is this a case of finding the things you want
> can
>be difficult in rust-land? Either way it might be a good idea for us to
> put
>together a list of "sanctioned" crates for various tasks.
>
> -e
>
> On Wed, Jul 12, 2017 at 1:36 AM, Gabriele Svelto 
> wrote:
>
> > On 10/07/2017 12:29, Nicholas Nethercote wrote:
> > > Hi,
> > > What are the obstacles? Here are some that I've heard.
> > > [...]
> > > Anything else?
> >
> > In the past year I wrote two tools (minidump-analyzer and pingsender)
> > that ship with Firefox but are separate executables so both were good
> > candidates for being written in Rust in the first place. I implemented
> > both in C++ because of different issues:
> >
> > - The minidump-analyzer relies on Google Breakpad machinery that is
> > currently C++. It's being rewritten in Rust but it's not there yet and
> > since we needed it to quickly get more crash information going the Rust
> > route would have simply taken too long. It is a good candidate for being
> > rewritten in Rust in the coming 12 months though.
> >
> > - The pingsender talks to our telemetry infrastructure so needs
> > HTTP/HTTPS functionality but without linking with libxul. In this case
> > if I had a crate that offered HTTP functionality I would have gone
> > straight to Rust but alas it wasn't there. So instead it's a blob of
> > platform-specific C++ that dlopen()s libcurl on Mac and Linux and relies
> > on WININET on Windows.
> >
> >  Gabriele
> >
> >
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
> >
> >
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Eric Rahm]

Interesting points.

   - *using breakpad* - was the problem that creating wrappers to access
   the c/c++ code was too tedious? Could bindgen help with that, if not it
   would be interesting gather some details about why it wouldn't work and
   file bugs against it.
   - *pingsender* - was something like https://hyper.rs/ not around when
   you were working on it or is this a case of finding the things you want can
   be difficult in rust-land? Either way it might be a good idea for us to put
   together a list of "sanctioned" crates for various tasks.

-e

On Wed, Jul 12, 2017 at 1:36 AM, Gabriele Svelto 
wrote:

> On 10/07/2017 12:29, Nicholas Nethercote wrote:
> > Hi,
> > What are the obstacles? Here are some that I've heard.
> > [...]
> > Anything else?
>
> In the past year I wrote two tools (minidump-analyzer and pingsender)
> that ship with Firefox but are separate executables so both were good
> candidates for being written in Rust in the first place. I implemented
> both in C++ because of different issues:
>
> - The minidump-analyzer relies on Google Breakpad machinery that is
> currently C++. It's being rewritten in Rust but it's not there yet and
> since we needed it to quickly get more crash information going the Rust
> route would have simply taken too long. It is a good candidate for being
> rewritten in Rust in the coming 12 months though.
>
> - The pingsender talks to our telemetry infrastructure so needs
> HTTP/HTTPS functionality but without linking with libxul. In this case
> if I had a crate that offered HTTP functionality I would have gone
> straight to Rust but alas it wasn't there. So instead it's a blob of
> platform-specific C++ that dlopen()s libcurl on Mac and Linux and relies
> on WININET on Windows.
>
>  Gabriele
>
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Gabriele Svelto]

On 10/07/2017 12:29, Nicholas Nethercote wrote:
> Hi,
> What are the obstacles? Here are some that I've heard.
> [...] 
> Anything else?

In the past year I wrote two tools (minidump-analyzer and pingsender)
that ship with Firefox but are separate executables so both were good
candidates for being written in Rust in the first place. I implemented
both in C++ because of different issues:

- The minidump-analyzer relies on Google Breakpad machinery that is
currently C++. It's being rewritten in Rust but it's not there yet and
since we needed it to quickly get more crash information going the Rust
route would have simply taken too long. It is a good candidate for being
rewritten in Rust in the coming 12 months though.

- The pingsender talks to our telemetry infrastructure so needs
HTTP/HTTPS functionality but without linking with libxul. In this case
if I had a crate that offered HTTP functionality I would have gone
straight to Rust but alas it wasn't there. So instead it's a blob of
platform-specific C++ that dlopen()s libcurl on Mac and Linux and relies
on WININET on Windows.

 Gabriele



signature.asc
Description: OpenPGP digital signature
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Mike Hommey]

On Wed, Jul 12, 2017 at 08:05:05AM +0900, Brian Birtles wrote:
> On Tue, Jul 11, 2017 at 11:34 PM, Jim Mathies  wrote:
> 
> > What's the debugging situation look like for Windows developers? I've heard
> > it's pretty painful. Can we step through rust code using common tools
> > (WinDBG/Visual Studio)?
> >
> 
> You can set breakpoints and step through rust code using Visual Studio.
> Sometimes you can make sense of values in the Locals/Auto/Watch panels too.
> (Perhaps someone has worked out how to write autoexp.dat-like debug
> visualizers for rust?) Then again, unoptimized rust code is very slow so
> you'll probably find you're building with optimizations on making most of
> those values useless.
> 
> The biggest problem I find is that even with RUST_BACKTRACE=1 I rarely get
> call stacks when rust code panics. I normally have to rebuild without the
> panic="abort" configuration[1] and then attach a debugger in order to see
> the failing call stack. I suspect this only applies to rust code called
> from C++ (and not, e.g., when debugging Servo), and at least once I have
> actually seen a call stack from a panic dumped to the console, so I'm not
> sure what's going on there.

This sounds like bug 1373881.

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Brian Birtles]

On Tue, Jul 11, 2017 at 11:34 PM, Jim Mathies  wrote:

> What's the debugging situation look like for Windows developers? I've heard
> it's pretty painful. Can we step through rust code using common tools
> (WinDBG/Visual Studio)?
>

You can set breakpoints and step through rust code using Visual Studio.
Sometimes you can make sense of values in the Locals/Auto/Watch panels too.
(Perhaps someone has worked out how to write autoexp.dat-like debug
visualizers for rust?) Then again, unoptimized rust code is very slow so
you'll probably find you're building with optimizations on making most of
those values useless.

The biggest problem I find is that even with RUST_BACKTRACE=1 I rarely get
call stacks when rust code panics. I normally have to rebuild without the
panic="abort" configuration[1] and then attach a debugger in order to see
the failing call stack. I suspect this only applies to rust code called
from C++ (and not, e.g., when debugging Servo), and at least once I have
actually seen a call stack from a panic dumped to the console, so I'm not
sure what's going on there.

Then there's the incredibly long build/link times.

[1] https://pastebin.mozilla.org/9026883
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Nicolas B. Pierron]

On 07/11/2017 03:46 PM, Nicolas B. Pierron wrote:
(Answering privately until I get more manager intent to get this project as 
part of any long term roadmap)


Or not so privately after all … :(

--
Nicolas B. Pierron
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

RE: More Rust code

[Jim Mathies]

What's the debugging situation look like for Windows developers? I've heard
it's pretty painful. Can we step through rust code using common tools
(WinDBG/Visual Studio)?

Jim

-Original Message-
From: dev-platform
[mailto:dev-platform-bounces+jmathies=mozilla@lists.mozilla.org] On
Behalf Of Nicholas Nethercote
Sent: Monday, July 10, 2017 5:30 AM
To: dev-platform ; firefox-dev

Subject: More Rust code

Hi,

Firefox now has multiple Rust components, and it's on track to get a bunch
more. See https://wiki.mozilla.org/Oxidation for details.

I think this is an excellent trend, and I've been thinking about how to
accelerate it. Here's a provocative goal worth considering: "when writing a
new compiled-code component, or majorly rewriting an existing one, Rust
should be considered / preferred / mandated."

What are the obstacles? Here are some that I've heard.

- Lack of Rust expertise for both writing and reviewing code. We have some
pockets of expertise, but these need to be expanded greatly. I've heard
that there has been some Rust training in the Paris and Toronto offices.
Would training in other offices (esp. MV and SF, given their size) be a
good idea? What about remoties?

- ARM/Android is not yet a Tier-1 platform for Rust. See
https://forge.rust-lang.org/platform-support.html and
https://internals.rust-lang.org/t/arm-android-to-tier-1/5227 for some
details.

- Interop with existing components can be difficult. IPDL codegen rust
bindings could be a big help.

- Compile times are high, especially for optimized builds.

Anything else?

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[smaug]

On 07/11/2017 04:27 PM, Ben Kelly wrote:

On Tue, Jul 11, 2017 at 4:57 AM, Nicholas Nethercote 
wrote:


If I were the owner of that module I would consider implementing a policy

something like the following:

"When a person writes a new compiled-code component, or majorly rewrites
an existing one, they should strongly consider writing it in Rust

instead

of C or C++. Such a decision will depend on the nature of the component.
Components that are relatively standalone and have simple interfaces to
other components are good candidates to be written in Rust, as are
components such as parsers that process untrusted input."



I think this is pretty uncontroversial. The high-level strategic decision
to bet on Rust has already been made, and the cost of depending on the
language is already sunk. Now that we're past that point, I haven't heard
anyone arguing why we shouldn't opt for memory safety when writing new
standalone code. If there are people out there who disagree and think

they

have the arguments/clout/allies to make the case, please speak up.



As Gibson said: "The future is already here — it's just not very evenly
distributed."

The paragraph you wrote is obvious to you, but I suspect it's not obvious
to everyone -- Mozilla has a lot of contributors. There may still be some
Firefox developers who think that Rust something that other people do, and
that isn't relevant to them or their component(s). There may be some
Firefox developers who are interested in Rust, but feel intimidated or
uncertain about using it. There may be some developers who are keen to use
Rust, but haven't realized that we are past the experimental stage.



(Picking a somewhat random response to reply here...)

It would be really nice to have IPDL codegen support for rust.  I
considered using rust for my current task, but decided not to since I
would've had to build even more boilerplate to work with IPC c++ actors.  I
would've ended up with more glue code than actual functional code.

Another advantage of building rust IPDL codegen targets would be that we
could build service oriented code in the parent process in rust.  This
would be an incremental improvement that could offer additional safety in
the parent process without having to tackle webidle, cycle collection,
etc.  In particular, PBackground parent actors already cannot interface
with xpcom since they are OMT and would be a good candidate here.


OMT doesn't mean no XPCOM. You're thinking xpconnect.




Anyway, I think fixing these kinds of integration pain points would
accelerate our ability to use rust in gecko.  I would hesitate to make any
kind of mandates until these issues are addressed.

Thanks.

Ben



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Alexis Beingessner]

I'm currently trying to improve the C++ <-> Rust FFI story a bit. I just
posted a draft proposal to add a mode to rustc that has it output the ABI
details of all public types:
https://internals.rust-lang.org/t/stabilizing-a-machine-readable-zprint-type-sizes/5525

This would theoretically reduce our maintenance/conversion burden at the
FFI boundary.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[smaug]

On 07/10/2017 01:29 PM, Nicholas Nethercote wrote:

Hi,

Firefox now has multiple Rust components, and it's on track to get a bunch
more. See https://wiki.mozilla.org/Oxidation for details.

I think this is an excellent trend, and I've been thinking about how to
accelerate it. Here's a provocative goal worth considering: "when writing a
new compiled-code component, or majorly rewriting an existing one, Rust
should be considered / preferred / mandated."

What are the obstacles? Here are some that I've heard.

- Lack of Rust expertise for both writing and reviewing code. We have some
pockets of expertise, but these need to be expanded greatly. I've heard
that there has been some Rust training in the Paris and Toronto offices.
Would training in other offices (esp. MV and SF, given their size) be a
good idea? What about remoties?

- ARM/Android is not yet a Tier-1 platform for Rust. See
https://forge.rust-lang.org/platform-support.html and
https://internals.rust-lang.org/t/arm-android-to-tier-1/5227 for some
details.

- Interop with existing components can be difficult. IPDL codegen rust
bindings could be a big help.

- Compile times are high, especially for optimized builds.

Anything else?



How is the performance when crossing Rust <-> C++ boundary? We need to make 
everything faster, not slower.
If I understood emilio's explanation on IRC correctly having the performance of 
an inlined (C++) function requires
handwritten rust bindings to access member variables of some C++ object.
That doesn't sound too good - hard to maintain and possibly easy to forget to 
optimize.

I don't claim to understand anything about the current setup, but
has anyone written down what would be needed to have fast and easy to maintain Rust 
<-> C++ boundary in
such way that also memory handling is easy to manage (aka, how to deal with 
CC/GC).
I think it would be better to sort out this kind of low level issues rather 
soon before we have too much
Rust code in tree, or perhaps we won't see much Rust usage before those issues 
are sorted out.

(I'm looking this all from DOM point of view, where pretty much all the objects need to be cycle collectable JS holders, but perhaps Rust would fit 
better in code outside DOM)

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Ben Kelly]

On Tue, Jul 11, 2017 at 4:57 AM, Nicholas Nethercote  wrote:

> On Tue, Jul 11, 2017 at 11:15 AM, Bobby Holley 
> wrote:
>
> > If I were the owner of that module I would consider implementing a policy
> >> something like the following:
> >>
> >> "When a person writes a new compiled-code component, or majorly rewrites
> >> an existing one, they should strongly consider writing it in Rust
> instead
> >> of C or C++. Such a decision will depend on the nature of the component.
> >> Components that are relatively standalone and have simple interfaces to
> >> other components are good candidates to be written in Rust, as are
> >> components such as parsers that process untrusted input."
> >>
> >
> > I think this is pretty uncontroversial. The high-level strategic decision
> > to bet on Rust has already been made, and the cost of depending on the
> > language is already sunk. Now that we're past that point, I haven't heard
> > anyone arguing why we shouldn't opt for memory safety when writing new
> > standalone code. If there are people out there who disagree and think
> they
> > have the arguments/clout/allies to make the case, please speak up.
> >
>
> As Gibson said: "The future is already here — it's just not very evenly
> distributed."
>
> The paragraph you wrote is obvious to you, but I suspect it's not obvious
> to everyone -- Mozilla has a lot of contributors. There may still be some
> Firefox developers who think that Rust something that other people do, and
> that isn't relevant to them or their component(s). There may be some
> Firefox developers who are interested in Rust, but feel intimidated or
> uncertain about using it. There may be some developers who are keen to use
> Rust, but haven't realized that we are past the experimental stage.
>

(Picking a somewhat random response to reply here...)

It would be really nice to have IPDL codegen support for rust.  I
considered using rust for my current task, but decided not to since I
would've had to build even more boilerplate to work with IPC c++ actors.  I
would've ended up with more glue code than actual functional code.

Another advantage of building rust IPDL codegen targets would be that we
could build service oriented code in the parent process in rust.  This
would be an incremental improvement that could offer additional safety in
the parent process without having to tackle webidle, cycle collection,
etc.  In particular, PBackground parent actors already cannot interface
with xpcom since they are OMT and would be a good candidate here.

Anyway, I think fixing these kinds of integration pain points would
accelerate our ability to use rust in gecko.  I would hesitate to make any
kind of mandates until these issues are addressed.

Thanks.

Ben
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Joshua Cranmer ]

On 7/10/17 5:29 AM, Nicholas Nethercote wrote:

- Interop with existing components can be difficult. IPDL codegen rust
bindings could be a big help.


Rust's C++ interop story is absolutely atrocious. The FFI basically runs 
on C ABI, even though Rust and C++ have some similar concepts that could 
be exposed more cleanly (e.g., this parameters, even mapping  or 
&[T] to appropriate semantics) without forcing such a step. Bindgen 
works a little, but really only for calling C++ from Rust, and only if 
the C++ code is simple enough--if the code includes headers of 
complexity doom (hi, std::string!), it really ends up being a game of 
"how can I force bindgen to ignore enough that I get access to what I need."


XPIDL and WebIDL express only a subset of functionality, so they're 
theoretically easier to support than "generic C++17." However, XPIDL is 
profoundly unnatural in C++ code (the array syntax is horrendous, 
particularly if you want to start passing string arrays), as well as 
being limited in some of its vocabulary. WebIDL is more generic, but the 
bindings produces source code, which means that the ABI isn't exactly 
standardized for easy cross-language calling.


--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Gabor Krizsanits]

On Mon, Jul 10, 2017 at 7:51 PM, Kris Maglione 
wrote:

> Combined with the fact that I would have needed to find and dig through
> various scattered mailing list posts and wiki pages, and then pester a
> bunch of people by email or IRC just to get started, I've always given up
> the idea pretty quickly.
>
>
This is by far the biggest obstacle for me. I guess the right approach here
is to take the time to walk someone through from the very beginning and
document the areas where it was not trivial how to progress (talking about
gecko development in Rust not general Rust). And iterate that a few times.
The lack of experience in the area and the sub-optimal tooling will result
a huge overhead in developer time for anyone new in this area especially
for remote contributors. Trying to minimize that overhead is important.
Convincing managers to encourage developers to pay that overhead is also a
requirement (tight deadlines will not help there).

I have been flooded with work for a while, and it has been difficult to
find more time for improving my Rust skills in general. Encouraging people
to pick up Rust related goals and make it a priority to learn more Rust
would be also important.

Gabor
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Nicholas Nethercote]

On Tue, Jul 11, 2017 at 11:15 AM, Bobby Holley 
wrote:

> If I were the owner of that module I would consider implementing a policy
>> something like the following:
>>
>> "When a person writes a new compiled-code component, or majorly rewrites
>> an existing one, they should strongly consider writing it in Rust instead
>> of C or C++. Such a decision will depend on the nature of the component.
>> Components that are relatively standalone and have simple interfaces to
>> other components are good candidates to be written in Rust, as are
>> components such as parsers that process untrusted input."
>>
>
> I think this is pretty uncontroversial. The high-level strategic decision
> to bet on Rust has already been made, and the cost of depending on the
> language is already sunk. Now that we're past that point, I haven't heard
> anyone arguing why we shouldn't opt for memory safety when writing new
> standalone code. If there are people out there who disagree and think they
> have the arguments/clout/allies to make the case, please speak up.
>

As Gibson said: "The future is already here — it's just not very evenly
distributed."

The paragraph you wrote is obvious to you, but I suspect it's not obvious
to everyone -- Mozilla has a lot of contributors. There may still be some
Firefox developers who think that Rust something that other people do, and
that isn't relevant to them or their component(s). There may be some
Firefox developers who are interested in Rust, but feel intimidated or
uncertain about using it. There may be some developers who are keen to use
Rust, but haven't realized that we are past the experimental stage.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Julien Cristau]

On Tue, Jul 11, 2017 at 3:04 AM, Chris Peterson 
wrote:

> On 7/10/17 4:48 PM, Xidorn Quan wrote:
>
>> The first thing comes to my mind is crash reports. It currently doesn't
>> always include useful panic message from Rust, see for example [1] and [2].
>> Also for Stylo, we generate lots of code (including using bindgen and mako
>> template system, bindgen is usually fine, but the code generated from
>> template can contain lots of code logic), and when the crash happens inside
>> generated code, it is pretty hard to understand what's going wrong, because
>> there is no useful source link, see for example [3].
>> There are also issues from Rust side. I've always been using an optimized
>> debug build locally (because that runs faster), and sometimes use Visual
>> Studio to investigate issues. C++ code works fine with this configuration,
>> but in Rust code, I cannot see any variable information. Stack backtrace
>> seems to work fine to me, though.
>> [1]https://crash-stats.mozilla.com/report/index/2abff06f-
>> d969-4ba5-845b-a98410170708[2]https://crash-stats.mozilla.
>> com/report/index/03718a9c-9d98-4832-b8a6-026220170706[3]
>> https://crash-stats.mozilla.com/report/index/6b7d1d78-
>> 8418-47ef-bee9-f49c20170710
>>
>
> Looking at those crash reports' signatures, we should probably add
> `core::option::expect_failed` and `core::str::slice_error_fail` to
> Socorro's list of function names to ignore [1]. Should Socorro ignore all
> Rust core::* or std::* function names when searching the backtrace for a
> useful signature?
>

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1379089 last week for
core::option::expect_failed.

Cheers,
Julien
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Bobby Holley]

On Mon, Jul 10, 2017 at 5:39 PM, Nicholas Nethercote  wrote:

> On Tue, Jul 11, 2017 at 2:33 AM, Bobby Holley 
> wrote:
>
>>
>> In other words, I think we should encourage people to consider Rust for
>> new code, but also make sure that we have a thoughtful and informed
>> discussion about whether a use-case makes sense and how best to support it.
>>
>
> Definitely! I deliberately described my "Rust should be considered /
> preferred / mandated" goal as "provocative" :)  We should think big, and
> then put engineering effort in where it makes sense and will have strong
> benefits.
>
>
>> Right now we have a module that's intended to govern decisions on these
>> issues [1]. However, the peer list for that module seems oriented around
>> C++ and build system expertise, and is sparse on people who have been
>> deeply involved in the Rust integration efforts over the past year.
>>
>> So we could expand that group with expertise to tackle Rust integration
>> issues. Or it could charter a committee of such experts to do so. Or some
>> combination. Either way, I think we want some amount of steering here, and
>> I can think of a handful of names whose input is likely required to get it
>> right.
>>
>
> That module's name is "C++/Rust usage, tools, and style", and its
> description is "Aspects of C++ use such as language feature usage, standard
> library versions/usage, compiler/toolchain versions, formatting and naming
> style, and aspects of Rust use as needs arise"
>
> To me that feels orthogonal to the notion of encouraging the adoption of
> more Rust components in Firefox. That kind of encouragement feels more
> project-wide, and so IMO would fall under the "mozilla-toplevel" component.
> IIRC Brendan resigned ownership of that module a while back but he's still
> listed as the owner at https://wiki.mozilla.org/Modul
> es/All#mozilla-toplevel. So that leaves things in an uncertain state.
>
> If I were the owner of that module I would consider implementing a policy
> something like the following:
>
> "When a person writes a new compiled-code component, or majorly rewrites
> an existing one, they should strongly consider writing it in Rust instead
> of C or C++. Such a decision will depend on the nature of the component.
> Components that are relatively standalone and have simple interfaces to
> other components are good candidates to be written in Rust, as are
> components such as parsers that process untrusted input."
>

I think this is pretty uncontroversial. The high-level strategic decision
to bet on Rust has already been made, and the cost of depending on the
language is already sunk. Now that we're past that point, I haven't heard
anyone arguing why we shouldn't opt for memory safety when writing new
standalone code. If there are people out there who disagree and think they
have the arguments/clout/allies to make the case, please speak up.

The tradeoffs come when the code is less standalone, and we need to weigh
the integration costs. This gets into questions like whether/how Rust code
should integrate into the cycle collector or into JS object reflection,
which is very much a technical decision that should be made by experts. I
have a decent sense of who some of those experts might be, and would like
to find the most lightweight mechanism for them to steer this ship.


>
> It could do with some tweaking and feedback, but in principle I don't see
> why such a policy couldn't be implemented right now. I don't know where
> this policy would be written down, though!
>
> Nick
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Chris Peterson]

On 7/10/17 4:48 PM, Xidorn Quan wrote:

The first thing comes to my mind is crash reports. It currently doesn't always 
include useful panic message from Rust, see for example [1] and [2]. Also for 
Stylo, we generate lots of code (including using bindgen and mako template 
system, bindgen is usually fine, but the code generated from template can 
contain lots of code logic), and when the crash happens inside generated code, 
it is pretty hard to understand what's going wrong, because there is no useful 
source link, see for example [3].
There are also issues from Rust side. I've always been using an optimized debug 
build locally (because that runs faster), and sometimes use Visual Studio to 
investigate issues. C++ code works fine with this configuration, but in Rust 
code, I cannot see any variable information. Stack backtrace seems to work fine 
to me, though.
[1]https://crash-stats.mozilla.com/report/index/2abff06f-d969-4ba5-845b-a98410170708[2]https://crash-stats.mozilla.com/report/index/03718a9c-9d98-4832-b8a6-026220170706[3]https://crash-stats.mozilla.com/report/index/6b7d1d78-8418-47ef-bee9-f49c20170710


Looking at those crash reports' signatures, we should probably add 
`core::option::expect_failed` and `core::str::slice_error_fail` to 
Socorro's list of function names to ignore [1]. Should Socorro ignore 
all Rust core::* or std::* function names when searching the backtrace 
for a useful signature?


[1] 
https://github.com/mozilla-services/socorro/tree/master/socorro/siglists#signatures-utilities-lists

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Nicholas Nethercote]

On Tue, Jul 11, 2017 at 9:48 AM, Xidorn Quan  wrote:

> Tooling around debug info in Rust code still isn't great.
>

The good news is that tooling issues are generally among the easier ones to
solve. Even the existing Rust code that is present in Firefox will require
at least some of these things to be fixed.


> The first thing comes to my mind is crash reports. It currently doesn't
> always include useful panic message from Rust, see for example [1] and [2].
>

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1379857 for this. It's
blocking https://bugzilla.mozilla.org/show_bug.cgi?id=1348896, which is the
Rust crash tracking bug.

For that, I just did an experiment. In my local repo, when I remove all
> linking-related files from objdir/toolkit/library (to do a fresh linking),
> it takes ~1.7min for linking.
>
> When I touch one cpp file (in my experiment, layout/style/nsCSSAnonBoxes.cpp
> which is a reasonable small cpp file I happen to open), it takes 6s to
> link, and size of xul.ilk (Incremental Linker File) increases by 856KB.
>
> When I touch one rs file (in my experiment, servo/components/gecko_binding
> s/sugar/style_complex_color.rs which is a reasonable small rs file I
> happen to be familiar with), it takes ~2min to link (even longer than the
> fresh linking!) in addition to the compile time, and size of xul.ilk
> increases by 27.44MB. And after that, when I touch that cpp file again, it
> takes ~2.1min to link and size of xul.ilk increases by 54.64MB this time.
>
> I suspect the more times you touch rs files, the longer linking would
> take, and I guess this is because we link all Rust code into a single
> library, and then static link it into xul.dll, which makes the linker do
> lots of wasteful work. There could also be some linker bug involves, though.
>
> To mitigate this, we can probably avoid incremental linking when Rust code
> is touched. I also wonder if it is possible to avoid linking the whole
> gkrust into xul.dll at all and see if that makes things better. It would
> also be good to see if Rust developers have any advice for this situation.
>

Would you mind filing a bug report for this issue?

I've posted a link to this thread on the Rust Internals board:
https://internals.rust-lang.org/t/dev-platform-discussion-
on-getting-more-rust-code-into-firefox/5523

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Nicholas Nethercote]

On Tue, Jul 11, 2017 at 2:33 AM, Bobby Holley  wrote:

>
> In other words, I think we should encourage people to consider Rust for
> new code, but also make sure that we have a thoughtful and informed
> discussion about whether a use-case makes sense and how best to support it.
>

Definitely! I deliberately described my "Rust should be considered /
preferred / mandated" goal as "provocative" :)  We should think big, and
then put engineering effort in where it makes sense and will have strong
benefits.


> Right now we have a module that's intended to govern decisions on these
> issues [1]. However, the peer list for that module seems oriented around
> C++ and build system expertise, and is sparse on people who have been
> deeply involved in the Rust integration efforts over the past year.
>
> So we could expand that group with expertise to tackle Rust integration
> issues. Or it could charter a committee of such experts to do so. Or some
> combination. Either way, I think we want some amount of steering here, and
> I can think of a handful of names whose input is likely required to get it
> right.
>

That module's name is "C++/Rust usage, tools, and style", and its
description is "Aspects of C++ use such as language feature usage, standard
library versions/usage, compiler/toolchain versions, formatting and naming
style, and aspects of Rust use as needs arise"

To me that feels orthogonal to the notion of encouraging the adoption of
more Rust components in Firefox. That kind of encouragement feels more
project-wide, and so IMO would fall under the "mozilla-toplevel" component.
IIRC Brendan resigned ownership of that module a while back but he's still
listed as the owner at https://wiki.mozilla.org/Modules/All#mozilla-toplevel.
So that leaves things in an uncertain state.

If I were the owner of that module I would consider implementing a policy
something like the following:

"When a person writes a new compiled-code component, or majorly rewrites an
existing one, they should strongly consider writing it in Rust instead of C
or C++. Such a decision will depend on the nature of the component.
Components that are relatively standalone and have simple interfaces to
other components are good candidates to be written in Rust, as are
components such as parsers that process untrusted input."

It could do with some tweaking and feedback, but in principle I don't see
why such a policy couldn't be implemented right now. I don't know where
this policy would be written down, though!

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Nicholas Nethercote]

On Mon, Jul 10, 2017 at 11:41 PM, smaug  wrote:

>
> - Interop with existing components can be difficult. IPDL codegen rust
>> bindings could be a big help.
>>
>
> ipdl? or do you mean idl? or perhaps webidl?
>

bkelly suggested that item to me on Twitter :)  Probably all three of those
are relevant!

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Xidorn Quan]

On Mon, Jul 10, 2017, at 08:29 PM, Nicholas Nethercote wrote:
> Firefox now has multiple Rust components, and it's on track to get a bunch 
> more. See https://wiki.mozilla.org/Oxidation for details.> 
> I think this is an excellent trend, and I've been thinking about how to 
> accelerate it. Here's a provocative goal worth considering: "when writing a 
> new compiled-code component, or majorly rewriting an existing one, Rust 
> should be considered / preferred / mandated."> 
> What are the obstacles? Here are some that I've heard.
> 
> - Lack of Rust expertise for both writing and reviewing code. We have some 
> pockets of expertise, but these need to be expanded greatly. I've heard that 
> there has been some Rust training in the Paris and Toronto offices. Would 
> training in other offices (esp. MV and SF, given their size) be a good idea? 
> What about remoties?> 
> - ARM/Android is not yet a Tier-1 platform for Rust. See 
> https://forge.rust-lang.org/platform-support.html and 
> https://internals.rust-lang.org/t/arm-android-to-tier-1/5227 for some 
> details.> 
> - Interop with existing components can be difficult. IPDL codegen rust 
> bindings could be a big help.> 
> - Compile times are high, especially for optimized builds.
> 
> Anything else?

Tooling around debug info in Rust code still isn't great.

The first thing comes to my mind is crash reports. It currently doesn't always 
include useful panic message from Rust, see for example [1] and [2]. Also for 
Stylo, we generate lots of code (including using bindgen and mako template 
system, bindgen is usually fine, but the code generated from template can 
contain lots of code logic), and when the crash happens inside generated code, 
it is pretty hard to understand what's going wrong, because there is no useful 
source link, see for example [3].
There are also issues from Rust side. I've always been using an optimized debug 
build locally (because that runs faster), and sometimes use Visual Studio to 
investigate issues. C++ code works fine with this configuration, but in Rust 
code, I cannot see any variable information. Stack backtrace seems to work fine 
to me, though.
[1] 
https://crash-stats.mozilla.com/report/index/2abff06f-d969-4ba5-845b-a98410170708[2]
 
https://crash-stats.mozilla.com/report/index/03718a9c-9d98-4832-b8a6-026220170706[3]
 
https://crash-stats.mozilla.com/report/index/6b7d1d78-8418-47ef-bee9-f49c20170710

Besides tooling, not just compile time, but also the link time is high. 
Specifically, I suspect our linking strategy for Rust code doesn't interact 
well with MSVC's incremental linker, which slows down this final sequential 
step.
For that, I just did an experiment. In my local repo, when I remove all 
linking-related files from objdir/toolkit/library (to do a fresh linking), it 
takes ~1.7min for linking.
When I touch one cpp file (in my experiment, layout/style/nsCSSAnonBoxes.cpp 
which is a reasonable small cpp file I happen to open), it takes 6s to link, 
and size of xul.ilk (Incremental Linker File) increases by 856KB.
When I touch one rs file (in my experiment, 
servo/components/gecko_bindings/sugar/style_complex_color.rs which is a 
reasonable small rs file I happen to be familiar with), it takes ~2min to link 
(even longer than the fresh linking!) in addition to the compile time, and size 
of xul.ilk increases by 27.44MB. And after that, when I touch that cpp file 
again, it takes ~2.1min to link and size of xul.ilk increases by 54.64MB this 
time.
I suspect the more times you touch rs files, the longer linking would take, and 
I guess this is because we link all Rust code into a single library, and then 
static link it into xul.dll, which makes the linker do lots of wasteful work. 
There could also be some linker bug involves, though.
To mitigate this, we can probably avoid incremental linking when Rust code is 
touched. I also wonder if it is possible to avoid linking the whole gkrust into 
xul.dll at all and see if that makes things better. It would also be good to 
see if Rust developers have any advice for this situation.

- Xidorn
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[smaug]

On 07/10/2017 09:04 PM, zbranie...@mozilla.com wrote:

One more thought. There's a project that fitzgen told me about that aims to 
allow for components to communicate between JS and Rust using Streams.

If we could get to the point where instead of WebIDL/XPIDL we could just plug 
streams between JS/CPP and Rust in Gecko, I believe the scope of Gecko 
components that can be written in Rust would skyrocket.

zb.



FWIW, that sounds rather heavy weight to me, and we should be optimizing 
everything.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[zbraniecki]

One more thought. There's a project that fitzgen told me about that aims to 
allow for components to communicate between JS and Rust using Streams.

If we could get to the point where instead of WebIDL/XPIDL we could just plug 
streams between JS/CPP and Rust in Gecko, I believe the scope of Gecko 
components that can be written in Rust would skyrocket.

zb.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Kris Maglione]

On Mon, Jul 10, 2017 at 08:29:59PM +1000, Nicholas Nethercote wrote:

I think this is an excellent trend, and I've been thinking about how to
accelerate it...

- Interop with existing components can be difficult. IPDL codegen rust
bindings could be a big help.


So far, this has been my biggest issue. Or, to be more specific, 
the lack of interop, combined with the lack of documentation for 
the current state of interop, or how to go about writing new 
Rust components, or guidelines for when and how to use it.


In particular, every piece of code I've considered writing in 
Rust would have needed to interact with DOM and/or Spidermonkey 
and/or IPDL bindings, and the amount of C++ glue code I would 
have had to write seemed to outweigh the benefits of writing it 
in Rust. Combined with the fact that I would have needed to find 
and dig through various scattered mailing list posts and wiki 
pages, and then pester a bunch of people by email or IRC just to 
get started, I've always given up the idea pretty quickly.


That said, there are things that I would *love* to be able to 
write in Rust rather than in C++, so if there's anything we can 
do to improve this situation, I'm very much on board.


-Kris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Bobby Holley]

There are obvious benefits to writing new things in Rust. In general, the
main barrier to doing so is the extent to which they may interact with
existing C++ code, and the adequacy of our tooling for managing that
interaction.

In London last year, we had a meeting to discuss how we wanted to use Rust
for DOM code. The consensus of that meeting was that Rust code could be
useful as a backend for DOM functionality, but that it still needed to be
wrapped inside a C++ shell to handle reflection, cycle collection, and
interaction with the rest of the (C++-based) DOM.

Our tooling (specifically around bindgen) has matured significantly since
then, largely to support stylo. But there are still a lot of unsolved use
cases, and some of them (like inlining) are pretty hard to solve. The costs
of haphazard integration can be high, specifically to performance, safety,
and complexity/maintainability. So I think we want to tread carefully here.

In other words, I think we should encourage people to consider Rust for new
code, but also make sure that we have a thoughtful and informed discussion
about whether a use-case makes sense and how best to support it.

Right now we have a module that's intended to govern decisions on these
issues [1]. However, the peer list for that module seems oriented around
C++ and build system expertise, and is sparse on people who have been
deeply involved in the Rust integration efforts over the past year.

So we could expand that group with expertise to tackle Rust integration
issues. Or it could charter a committee of such experts to do so. Or some
combination. Either way, I think we want some amount of steering here, and
I can think of a handful of names whose input is likely required to get it
right.

Ehsan, as module owner, wdyt?
bholley

[1]
https://wiki.mozilla.org/Modules/All#C.2B.2B.2FRust_usage.2C_tools.2C_and_style

On Mon, Jul 10, 2017 at 3:29 AM, Nicholas Nethercote  wrote:

> Hi,
>
> Firefox now has multiple Rust components, and it's on track to get a bunch
> more. See https://wiki.mozilla.org/Oxidation for details.
>
> I think this is an excellent trend, and I've been thinking about how to
> accelerate it. Here's a provocative goal worth considering: "when writing a
> new compiled-code component, or majorly rewriting an existing one, Rust
> should be considered / preferred / mandated."
>
> What are the obstacles? Here are some that I've heard.
>
> - Lack of Rust expertise for both writing and reviewing code. We have some
> pockets of expertise, but these need to be expanded greatly. I've heard
> that there has been some Rust training in the Paris and Toronto offices.
> Would training in other offices (esp. MV and SF, given their size) be a
> good idea? What about remoties?
>
> - ARM/Android is not yet a Tier-1 platform for Rust. See
> https://forge.rust-lang.org/platform-support.html and
> https://internals.rust-lang.org/t/arm-android-to-tier-1/5227 for some
> details.
>
> - Interop with existing components can be difficult. IPDL codegen rust
> bindings could be a big help.
>
> - Compile times are high, especially for optimized builds.
>
> Anything else?
>
> Nick
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Zbigniew Braniecki (Gandalf)]

Hi,

This year I've been working on three new components[0] and I considered
Rust for both.

At the time, when asked people on #rust and #developers, the answer was
that oxidation targets should have very simple APIs, because getting
anything more complex than strings/ints between rust and cpp/js is
non-trivial.

I don't know if that changed, but I'd appreciate any tutorial on how to
write a WebIDL/XPIDL module or transfer JSON between Rust and JS/CPP.

Thanks,
zb.

[0] intl/locale/LocaleService, intl/locale/OSPreferences and
intl/l10n/Localization

On Mon, Jul 10, 2017 at 8:41 AM, Sylvestre Ledru 
wrote:

> I am organizing a second rust training in Paris in September or October
> (amongst others)
> The registration has been sent to the French staff last week. Will be
> shared with the European staff at the end of the week.
>
>
> Le lun. 10 juil. 2017 à 12:30, Nicholas Nethercote 
> a écrit :
>
>> Hi,
>>
>> Firefox now has multiple Rust components, and it's on track to get a
>> bunch more. See https://wiki.mozilla.org/Oxidation for details.
>>
>> I think this is an excellent trend, and I've been thinking about how to
>> accelerate it. Here's a provocative goal worth considering: "when writing a
>> new compiled-code component, or majorly rewriting an existing one, Rust
>> should be considered / preferred / mandated."
>>
>> What are the obstacles? Here are some that I've heard.
>>
>> - Lack of Rust expertise for both writing and reviewing code. We have
>> some pockets of expertise, but these need to be expanded greatly. I've
>> heard that there has been some Rust training in the Paris and Toronto
>> offices. Would training in other offices (esp. MV and SF, given their size)
>> be a good idea? What about remoties?
>>
>> - ARM/Android is not yet a Tier-1 platform for Rust. See
>> https://forge.rust-lang.org/platform-support.html and
>> https://internals.rust-lang.org/t/arm-android-to-tier-1/5227 for some
>> details.
>>
>> - Interop with existing components can be difficult. IPDL codegen rust
>> bindings could be a big help.
>>
>> - Compile times are high, especially for optimized builds.
>>
>> Anything else?
>>
>> Nick
>> ___
>> firefox-dev mailing list
>> firefox-...@mozilla.org
>> https://mail.mozilla.org/listinfo/firefox-dev
>>
>
> ___
> firefox-dev mailing list
> firefox-...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Sylvestre Ledru]

I am organizing a second rust training in Paris in September or October
(amongst others)
The registration has been sent to the French staff last week. Will be
shared with the European staff at the end of the week.


Le lun. 10 juil. 2017 à 12:30, Nicholas Nethercote 
a écrit :

> Hi,
>
> Firefox now has multiple Rust components, and it's on track to get a bunch
> more. See https://wiki.mozilla.org/Oxidation for details.
>
> I think this is an excellent trend, and I've been thinking about how to
> accelerate it. Here's a provocative goal worth considering: "when writing a
> new compiled-code component, or majorly rewriting an existing one, Rust
> should be considered / preferred / mandated."
>
> What are the obstacles? Here are some that I've heard.
>
> - Lack of Rust expertise for both writing and reviewing code. We have some
> pockets of expertise, but these need to be expanded greatly. I've heard
> that there has been some Rust training in the Paris and Toronto offices.
> Would training in other offices (esp. MV and SF, given their size) be a
> good idea? What about remoties?
>
> - ARM/Android is not yet a Tier-1 platform for Rust. See
> https://forge.rust-lang.org/platform-support.html and
> https://internals.rust-lang.org/t/arm-android-to-tier-1/5227 for some
> details.
>
> - Interop with existing components can be difficult. IPDL codegen rust
> bindings could be a big help.
>
> - Compile times are high, especially for optimized builds.
>
> Anything else?
>
> Nick
> ___
> firefox-dev mailing list
> firefox-...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Michael Layzell]

On Mon, Jul 10, 2017 at 9:41 AM, smaug  wrote:

> ipdl? or do you mean idl? or perhaps webidl?
> Also, xpconnect doesn't deal with rust, and our chrome code still heavily
> relies on idl+xpconnect.
>

 I have written https://bugzilla.mozilla.org/show_bug.cgi?id=1293362, which
while I'm not actively working on, could be used to help close that
idl+xpconnect gap a bit. I've definitely had some opposition to the idea of
adding rust bindings to XPCOM interfaces because we're theoretically trying
to kill them however. If we decide that getting good bindings is a thing
which we want in order to to develop more stuff in rust I can polish the
patches back up.

memory management. As far as I know there is no reasonable way to deal with
> cycle collection (which means also no reasonable way to have references to
> JS objects).
>

I don't have my WIP cycle collection with rust XPCOM interfaces patches on
that bug, but both of these things are doable with XPCOM bindings for rust.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: More Rust code

[Frederik Braun]

On 10.07.2017 12:29, Nicholas Nethercote wrote:
> 
> What are the obstacles? Here are some that I've heard.
> 
> - Lack of Rust expertise for both writing and reviewing code. We have some
> pockets of expertise, but these need to be expanded greatly. I've heard
> that there has been some Rust training in the Paris and Toronto offices.
> Would training in other offices (esp. MV and SF, given their size) be a
> good idea? What about remoties?

FYI I have been (secretly[1]) thinking about Rust trainings in the
Berlin office.
AFAIU the previous trainings used the professional development budget
from those who attended. Naturally, the same could be true for remoties?
Though of course they'd also have to budget for travel/accomodation.





[1] Maybe today is the day, I should start finding out what the demand is
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform