Re: Dead-code removal of unused Rust FFI exports
On Thu, Aug 30, 2018 at 10:21:09AM -0500, Tom Ritter wrote: > CFI vcall requires one to specify a -fvisibility flag on the command line, > with hidden being the preffered. We set visibility explicitly in some > difficult-to-quickly-identify ways, and adding -fvisibility=hidden > triggered issues with NSS (as well as apparently being redundant to what we > currently do). I tracked them in > https://bugzilla.mozilla.org/show_bug.cgi?id=1459314 The real requirement is for the maximum number of symbols not to be exported, which usually doesn't happen. But it does in our build system, without -fvisibility. If clang doesn't want to honor this, that is a problem, and short-sighted on their part. > That bug includes a monster patch to set visibility manually on a ton of > NSS stuff to get the browser running as a POC, but CFI vcall would need > something much more intelligent. I think the answer is to change the > compiler to not require the flag be present. > > > For Firefox, simply having an equivalent to `-fvisilbility=hidden` so > that all the symbols in the generated static library are hidden would be > sufficient to meet our current needs > > The understanding I got from my bug was that we were already doing the > equivalent of this... somehow. Rust doesn't export as many symbols as C++ does by default. Practically speaking, it's doing the same as we do for C++, without an explicit compiler flag. The "problem" is that it still places *some* symbols public, but we don't need them to be, and it would be better for us to be able to *force* those to be hidden. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Dead-code removal of unused Rust FFI exports
CFI vcall requires one to specify a -fvisibility flag on the command line,
with hidden being the preffered. We set visibility explicitly in some
difficult-to-quickly-identify ways, and adding -fvisibility=hidden
triggered issues with NSS (as well as apparently being redundant to what we
currently do). I tracked them in
https://bugzilla.mozilla.org/show_bug.cgi?id=1459314
That bug includes a monster patch to set visibility manually on a ton of
NSS stuff to get the browser running as a POC, but CFI vcall would need
something much more intelligent. I think the answer is to change the
compiler to not require the flag be present.
> For Firefox, simply having an equivalent to `-fvisilbility=hidden` so
that all the symbols in the generated static library are hidden would be
sufficient to meet our current needs
The understanding I got from my bug was that we were already doing the
equivalent of this... somehow.
-tom
On Thu, Aug 30, 2018 at 9:52 AM, Ted Mielczarek wrote:
> I thought we had filed a Rust issue on this but apparently not. There's
> some good discussion in this issue:
> https://github.com/rust-lang/rust/issues/37530
>
> For Firefox, simply having an equivalent to `-fvisilbility=hidden` so that
> all the symbols in the generated static library are hidden would be
> sufficient to meet our current needs, since we don't need to export any
> actual public APIs from Rust code. For that feature to be usable for more
> use cases you'd probably also want an equivalent to `__attribute__
> ((visibility ("default")))`, but I don't know what exactly that'd look like.
>
> -Ted
>
> On Wed, Aug 29, 2018, at 3:38 AM, Michael Woerister wrote:
> > For some background: The Rust compiler will mark everything as `hidden`
> in
> > LLVM IR unless it is exported via a C interface (via either #[no_mangle]
> or
> > #[export_name]). So the FFI symbols in gkrust will have default
> visibility.
> > LLD will thus probably retain and re-export them, even if they are not
> > called from within libXUL. But we should double-check that.
> >
> > It might be a good idea to provide a way to tell the Rust compiler to
> make
> > everything `hidden` by default.
> >
> > On Tue, Aug 28, 2018 at 3:56 PM, Till Schneidereit <
> > t...@tillschneidereit.net> wrote:
> >
> > > CC'ing mw, who is working on cross-language LTO.
> > >
> > > On Tue, Aug 28, 2018 at 3:20 PM Mike Hommey wrote:
> > >
> > >> On Tue, Aug 28, 2018 at 09:14:02AM -0400, Jeff Muizelaar wrote:
> > >> > We don't LTO yet on Mac.
> > >>
> > >> We don't LTO across languages on any platform yet. Rust is LTOed on
> all
> > >> platforms, which removes a bunch of its symbols. Everything that is
> > >> exposed for C/C++ from Rust, though, is left alone. That's likely to
> > >> stay true even with cross-language LTO, because as far as the linker
> is
> > >> concerned, those FFI symbols might be used by code that link against
> > >> libxul, so it would still export them. We'd essentially need the
> > >> equivalent to -fvisibility=hidden for Rust for that to stop being
> true.
> > >>
> > >> Mike
> > >>
> > >>
> > >> > On Tue, Aug 28, 2018 at 5:17 AM, Emilio Cobos Álvarez <
> emi...@crisal.io>
> > >> wrote:
> > >> > >
> > >> > >
> > >> > > On 8/28/18 9:35 AM, Henri Sivonen wrote:
> > >> > >>
> > >> > >> Does some lld mechanism successfully remove dead code when gkrust
> > >> > >> exports some FFI function that the rest of Gecko never ends up
> > >> > >> calling?
> > >> > >
> > >> > >
> > >> > > I would expect LTO to get rid of it, but haven't checked myself.
> > >> > >
> > >> > >> I.e. in terms of code size, is it OK to vendor an FFI-exposing
> Rust
> > >> > >> crate where not every FFI function is used (at least right away)?
> > >> > >>
> > >> > > ___
> > >> > > dev-platform mailing list
> > >> > > dev-platform@lists.mozilla.org
> > >> > > https://lists.mozilla.org/listinfo/dev-platform
> > >> > ___
> > >> > dev-platform mailing list
> > >> > dev-platform@lists.mozilla.org
> > >> > https://lists.mozilla.org/listinfo/dev-platform
> > >> ___
> > >> dev-platform mailing list
> > >> dev-platform@lists.mozilla.org
> > >> https://lists.mozilla.org/listinfo/dev-platform
> > >>
> > >
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Dead-code removal of unused Rust FFI exports
I thought we had filed a Rust issue on this but apparently not. There's some
good discussion in this issue:
https://github.com/rust-lang/rust/issues/37530
For Firefox, simply having an equivalent to `-fvisilbility=hidden` so that all
the symbols in the generated static library are hidden would be sufficient to
meet our current needs, since we don't need to export any actual public APIs
from Rust code. For that feature to be usable for more use cases you'd probably
also want an equivalent to `__attribute__ ((visibility ("default")))`, but I
don't know what exactly that'd look like.
-Ted
On Wed, Aug 29, 2018, at 3:38 AM, Michael Woerister wrote:
> For some background: The Rust compiler will mark everything as `hidden` in
> LLVM IR unless it is exported via a C interface (via either #[no_mangle] or
> #[export_name]). So the FFI symbols in gkrust will have default visibility.
> LLD will thus probably retain and re-export them, even if they are not
> called from within libXUL. But we should double-check that.
>
> It might be a good idea to provide a way to tell the Rust compiler to make
> everything `hidden` by default.
>
> On Tue, Aug 28, 2018 at 3:56 PM, Till Schneidereit <
> t...@tillschneidereit.net> wrote:
>
> > CC'ing mw, who is working on cross-language LTO.
> >
> > On Tue, Aug 28, 2018 at 3:20 PM Mike Hommey wrote:
> >
> >> On Tue, Aug 28, 2018 at 09:14:02AM -0400, Jeff Muizelaar wrote:
> >> > We don't LTO yet on Mac.
> >>
> >> We don't LTO across languages on any platform yet. Rust is LTOed on all
> >> platforms, which removes a bunch of its symbols. Everything that is
> >> exposed for C/C++ from Rust, though, is left alone. That's likely to
> >> stay true even with cross-language LTO, because as far as the linker is
> >> concerned, those FFI symbols might be used by code that link against
> >> libxul, so it would still export them. We'd essentially need the
> >> equivalent to -fvisibility=hidden for Rust for that to stop being true.
> >>
> >> Mike
> >>
> >>
> >> > On Tue, Aug 28, 2018 at 5:17 AM, Emilio Cobos Álvarez
> >> wrote:
> >> > >
> >> > >
> >> > > On 8/28/18 9:35 AM, Henri Sivonen wrote:
> >> > >>
> >> > >> Does some lld mechanism successfully remove dead code when gkrust
> >> > >> exports some FFI function that the rest of Gecko never ends up
> >> > >> calling?
> >> > >
> >> > >
> >> > > I would expect LTO to get rid of it, but haven't checked myself.
> >> > >
> >> > >> I.e. in terms of code size, is it OK to vendor an FFI-exposing Rust
> >> > >> crate where not every FFI function is used (at least right away)?
> >> > >>
> >> > > ___
> >> > > dev-platform mailing list
> >> > > dev-platform@lists.mozilla.org
> >> > > https://lists.mozilla.org/listinfo/dev-platform
> >> > ___
> >> > dev-platform mailing list
> >> > dev-platform@lists.mozilla.org
> >> > https://lists.mozilla.org/listinfo/dev-platform
> >> ___
> >> dev-platform mailing list
> >> dev-platform@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-platform
> >>
> >
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Dead-code removal of unused Rust FFI exports
For some background: The Rust compiler will mark everything as `hidden` in LLVM IR unless it is exported via a C interface (via either #[no_mangle] or #[export_name]). So the FFI symbols in gkrust will have default visibility. LLD will thus probably retain and re-export them, even if they are not called from within libXUL. But we should double-check that. It might be a good idea to provide a way to tell the Rust compiler to make everything `hidden` by default. On Tue, Aug 28, 2018 at 3:56 PM, Till Schneidereit < t...@tillschneidereit.net> wrote: > CC'ing mw, who is working on cross-language LTO. > > On Tue, Aug 28, 2018 at 3:20 PM Mike Hommey wrote: > >> On Tue, Aug 28, 2018 at 09:14:02AM -0400, Jeff Muizelaar wrote: >> > We don't LTO yet on Mac. >> >> We don't LTO across languages on any platform yet. Rust is LTOed on all >> platforms, which removes a bunch of its symbols. Everything that is >> exposed for C/C++ from Rust, though, is left alone. That's likely to >> stay true even with cross-language LTO, because as far as the linker is >> concerned, those FFI symbols might be used by code that link against >> libxul, so it would still export them. We'd essentially need the >> equivalent to -fvisibility=hidden for Rust for that to stop being true. >> >> Mike >> >> >> > On Tue, Aug 28, 2018 at 5:17 AM, Emilio Cobos Álvarez >> wrote: >> > > >> > > >> > > On 8/28/18 9:35 AM, Henri Sivonen wrote: >> > >> >> > >> Does some lld mechanism successfully remove dead code when gkrust >> > >> exports some FFI function that the rest of Gecko never ends up >> > >> calling? >> > > >> > > >> > > I would expect LTO to get rid of it, but haven't checked myself. >> > > >> > >> I.e. in terms of code size, is it OK to vendor an FFI-exposing Rust >> > >> crate where not every FFI function is used (at least right away)? >> > >> >> > > ___ >> > > dev-platform mailing list >> > > dev-platform@lists.mozilla.org >> > > https://lists.mozilla.org/listinfo/dev-platform >> > ___ >> > dev-platform mailing list >> > dev-platform@lists.mozilla.org >> > https://lists.mozilla.org/listinfo/dev-platform >> ___ >> dev-platform mailing list >> dev-platform@lists.mozilla.org >> https://lists.mozilla.org/listinfo/dev-platform >> > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Dead-code removal of unused Rust FFI exports
On Tue, Aug 28, 2018 at 4:56 PM, Till Schneidereit wrote: > On Tue, Aug 28, 2018 at 3:20 PM Mike Hommey wrote: >> We don't LTO across languages on any platform yet. Rust is LTOed on all >> platforms, which removes a bunch of its symbols. Everything that is >> exposed for C/C++ from Rust, though, is left alone. That's likely to >> stay true even with cross-language LTO, because as far as the linker is >> concerned, those FFI symbols might be used by code that link against >> libxul, so it would still export them. We'd essentially need the >> equivalent to -fvisibility=hidden for Rust for that to stop being true. Exporting Rust FFI symbols from libxul seems bad not just for binary size but also in terms of giving contact surface to invasive third-party Windows software. Do we have a bug on file tracking the hiding of FFI symbols from the outside of libxul? -- Henri Sivonen hsivo...@mozilla.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Dead-code removal of unused Rust FFI exports
CC'ing mw, who is working on cross-language LTO. On Tue, Aug 28, 2018 at 3:20 PM Mike Hommey wrote: > On Tue, Aug 28, 2018 at 09:14:02AM -0400, Jeff Muizelaar wrote: > > We don't LTO yet on Mac. > > We don't LTO across languages on any platform yet. Rust is LTOed on all > platforms, which removes a bunch of its symbols. Everything that is > exposed for C/C++ from Rust, though, is left alone. That's likely to > stay true even with cross-language LTO, because as far as the linker is > concerned, those FFI symbols might be used by code that link against > libxul, so it would still export them. We'd essentially need the > equivalent to -fvisibility=hidden for Rust for that to stop being true. > > Mike > > > > On Tue, Aug 28, 2018 at 5:17 AM, Emilio Cobos Álvarez > wrote: > > > > > > > > > On 8/28/18 9:35 AM, Henri Sivonen wrote: > > >> > > >> Does some lld mechanism successfully remove dead code when gkrust > > >> exports some FFI function that the rest of Gecko never ends up > > >> calling? > > > > > > > > > I would expect LTO to get rid of it, but haven't checked myself. > > > > > >> I.e. in terms of code size, is it OK to vendor an FFI-exposing Rust > > >> crate where not every FFI function is used (at least right away)? > > >> > > > ___ > > > dev-platform mailing list > > > dev-platform@lists.mozilla.org > > > https://lists.mozilla.org/listinfo/dev-platform > > ___ > > dev-platform mailing list > > dev-platform@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-platform > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Dead-code removal of unused Rust FFI exports
On Tue, Aug 28, 2018 at 09:14:02AM -0400, Jeff Muizelaar wrote: > We don't LTO yet on Mac. We don't LTO across languages on any platform yet. Rust is LTOed on all platforms, which removes a bunch of its symbols. Everything that is exposed for C/C++ from Rust, though, is left alone. That's likely to stay true even with cross-language LTO, because as far as the linker is concerned, those FFI symbols might be used by code that link against libxul, so it would still export them. We'd essentially need the equivalent to -fvisibility=hidden for Rust for that to stop being true. Mike > On Tue, Aug 28, 2018 at 5:17 AM, Emilio Cobos Álvarez > wrote: > > > > > > On 8/28/18 9:35 AM, Henri Sivonen wrote: > >> > >> Does some lld mechanism successfully remove dead code when gkrust > >> exports some FFI function that the rest of Gecko never ends up > >> calling? > > > > > > I would expect LTO to get rid of it, but haven't checked myself. > > > >> I.e. in terms of code size, is it OK to vendor an FFI-exposing Rust > >> crate where not every FFI function is used (at least right away)? > >> > > ___ > > dev-platform mailing list > > dev-platform@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-platform > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Dead-code removal of unused Rust FFI exports
We don't LTO yet on Mac. -Jeff On Tue, Aug 28, 2018 at 5:17 AM, Emilio Cobos Álvarez wrote: > > > On 8/28/18 9:35 AM, Henri Sivonen wrote: >> >> Does some lld mechanism successfully remove dead code when gkrust >> exports some FFI function that the rest of Gecko never ends up >> calling? > > > I would expect LTO to get rid of it, but haven't checked myself. > >> I.e. in terms of code size, is it OK to vendor an FFI-exposing Rust >> crate where not every FFI function is used (at least right away)? >> > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Dead-code removal of unused Rust FFI exports
On 8/28/18 9:35 AM, Henri Sivonen wrote: Does some lld mechanism successfully remove dead code when gkrust exports some FFI function that the rest of Gecko never ends up calling? I would expect LTO to get rid of it, but haven't checked myself. I.e. in terms of code size, is it OK to vendor an FFI-exposing Rust crate where not every FFI function is used (at least right away)? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Dead-code removal of unused Rust FFI exports
Does some lld mechanism successfully remove dead code when gkrust exports some FFI function that the rest of Gecko never ends up calling? I.e. in terms of code size, is it OK to vendor an FFI-exposing Rust crate where not every FFI function is used (at least right away)? -- Henri Sivonen hsivo...@mozilla.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
