Re: Linux distro readiness for Rust in Gecko
On Thu, Mar 17, 2016 at 1:58 PM, Martin Stransky wrote: > Is it possible to build Rust from sources before Firefox build executes? rustc is written in Rust, so if you only have source code and the compilers that are currently available in Fedora, you can't build rustc. At minimum, you need a "stage0" compiler, which is an archived older rustc binary. (rustc originally bootstrapped with OCaml, but building the whole history of Rust from the OCaml days to present every time Fedora builds Firefox seems excessively impractical.) Fedora's policy seems to prohibit continuous shipping of stage0 as a Mozilla-provided binary, so it seems to me that either Fedora needs to maintain rustc bootstrapped according to policy (bootstrapping using Mozilla-provided stage0 once) or a policy waiver for Firefox is needed. See also https://internals.rust-lang.org/t/perfecting-rust-packaging/2623 -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: An analysis of content process memory overhead
On 03/17/2016 08:05 AM, Thinker Li wrote: On Wednesday, March 16, 2016 at 10:22:40 PM UTC+8, Nicholas Nethercote wrote: Even if we can fix that, it's just a lot of JS code. We can lazily import JSMs; I wonder if we are failing to do that as much as we could, i.e. are all these modules really needed at start-up? It would be great if we could instrument module-loading code in some way that answers this question. B2G also did dropping JS source, for Tarako branch, since source is useless for loaded module save for stringify functions. (Gecko compress in-memory source.) But, I am not sure if it was landed on m-c then. Note, this worked on B2G, but this would not work for Gecko. For example all tabs addons have to use toSource to patch the JS functions. Source compressions should already be enabled. I think we do not do it for small sources, and for Huge sources, as the compression would either be useless, or it would take a noticeable amount of time. -- Nicolas B. Pierron ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to switch to Visual Studio 2015
On Thu, Mar 17, 2016 at 3:30 PM, wrote: > What about the depreciation of XP SP2? > Results from bug 1124017 say XP SP2 still works on binaries built with VS2015u1. This may not always hold true. So we will need to have the XP SP2 discussion at some point. I defer to bsmedberg to start that discussion. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On 3/19/16 5:27 AM, cosinusoida...@gmail.com wrote: On Thursday, 17 March 2016 12:23:32 UTC, Henri Sivonen wrote: On Thu, Mar 17, 2016 at 1:58 PM, Martin Stransky wrote: Is it possible to build Rust from sources before Firefox build executes? rustc is written in Rust, so if you only have source code and the compilers that are currently available in Fedora, you can't build rustc. At minimum, you need a "stage0" compiler, which is an archived older rustc binary. (rustc originally bootstrapped with OCaml, but building the whole history of Rust from the OCaml days to present every time Fedora builds Firefox seems excessively impractical.) Out of interest, would that actually involve building every single Linux snapshot from https://github.com/rust-lang/rust/blob/master/src/snapshots.txt in sequence? All 319 of them? And are those the steps you'd actually have to take to bring Rust up from scratch on a new platform? Cameron Kaiser tier-3s in rain dept. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On Fri, Mar 18, 2016 at 4:01 AM, Sylvestre Ledru wrote: > Le 17/03/2016 à 21:30, Henri Sivonen a écrit : > > On Thu, Mar 17, 2016 at 2:28 PM, Sylvestre Ledru > wrote: > >> In Debian & Ubuntu, > > I didn't find a rustc package on http://packages.ubuntu.com/ . What's > > the situation on Ubuntu? > I thought it was already synced for xenial. I just filled the request: > https://bugs.launchpad.net/bugs/1559001 > > > >> we use the official binaries provided to be able to build rust. > > My same logic, can Mozilla-built rustc be used to build the Firefox > > package as far as policy matters, with exceptions and waivers taken > > into account, go? Or is this what's keeping rustc not getting past > > testing? > Debian stable will use the version of rustc at the time of the Debian > freeze (January 2017) > > > > > > On Thu, Mar 17, 2016 at 2:47 PM, Sylvestre Ledru > wrote: > >> One way which would make the life of Linux distro way easier would be > >> to maintain the Firefox rust code in a way it could compile using older > rust compiler. > > In order to be competitive, we need all leverage we can get from our > > Mozilla magic sauce (Rust). I think it's unacceptable to limit our > > ability to leverage Rust in Gecko by forgoing the ability to co-evolve > > Rust and Gecko at a rapid pace. > I understand your point, it is just conflicting with the goals of Debian > and Ubuntu LTS. > Mozilla wants to move fast, distros wants to provide stable products... > > > >> Now, with my Debian/Ubuntu hat, maintaining rust backports to be able > to build new versions of Firefox on stable/LTS releases > >> is not going to be easy > > Could Firefox in Debian stable have a build dependency on rustc from > > Debian unstable? > Nope, this doesn't work this way. Packages must be built in the env they > are going to be used. > Otherwise, it would lead to side effects (different version of libraries > used for build time and runtimes). > > One dirty solution would be to ship rustc+llvm sources into Firefox > sources when targeting stable distros. > But this increases the load on the maintainers by an order of magnitude > (maintainers will have to manage rust & llvm) > and might not work if llvm starts requiring a more recent of gcc to build. > However, this is really the last option distros will consider (and I am > sure Glandium will choke when he will read this). Related to this, Tor has deterministic and reproducible Firefox builds. They basically have a trusted base image providing GCC, glibc, binutils, etc then rebuild the world from source. When we introduce Rust, they will presumably need a way to build rustc from source without bootstrapping from pre-built binaries [that can't be trusted]. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to implement: W3C WebAppSec credentialmanagement API
On 3/14/16 3:01 PM, Martin Thomson wrote: The actual benefit is something that is only realized once a site puts in the effort required. That is small, yes, but we're seeing sites actively avoid password managers, hence the aggressive heuristics, and rAC is much more likely to work for that, since it's implemented and deployed already. This is the key issue, IMO, which makes me not interested in having Firefox implement this API. Far too many sites either simply don't care about user password management (ie, they do problematic things that could easily be fixed), or actively take steps to intentionally break password managers. In the past we considered this an advocacy/evangelism problem, and it was deemed the site's responsibility to play nice. That's worked poorly, and sucks for users. We now believe that we have to assume a adversarial environment: it's our job to serve as the user's agent and do whatever it takes to work on a site. If there's interest in assisting sites that want to play nice, I think it would be better to start with documenting a set of cross-browser "best practices" that they can follow, for the standards and implementations that exist today. I'd also note that Mozilla Labs tried going down a similar path in the past, with Account Manager -- see https://hacks.mozilla.org/2010/04/account-manager-coming-to-firefox/. The team involved in that did a _lot_ of outreach with sites and web developers. I remember it as having a somewhat tepid reception... Especially around sites not wanting to cede any UX control to the browser in the signup/login/logout experience, and some industries being very disinterested doing anything beyond what was mandated by regulations (ie, they saw doing anything outside their rulebook as adding legal risk). Justin ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On Sat, Mar 19, 2016 at 01:45:49AM +0900, Mike Hommey wrote: > On Fri, Mar 18, 2016 at 05:23:21PM +0200, Henri Sivonen wrote: > > You say you don't see #5 happening. Do you see #4 happening? If not, > > what do you see happening? > > At this point, I'm wondering if the best outcome wouldn't be 6) Distros > die. I'm almost not joking. > > > > LTS distros do update Firefox because there is no way they can support > > > security updates on older releases (I've done it with 3.5 long enough to > > > know it's not tractable). But they do it once a year (at every ESR bump), > > > not every 6 weeks. > > > > This is not the case for Ubuntu LTS. Even Ubuntu 12.04 gets a new > > Firefox release every six weeks, and there is a package gcc-mozilla > > that backports a GCC newer than the original GCC in 12.04 as a build > > dependency: > > http://archive.ubuntu.com/ubuntu/pool/main/f/firefox/firefox_45.0+build2-0ubuntu0.12.04.1.dsc > > > > So, clearly, at least in the case of Ubuntu, there is precedent for 1) > > updating Firefox every six weeks in LTS and 2) the Firefox package > > having a build dependency on a compiler that's newer than the > > compilers that originally shipped with the LTS system release. > > > > When I started this thread, I thought the s/IceWeasel/Firefox/ change > > in Debian involved Debian starting to ship Firefox the way Ubuntu > > does. For clarity: Is that not the case and Debian will only ship ESR > > but an ESR that's within Mozilla's support period? I can see how > > shipping ESR is the closest approximation of compliance with a policy > > to ship outdated software, but how does ESR address Debian's package > > dependency issues? If the next ESR requires a compiler that's not in > > the current Debian stable, what then? > > > > Looking at > > https://www.debian.org/releases/stable/amd64/release-notes/ch-information.en.html#browser-security > > , it seems that Debian users get a more up-to-date Blink than Gecko. > > If that policy is any indication, if ESR didn't exist, Gecko would get > > the same deal as Blink... In other words, it looks like Debian > > penalizes Gecko relative to Blink because ESR exists. :-( > > Well, at some point Blink wasn't even in stable. I'm actually surprised > that it is now. But as a matter of fact, Debian's old stable is not > receiving Blink/Chromium updates (it's stuck on 37), while it receives > updates for Iceweasel (it has 38.7 as or writing, will receive 38.8, and > 45.2 after that) Note that this is why Blink/Chromium can get away with very frequent updates in stable and not Iceweasel/Firefox: $ grep-dctrl -sPackage -FDepends chromium --and --not -FSource chromium-browser /var/lib/apt/lists/ftp.jp.debian.org_debian_dists_jessie_main_binary-amd64_Packages | wc -l 2 (one is http://chromium-bsu.sourceforge.net/, the other is mozplugger, which... sounds like a mistake... I think it's an NPAPI plugin) $ grep-dctrl -sPackage -FDepends iceweasel --and --not -FSource iceweasel /var/lib/apt/lists/ftp.jp.debian.org_debian_dists_jessie_main_binary-amd64_Packages | wc -l 64 Iceweasel/Firefox is part of an ecosystem. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
> "Henri" == Henri Sivonen writes: Henri> Ubuntu and Fedora don't have any version of rustc anywhere at Henri> all. For Fedora it's in COPR, but of course that's not official, just something to play with. You can follow the Fedora Rust packaging process here: https://bugzilla.redhat.com/show_bug.cgi?id=915043 Tom ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On Friday, March 18, 2016 at 4:53:58 AM UTC-7, Sylvestre Ledru wrote: > Le 17/03/2016 à 21:30, Henri Sivonen a écrit : > > On Thu, Mar 17, 2016 at 2:28 PM, Sylvestre Ledru wrote: > >> In Debian & Ubuntu, > > I didn't find a rustc package on http://packages.ubuntu.com/ . What's > > the situation on Ubuntu? > I thought it was already synced for xenial. I just filled the request: > https://bugs.launchpad.net/bugs/1559001 > > > >> we use the official binaries provided to be able to build rust. > > My same logic, can Mozilla-built rustc be used to build the Firefox > > package as far as policy matters, with exceptions and waivers taken > > into account, go? Or is this what's keeping rustc not getting past > > testing? > Debian stable will use the version of rustc at the time of the Debian freeze > (January 2017) > > > > > > On Thu, Mar 17, 2016 at 2:47 PM, Sylvestre Ledru wrote: > >> One way which would make the life of Linux distro way easier would be > >> to maintain the Firefox rust code in a way it could compile using older > >> rust compiler. > > In order to be competitive, we need all leverage we can get from our > > Mozilla magic sauce (Rust). I think it's unacceptable to limit our > > ability to leverage Rust in Gecko by forgoing the ability to co-evolve > > Rust and Gecko at a rapid pace. > I understand your point, it is just conflicting with the goals of Debian and > Ubuntu LTS. > Mozilla wants to move fast, distros wants to provide stable products... > > > >> Now, with my Debian/Ubuntu hat, maintaining rust backports to be able to > >> build new versions of Firefox on stable/LTS releases > >> is not going to be easy > > Could Firefox in Debian stable have a build dependency on rustc from > > Debian unstable? > Nope, this doesn't work this way. Packages must be built in the env they are > going to be used. > Otherwise, it would lead to side effects (different version of libraries used > for build time and runtimes). > > One dirty solution would be to ship rustc+llvm sources into Firefox sources > when targeting stable distros. > But this increases the load on the maintainers by an order of magnitude > (maintainers will have to manage rust & llvm) > and might not work if llvm starts requiring a more recent of gcc to build. > However, this is really the last option distros will consider (and I am sure > Glandium will choke when he will read this). This seems like a workable solution, as horrible as it is. One could similarly make distro packages of rustc specifically for building firefox (called 'rustc-fx' for example), and put them on the same ESR upgrade schedule as Firefox. As another idea: Rust could maintain an LTS branch designed to be upgraded in conjunction with Firefox, that both keeps the language on a stable revision while also exposing backported features that Firefox needs in a controlled way. So Firefox would commit to being on e.g. 1.12 for 3 years, but could import 'future' features. Then LTS distros would put rustc LTS point releases on the same upgrade schedule as Firefox ESR upgrades. For Firefox this would require a lot of discipline. For Rust it would be a massive challenge coming up with a maintainable scheme. I'm actually thinking Rust could support multiple language revisions with a single compiler in a way that would allow old revisions to receive continual bugfixes and integrate 'future' features; and that would be reliable enough that LTS distros could make major version bumps to rustc alongside Firefox while still deploying the language revision compatible with their LTS. Difficult and crazy. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Linux distro readiness for Rust in Gecko
Do we have a plan being executed to make sure that when Gecko components written in Rust ship on the release channel in Mozilla-produced builds the also ship in the Firefox packages of various Linux distributions? If not, we really need to put such a plan in place, since it would be bad to have a conflict between our ability to reach Linux users and our ability to benefit from Rust quickly. My concern is that Linux distributions tend to have a policy that binary packages have to be buildable from source packages using the tools that are available in the package repository of the distro. Considering the current state of rustc availability in distro repositories, it seems to me that we are headed towards a problem. As far as I can tell, of the well-known distributions only Arch and openSUSE have the current rustc in their package repositories (and the openSUSE packages are categorized as "unstable"). Debian has it in unstable and testing only. Gentoo is a couple of rustc releases out of date. Ubuntu and Fedora don't have any version of rustc anywhere at all. At least in the case of Fedora, just shipping a rustc binary alongside the Firefox source is explicitly not good enough for their policy: https://fedoraproject.org/wiki/Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries Can we get Debian stable, Ubuntu and Fedora to ship a rustc package as a build dependency for Firefox every six weeks like the Firefox package itself in order for Firefox not to miss the 6-week train when Rust code ships in Mozilla-built Firefox and for Firefox to be able to co-evolve with rustc instead of getting stuck to an old rustc due to distro LTS cycles? (I'm assuming that it won't be a big deal to get Gentoo to pick up the most recent rustc when the use case shows up.) (Part of the problem seems to be at Mozilla's end, since rustc and the Rust standard lib don't compile with the stable rustc itself and the previous stable version of rustc making it hard to maintain a truly self-hosting package lineage as required by the Fedora policy.) -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: The convention to pass a parameter when ownership transfer is optional
On Friday, March 18, 2016 at 2:07:34 PM UTC+11, jww...@mozilla.com wrote: > https://hg.mozilla.org/mozilla-central/file/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/mfbt/UniquePtr.h#l182 > > "To unconditionally transfer ownership of a UniquePtr into a method, use a > |UniquePtr| argument. To conditionally transfer ownership of a resource into > a method, should the method want it, use a |UniquePtr&&| argument." > > Does that also apply to already_AddRefed<>&& or we stick to > https://bugzilla.mozilla.org/show_bug.cgi?id=1247972#c19? > > Btw, we have some code like > https://hg.mozilla.org/mozilla-central/file/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/dom/base/WebSocket.cpp#l2790 > where it really should be just already_AddRefed<> since the ownership is > always transferred. One concern I have with foo(T&&) being used to mark optional transfer, is that on the caller side there will be a foo(Move(t)) followed by code that may *still* uses 't'. I can't remember exactly where I read/heard that (Herb Sutter I think?), but it was advised that moved-from objects should only be destroyed, or completely overwritten through an assignment; and this philosophy was used to design STL containers move-related APIs. -- Or am I imagining things? Specific classes (like UniquePtr) might in fact be designed for this particular idiom, and I think it's fine when fully documented and tested. But it'd be great to come to agreement and document it in coding guidelines, whether we're going all guns blazing for T&&/Move-and-keep-using, or non-recommended-but-allowed-if-you-really-really-know-what-you're-doing, or something else entirely... gerald ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On Thu, Mar 17, 2016 at 2:28 PM, Sylvestre Ledru wrote: > In Debian & Ubuntu, I didn't find a rustc package on http://packages.ubuntu.com/ . What's the situation on Ubuntu? > we use the official binaries provided to be able to build rust. My same logic, can Mozilla-built rustc be used to build the Firefox package as far as policy matters, with exceptions and waivers taken into account, go? Or is this what's keeping rustc not getting past testing? > However, in the future, I have been told that we would be able to build rust > using rust version-1. When is rustc expected to reach that level of bootstrapping predictability? On Thu, Mar 17, 2016 at 2:47 PM, Sylvestre Ledru wrote: > One way which would make the life of Linux distro way easier would be > to maintain the Firefox rust code in a way it could compile using older rust > compiler. In order to be competitive, we need all leverage we can get from our Mozilla magic sauce (Rust). I think it's unacceptable to limit our ability to leverage Rust in Gecko by forgoing the ability to co-evolve Rust and Gecko at a rapid pace. > For example, imagine that the next Debian stable or Ubuntu LTS ships using > Rust 2.0, > we would make sure that the new rust code still compile with Rust 2.0 > (limiting our capability to use new > language trick). I think we must not allow ourselves to wait for a Debian or Ubuntu LTS cycle before Rust improvements can be used in Gecko. We need to leverage our advantages, such as Rust, fully to be competitive, and it would be crazy to work at LTS pace when we are free to ship to the majority of our users (on Windows and Mac) using whatever compiler works best. (In general, regardless of building Gecko, I think it would be harmful to the evolution of Rust at this stage for a version to get stuck out there for the duration of a Debian or Ubuntu LTS cycle. I agree with Brian Smith's sentiment at https://internals.rust-lang.org/t/perfecting-rust-packaging/2623/67 .) > Just like we do with C++... And we fail to make best competitive use of our time and tools with C++, too, when the latest clang, GCC and MSVC support a C++ feature, we put effort into writing code and find out it has to rot without landing because of old GCC. > Now, with my Debian/Ubuntu hat, maintaining rust backports to be able to > build new versions of Firefox on stable/LTS releases > is not going to be easy Could Firefox in Debian stable have a build dependency on rustc from Debian unstable? On Thu, Mar 17, 2016 at 3:01 PM, Martin Stransky wrote: > Well, what about other arches? ppc(64), s390, ai64...? All architectures currently supported by Fedora, Debian and Ubuntu (but not Gentoo!) already have LLVM back ends. rustc hasn't been ported to all those architectures, but the level of effort required should be within reason considering that the LLVM back ends are there and rustc has already been ported to target multiple ones of the LLVM-supported ISAs. If distro policy bans ongoing cross-compliation, I guess the distro would need to replicate the Rust project's compiler compilation version lineage on each architecture after bootstrapping with cross-compilation. On Thu, Mar 17, 2016 at 4:11 PM, Nicolas B. Pierron wrote: > I guess one of the thing we could do is use an alternative solution, such as > an external package manager which can work side-by-side with the host, such > as zero-install, Portage, or Nix. I don't see how that would be any better from the distro policy perspective than using Mozilla-shipped rustc. -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
The convention to pass a parameter when ownership transfer is optional
https://hg.mozilla.org/mozilla-central/file/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/mfbt/UniquePtr.h#l182 "To unconditionally transfer ownership of a UniquePtr into a method, use a |UniquePtr| argument. To conditionally transfer ownership of a resource into a method, should the method want it, use a |UniquePtr&&| argument." Does that also apply to already_AddRefed<>&& or we stick to https://bugzilla.mozilla.org/show_bug.cgi?id=1247972#c19? Btw, we have some code like https://hg.mozilla.org/mozilla-central/file/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/dom/base/WebSocket.cpp#l2790 where it really should be just already_AddRefed<> since the ownership is always transferred. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
Le 17/03/2016 à 14:01, Martin Stransky a écrit : > On 03/17/2016 01:47 PM, Sylvestre Ledru wrote: > > [...] > >> One way which would make the life of Linux distro way easier would be >> to maintain the Firefox rust code in a way it could compile using older rust >> compiler. >> >> For example, imagine that the next Debian stable or Ubuntu LTS ships using >> Rust 2.0, >> we would make sure that the new rust code still compile with Rust 2.0 >> (limiting our capability to use new >> language trick). Just like we do with C++... >> >> Now, with my Debian/Ubuntu hat, maintaining rust backports to be able to >> build new versions of Firefox on stable/LTS releases >> is not going to be easy: rust needs recent versions of LLVM, LLVM is a >> complex beast (with regular bump the minimal version of gcc, etc). >> And here, I am just mentioning i386 & amd64... (Debian/ubuntu don't have any >> rust builds for arm CPU). > > Well, what about other arches? ppc(64), s390, ai64...? For now, nothing has been done on Debian for this (except opening a bug [1]). Using upstream binaries makes our life very easy for i386/amd64. To start doing the porting job to other archs, being able to build using the previous version of the compiler is a requirement. Otherwise, the cross compilation would be needed for each upstream version. Sylvestre [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=809316 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to switch to Visual Studio 2015
On Thu, Mar 10, 2016 at 8:14 AM, Gregory Szorc wrote: > Visual Studio 2015 has been out for a while. Many people have put in work > to make Firefox build on it. The time has come to officially transition > release builds from Visual Studio 2013 to Visual Studio 2015. > > This email serves as an intent to switch automation to Visual Studio 2015 > Update 1 (latest stable release) as soon as possible, hopefully in the next > week or two. > > A big driver for the switch is that builds with VS2015 are faster. PGO > builds in automation are 1+ hour faster than with VS2013 (see data in bug > 1250797)! (Windows PGO builds are a long pole in the release process and > therefore prevent us from releasing faster - this is highly relevant during > chemspills.) > > A host of new C++ features should be available after the switch. Although, > we may have to drop support for VS2013 before those can be fully realized. > I defer to others to determine when VS2013 will be dropped. > > I feel like 95% of the transition work is completed. However, the > following Try pushes appear to have some new failures: > > https://treeherder.mozilla.org/#/jobs?repo=try&revision=d67e4a1f3735 > https://treeherder.mozilla.org/#/jobs?repo=try&revision=5c2a7b4e81d0 (PGO) > > (Ignore SpiderMonkey failures - they are due to how the toolchain is being > hackily installed in those Try pushes.) > > I could use your help triaging test failures and fixing fallout so we can > complete the transition to VS2015. > > I'm very much a fan of perfect is the enemy of done and I feel temporary > workarounds (like e.g. disabling tests if there appears to be a minor > regression) may be warranted so we can give VS2015 extra time to bake on > Nightly. Otherwise, this may slip ~6 weeks until the next release. I feel > like we're too close to being able to transition to VS2015 to wait ~6 more > weeks. > > Bug 1186060 is our master tracking bug for the VS2015 switch. > > Thank you for everyone that has contributed to VS2015 fixes so far. Thank > you in advance for helping complete the transition. > (+1 week progress report) All Windows builds are now working with VS2015u1 running from tooltool (read: we have a zip file containing all the toolchain files so we don't need changes to build machines to roll out new Visual Studio / SDK versions going forward). This includes PGO and SpiderMonkey builds. Talos numbers are at https://bugzilla.mozilla.org/show_bug.cgi?id=1254767#c9 and investigation into changes is ongoing. There are some concerning regressions - particularly around e10s - that have yet to be explained. Bug 1124033 has turned into a rabbit hole *and I could use help*. When VS2015 support was first added to the build system, new-to-VS2015 compiler warnings were blanket disabled. Bug 1124033 is about undoing that and enabling those warnings. There are 20+ open bugs tracking fixing compiler warnings. I've submitted patches to wallpaper over them by disabling warnings in specific files or directories. But this is not optimal: we should just fix the underlying warnings and leave the warning detection enabled. Unfortunately, my C++ knowledge is about 10 years out of date, very rusty, and I know very little about the C++ conventions used in mozilla-central. This is where I could use help. *If you see a bug chained up to bug 1124033 related to VS2015 compiler warnings, I'd appreciate help with patches to C++ to fix the warnings.* If you'd like to submit Try pushes using VS2015, https://bugzilla.mozilla.org/show_bug.cgi?id=1124033#c21 contains instructions. I'm still optimistic we'll be able to perform the switch (or at least attempt the switch) this release cycle. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
Le 17/03/2016 à 11:59, Henri Sivonen a écrit : > Do we have a plan being executed to make sure that when Gecko > components written in Rust ship on the release channel in > Mozilla-produced builds the also ship in the Firefox packages of > various Linux distributions? [...] > > Can we get Debian stable, Ubuntu and Fedora to ship a rustc package as > a build dependency for Firefox every six weeks like the Firefox > package itself in order for Firefox not to miss the 6-week train when > Rust code ships in Mozilla-built Firefox and for Firefox to be able to > co-evolve with rustc instead of getting stuck to an old rustc due to > distro LTS cycles? (I'm assuming that it won't be a big deal to get > Gentoo to pick up the most recent rustc when the use case shows up.) One way which would make the life of Linux distro way easier would be to maintain the Firefox rust code in a way it could compile using older rust compiler. For example, imagine that the next Debian stable or Ubuntu LTS ships using Rust 2.0, we would make sure that the new rust code still compile with Rust 2.0 (limiting our capability to use new language trick). Just like we do with C++... Now, with my Debian/Ubuntu hat, maintaining rust backports to be able to build new versions of Firefox on stable/LTS releases is not going to be easy: rust needs recent versions of LLVM, LLVM is a complex beast (with regular bump the minimal version of gcc, etc). And here, I am just mentioning i386 & amd64... (Debian/ubuntu don't have any rust builds for arm CPU). Sylvestre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On Thu, Mar 17, 2016 at 04:59:29PM -0700, bander...@mozilla.com wrote: > Hi, > > This is a matter that concerns the Rust team a lot. From the Rust side > we want to make it possible for Debian and Fedora (as a proxy for all > distros) to be able to bootstrap off of their own rustc's and keep up > with the Rust release process. > > We have a [good idea][1] about how to accomplish that part, though I > haven't followed through on the [implementation][2]. In particular, as > noted in this thread, you can't bootstrap off of either the current > version of Rust or the previous. This is actually a problem that could > be fixed in about one release cycle. There's little excuse not to do it > yesterday, though some on the team have lingering worry about having > to wait 6 weeks to complete unstable language changes. > > We have less understanding of what it will take to get the major > distros to a) actually deploy rust in a stable release, b) keep rust updated > every 6 weeks. I don't see b) happening. > > Do we have a plan being executed to make sure that when Gecko > components written in Rust ship on the release channel in > Mozilla-produced builds the also ship in the Firefox packages of > various Linux distributions? > > On the Rust side we have a plan to unblock the distros ability to > package Rust [1]. As far as getting Rust into distros, we continue to > go through cycles of finding blockers on the Rust side, fixing them, > and then discovering what else is blocking the distros Unfortunately, > after the initial push in the above threads, this has been on the > backburner. I intend to push on it again next quarter. > > > When is rustc expected to reach that level of bootstrapping predictability? > > It should be feasible to make this change soon, but we'll need to > discuss at the next core team meeting. I'll get back to you on that. > > > I think it's unacceptable to limit our ability to leverage Rust in > Gecko by forgoing the ability to co-evolve Rust and Gecko at a rapid > pace. > > Even after hurdles of getting stable rustc into distros is solved I > think this is one of the most difficult issues. In particular, Firefox > needs to be buildable on the *stable* Rust compiler in order for > distros to build it. Rust's nightly compiler contains unstable > features that we don't want used generally; it's the stable compiler > that we promote and want distros to package. If Firefox requires > nightly unstable features, then distros will be forced to package > nightly Rust. If those distros users learn to expect that the nightly > compiler is available then that could severely compromise Rust's > strategy for evolving the language. > > Note that there's a big diference between Firefox needing to > continually update to a bleeding edge Rust, and Firefox using > *unstable* Rust features. It's fine to update to a new Rust compiler > every few weeks, even Rust nightlies, as long as Firefox doesn't turn > on unstable features (which must be done explicitly). > > > I think we must not allow ourselves to wait for a Debian or Ubuntu LTS > cycle before Rust improvements can be used in Gecko. > > > I agree with Brian Smith's sentiment at > https://internals.rust-lang.org/t/perfecting-rust-packaging/2623/67 > > I agree that Rust being stuck on the distro LTS cycle is impracticle > for real software. > > If Rust produces 'universal' debs and rpms as suggested by @briansmith > that would be enough for distro *users* to build Firefox, but it's not > sufficient for the distro's themselves to keep their Firefox's up to > date. I think Rust should do this but it isn't clear that it solves a > critical problem. > > Firefox's release model also has this tension with distros, and even > LTS distros *do* update Firefox, right? Is there any chance Rust can > recieve updates like Firefox? LTS distros do update Firefox because there is no way they can support security updates on older releases (I've done it with 3.5 long enough to know it's not tractable). But they do it once a year (at every ESR bump), not every 6 weeks. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to switch to Visual Studio 2015
What about the depreciation of XP SP2? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to ship: navigator.hardwareConcurrency
On 3/16/16 2:39 PM, Domenic Denicola wrote: Now at https://html.spec.whatwg.org/#navigator.hardwareconcurrency (plus https://html.spec.whatwg.org/#navigator and https://html.spec.whatwg.org/#workernavigator). Thank you. To close the loop on this, given lack of objections I am going to go ahead and check this in. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: An analysis of content process memory overhead
On Wednesday, March 16, 2016 at 10:22:40 PM UTC+8, Nicholas Nethercote wrote: > Even if we can fix that, it's just a lot of JS code. We can lazily import > JSMs; I wonder if we are failing to do that as much as we could, i.e. are > all these modules really needed at start-up? It would be great if we > could instrument module-loading code in some way that answers this > question. B2G also did dropping JS source, for Tarako branch, since source is useless for loaded module save for stringify functions. (Gecko compress in-memory source.) But, I am not sure if it was landed on m-c then. > > - "heap-unclassified" memory is 4.5 MiB per process. I've analyzed this with > DMD and this is mostly GTK and glib memory that we can't measure in our > memory reporters. I haven't investigated closely to see if any of this > could > be avoided. > > - "heap-overhead" is 4 MiB per process. I've looked at this closely. > The numbers tend to be noisy. > > - "page-cache" is pages that jemalloc holds onto for fast recycling. It is > capped at 4 MiB per process and we can reduce that with a jemalloc > configuration, though this may make allocation slightly slower. > > - "bin-unused" is fragmentation in smaller allocations and very hard to > reduce. > > - "bookkeeping" is jemalloc's internal data structures and very hard to > reduce. > > - Then there's the not-so-long tail of things less than 1 MiB per process. > Some of these may be shrinkable with effort, or made shareable between > processes with effort. (E.g. I reduced xpti-working-set by 216 KiB per > process in bug 1249174, and I've heard that making it shared was > considered > for B2G but never implemented.) It's getting into diminishing returns, > though. xpti sharing was implemented for B2G. It would be easy to enable them on Linux and Mac, but I am not sure on Windows. I guess preference is worth to be shared too. atoms-table maybe! ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
A lot of problems with Firefox Sync...
Hi! I tested over the last months Firefox Sync... and because of some problems I did in the last days a lot more test with it and noted a lot of problems with it to report... Because it is really a lot, I will split it in multiple mails within this topic... Feedback to the problems would be nice to get here, because so I don't have to follow a lot of bugs and can re-test it, when something have changed and give a new feedback from my side... OK, but first why I did a lot of tests in the last days: I recognized that I have some problems with some extension in the new FF48.0a1 builds ... With "About Sessionstore" and the "Xmarks" Extensions ... both doesn't work with the new version anymore (for now)... So I tried to switch back to FF47.0a2... Then there was my first problem: If I use my main profile in Aurora, it updates FF immediately to the Nightly Version because the update channel is saved in the profile! Is this really necessary/right this way? I mean: Why not save the update channel in the FF installation (folder) ??? Saving the update channel in the profiles makes it impossible to use them with a older/other version of FF! If somebody do it anyway (by mistake) it f*cks up his installation! I don't know what happened to the new profile manager... (Why was he removed/deactivated again ???) ...but with a new & better profile manager and the update channel saved in the installation of FF - not the profiles - it would be e.g. possible that a person can save his FF profile on a flash drive and use it on any other computer! A option like "Load a not listed profile" in the profile manager to open a profile folder e.g. on a flash drive, should make this (in future) possible... OK, but this was (just) the reason to test the FF Sync a lot more... A other thing I recognize since a long time: Using FF Sync (with maybe a lot of bookmarks) slows really the performance of FF down when I try to manage the bookmarks! I have a lot of bookmarks (>20k) since I started to use the bookmarks again and use FF Sync as a kind of backup... ... when I now create, delete, rename, or move a bookmark or a bookmark folder, FF uses 100% of my first CPU for some seconds and "blocks" the use of the bookmark manager for this time... This means, that when I do a lot of operations in the bookmarks manager like resort my bookmarks, I have to wait after every operations for some seconds (1-3sec.). The "funny" thing is, that going into offline mode with FF, making the management of my bookmarks, and then go online again, saves me a lot of time! Because I have no delay between the ops, my CPU don't go up to 100% and this "problems" come only ones, when I go back online again... Some more problems with this mail: - After sync to a other profile, a lot of my bookmarks are now in "Unsorted Bookmarks"! But, not the new ones I make! But a lot of the old ones, but not all! Maybe some of my bookmarks haven't (because of old builds) all information in the database and snyc isn't able without this information to sync the position (folders, sub-folders) where the bookmarks should be ??? Or a problem because of my huge amount of bookmarks ??? But why are the new created bookmarks get synced with the (folder) position ??? - Addon Sync seems to doesn't sync extensions in special categories like dictionaries and language packs! - Extensions not installed from AMO doesn't get synced/installed! - The toolbars (show/not show) and the icons (and possition of them) got not synced between the profiles! Plz give me some feedback to this first problems and I will write a lot more test results later. Thx in advance! Greets, Tobias. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: The convention to pass a parameter when ownership transfer is optional
Conceptually already_AddRefed is a move reference. And it's destructor asserts (in debug builds) that the reference was indeed moved, so it cannot be used as a conditional transfer under any circumstances. - Kyle On Thu, Mar 17, 2016 at 8:07 PM, wrote: > > https://hg.mozilla.org/mozilla-central/file/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/mfbt/UniquePtr.h#l182 > > "To unconditionally transfer ownership of a UniquePtr into a method, use a > |UniquePtr| argument. To conditionally transfer ownership of a resource > into a method, should the method want it, use a |UniquePtr&&| argument." > > Does that also apply to already_AddRefed<>&& or we stick to > https://bugzilla.mozilla.org/show_bug.cgi?id=1247972#c19? > > Btw, we have some code like > https://hg.mozilla.org/mozilla-central/file/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/dom/base/WebSocket.cpp#l2790 > where it really should be just already_AddRefed<> since the ownership is > always transferred. > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On 03/17/2016 01:47 PM, Sylvestre Ledru wrote: [...] One way which would make the life of Linux distro way easier would be to maintain the Firefox rust code in a way it could compile using older rust compiler. For example, imagine that the next Debian stable or Ubuntu LTS ships using Rust 2.0, we would make sure that the new rust code still compile with Rust 2.0 (limiting our capability to use new language trick). Just like we do with C++... Now, with my Debian/Ubuntu hat, maintaining rust backports to be able to build new versions of Firefox on stable/LTS releases is not going to be easy: rust needs recent versions of LLVM, LLVM is a complex beast (with regular bump the minimal version of gcc, etc). And here, I am just mentioning i386 & amd64... (Debian/ubuntu don't have any rust builds for arm CPU). Well, what about other arches? ppc(64), s390, ai64...? Sylvestre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: An analysis of content process memory overhead
On 15/03/2016 04:34, Nicholas Nethercote wrote: > - "heap-overhead" is 4 MiB per process. I've looked at this closely. > The numbers tend to be noisy. > > - "page-cache" is pages that jemalloc holds onto for fast recycling. It is > capped at 4 MiB per process and we can reduce that with a jemalloc > configuration, though this may make allocation slightly slower. We aggressively got rid of that on B2G by sending memory-pressure events to apps that were unused. We did have the advantage there that we had only one page per process so establishing if one was not being used was very easy. On desktop Firefox we might consider to try and minimize the memory usage of processes which do not have active tabs (e.g. none of the tabs is visible, or none of the tabs has received input for a while). Besides the immediate memory usage reduction this had the important side-effect of reducing steady-state consumption. A lot of the structures and caches that were purged had often been bloated by transient data required only during startup. Once minimized they would start to grow again once a process would become active again but never as much as before the minimization. Gabriele signature.asc Description: OpenPGP digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
Le 17/03/2016 à 21:30, Henri Sivonen a écrit : > On Thu, Mar 17, 2016 at 2:28 PM, Sylvestre Ledru wrote: >> In Debian & Ubuntu, > I didn't find a rustc package on http://packages.ubuntu.com/ . What's > the situation on Ubuntu? I thought it was already synced for xenial. I just filled the request: https://bugs.launchpad.net/bugs/1559001 > >> we use the official binaries provided to be able to build rust. > My same logic, can Mozilla-built rustc be used to build the Firefox > package as far as policy matters, with exceptions and waivers taken > into account, go? Or is this what's keeping rustc not getting past > testing? Debian stable will use the version of rustc at the time of the Debian freeze (January 2017) > > > On Thu, Mar 17, 2016 at 2:47 PM, Sylvestre Ledru wrote: >> One way which would make the life of Linux distro way easier would be >> to maintain the Firefox rust code in a way it could compile using older rust >> compiler. > In order to be competitive, we need all leverage we can get from our > Mozilla magic sauce (Rust). I think it's unacceptable to limit our > ability to leverage Rust in Gecko by forgoing the ability to co-evolve > Rust and Gecko at a rapid pace. I understand your point, it is just conflicting with the goals of Debian and Ubuntu LTS. Mozilla wants to move fast, distros wants to provide stable products... > >> Now, with my Debian/Ubuntu hat, maintaining rust backports to be able to >> build new versions of Firefox on stable/LTS releases >> is not going to be easy > Could Firefox in Debian stable have a build dependency on rustc from > Debian unstable? Nope, this doesn't work this way. Packages must be built in the env they are going to be used. Otherwise, it would lead to side effects (different version of libraries used for build time and runtimes). One dirty solution would be to ship rustc+llvm sources into Firefox sources when targeting stable distros. But this increases the load on the maintainers by an order of magnitude (maintainers will have to manage rust & llvm) and might not work if llvm starts requiring a more recent of gcc to build. However, this is really the last option distros will consider (and I am sure Glandium will choke when he will read this). Sylvestre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On 17/03/2016 11:59, Henri Sivonen wrote: > (I'm assuming that it won't be a big deal to get > Gentoo to pick up the most recent rustc when the use case shows up.) Gentoo has an official portage overlay for rust with the latest stuff available: https://github.com/gentoo/gentoo-rust It should be trivial for them to move the packages from there into the mainline portage if firefox needs them as a dependency. Also keep in mind that by default only Firefox ESR is flagged as stable on Gentoo. Recent versions of Firefox are always available but the user needs to explicitly unmask them. Gabriele signature.asc Description: OpenPGP digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
Le 17/03/2016 à 13:22, Henri Sivonen a écrit : > On Thu, Mar 17, 2016 at 1:58 PM, Martin Stransky wrote: >> Is it possible to build Rust from sources before Firefox build executes? > rustc is written in Rust, so if you only have source code and the > compilers that are currently available in Fedora, you can't build > rustc. At minimum, you need a "stage0" compiler, which is an archived > older rustc binary. (rustc originally bootstrapped with OCaml, but > building the whole history of Rust from the OCaml days to present > every time Fedora builds Firefox seems excessively impractical.) > > Fedora's policy seems to prohibit continuous shipping of stage0 as a > Mozilla-provided binary, so it seems to me that either Fedora needs to > maintain rustc bootstrapped according to policy (bootstrapping using > Mozilla-provided stage0 once) or a policy waiver for Firefox is > needed. In Debian & Ubuntu, we use the official binaries provided to be able to build rust. Otherwise, it would be way too hard for a small gain (except to bootstrap other archs) However, in the future, I have been told that we would be able to build rust using rust version-1. Addressing this the core issue you mentioned (you need a special permission from the distro to get the initial package). Sylvestre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On Thursday, 17 March 2016 12:23:32 UTC, Henri Sivonen wrote: > On Thu, Mar 17, 2016 at 1:58 PM, Martin Stransky wrote: > > Is it possible to build Rust from sources before Firefox build executes? > > rustc is written in Rust, so if you only have source code and the > compilers that are currently available in Fedora, you can't build > rustc. At minimum, you need a "stage0" compiler, which is an archived > older rustc binary. (rustc originally bootstrapped with OCaml, but > building the whole history of Rust from the OCaml days to present > every time Fedora builds Firefox seems excessively impractical.) Out of interest, would that actually involve building every single Linux snapshot from https://github.com/rust-lang/rust/blob/master/src/snapshots.txt in sequence? All 319 of them? Thanks Liam Wilson ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On Fri, Mar 18, 2016 at 12:05 PM, wrote: > For Firefox this would require a lot of discipline. For Rust it would be a > massive challenge coming up with a maintainable scheme. Frankly, I think the probability that either the Gecko or Rust teams would commit significant portions of their limited engineering resources to this issue is basically nil. This is a policy problem. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On Fri, Mar 18, 2016 at 1:10 AM, Nicolas B. Pierron wrote: > This is not better from the point of view of distributions policy. > > This is better because you remove a lot of unknown variables from the > equation, and rely on a real package manager to package and distribute > software with its dependencies. This seems off-topic to me. The matter at hand is dealing with distros' (particularly Debian's and Fedora's) policies in such a manner that there is no disruption to Firefox reaching users via these channels while Mozilla leverages Mozilla-developed tech in Firefox at the pace that makes sense for Mac, Windows and Mozilla's own Linux builds. Building a distro's Firefox binary package with a package manager that's not the distro's native one is very far off policy compliance. On Fri, Mar 18, 2016 at 1:59 AM, wrote: > In particular, as > noted in this thread, you can't bootstrap off of either the current > version of Rust or the previous. This is actually a problem that could > be fixed in about one release cycle. Nice! I didn't realize it's that close to happening. I think it would be good to make "rustc stable N compiles with rustc stable N or rustc stable N-1" happen, then. It would probably make life easier not just for distro packaging but for non-x86/x86_64 ports as well, since those could self-host in a predictable way. > Even after hurdles of getting stable rustc into distros is solved I > think this is one of the most difficult issues. In particular, Firefox > needs to be buildable on the *stable* Rust compiler in order for > distros to build it. I've been assuming that Firefox would use the Rust "stable" channel compiler on the Firefox "release" channel, but I don't know if that has ever been really *decided* anywhere. Has it been decided? > Rust's nightly compiler contains unstable > features that we don't want used generally; it's the stable compiler > that we promote and want distros to package. If Firefox requires > nightly unstable features, then distros will be forced to package > nightly Rust. If those distros users learn to expect that the nightly > compiler is available then that could severely compromise Rust's > strategy for evolving the language. I think the best way to make sure that Firefox "release" channel builds with Rust "stable" channel rustc is to make sure that the theoretical notion of stability as the Rust project applies it and practical de facto stability do indeed go pretty closely together. An example of this *not* being the case: I expect to have to import https://github.com/gz/rust-cpuid into Gecko in order to cater to the Mozilla-side policy sadness of having to support Windows XP users whose computers don't have SSE2. That crate requires the asm! macro, which, according to https://internals.rust-lang.org/t/stabilization-path-for-asm/2610 , has been "barely updated" since Rust 0.6 and no one seems to have a concrete plan to redesign the feature, either. So the feature is de facto stable, but it's not theoreticall stable and continues to be unavailable in the "stable" channel. > If Rust produces 'universal' debs and rpms as suggested by @briansmith > that would be enough for distro *users* to build Firefox, but it's not > sufficient for the distro's themselves to keep their Firefox's up to > date. I think Rust should do this but it isn't clear that it solves a > critical problem. Indeed. I meant I agreed with this part of what Brian Smith said: "Debian Stable and Red Hat are notorious for maintaining old versions of packages way longer than anybody wants to support. With the great amount of improvement to rustc and Cargo, I think we're actually 1 or 2 years away from being able to expect any Rust library or application author to support any version of Rust older than the latest stable release. I personally don't want to be bothered by Linux distros asking me for free help in backporting changes or adding compatibility hacks to support old versions of rustc and Cargo that they ship. I imagine other people will feel similar." > Firefox's release model also has this tension with distros, and even > LTS distros *do* update Firefox, right? Ubuntu LTS updates Firefox at the Mozilla every-six-weeks schedule. >From Web sources, Fedora seems to, too, but I haven't actually verified this empirically. > Is there any chance Rust can > recieve updates like Firefox? >From my perspective, the best-case outcome of this thread would be exactly >that. On Fri, Mar 18, 2016 at 3:08 AM, Mike Hommey wrote: >> We have less understanding of what it will take to get the major >> distros to a) actually deploy rust in a stable release, b) keep rust updated >> every 6 weeks. > > I don't see b) happening. Why not if 1) Firefox has to update every six weeks to get security updates and 2) rustc will become a build dependency for Firefox? That is, why wouldn't whatever policy exception allows the Firefox package pull a new upstream release every six weeks not allow rustc *as a
Re: Linux distro readiness for Rust in Gecko
On 03/17/2016 10:59 AM, Henri Sivonen wrote: My concern is that Linux distributions tend to have a policy that binary packages have to be buildable from source packages using the tools that are available in the package repository of the distro. Considering the current state of rustc availability in distro repositories, it seems to me that we are headed towards a problem. I guess one of the thing we could do is use an alternative solution, such as an external package manager which can work side-by-side with the host, such as zero-install, Portage, or Nix. I already use Nix, as a package manager to manage the dependencies I need for building Firefox [1]. One of the advantage I see with this solution is that such Nix expression can be part of the repository [2] and be a fallback for all the platforms where we have no support. [1] https://github.com/nbp/firefox-build-env [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1115107 -- Nicolas B. Pierron ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
RE: Intent to switch to Visual Studio 2015
> On Thu, Mar 17, 2016 at 3:30 PM, > mailto:yuhongbao_...@hotmail.com>> wrote: > What about the depreciation of XP SP2? > > Results from bug 1124017 say XP SP2 still works on binaries built with > VS2015u1. This may not always hold true. So we will need to have the XP > SP2 discussion at some point. I defer to bsmedberg to start that > discussion. I was originally thinking of dropping XP SP2 at the same time as moving to VS2015 this year. Yuhong Bao ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: An analysis of content process memory overhead
I seem to remember that our ChromeWorkers (SessionWorker, PageThumbsWorker, OS.File Worker) were pretty memory-hungry, but I don't see any workers there. Does this mean that they have negligible overhead or that they are only in the parent process? Cheers, David On 15/03/16 04:34, Nicholas Nethercote wrote: > Greetings, > > erahm recently wrote a nice blog post with measurements showing the > overhead of > enabling multiple content processes: > > http://www.erahm.org/2016/02/11/memory-usage-of-firefox-with-e10s-enabled/ > > The overhead is high -- 8 content processes *doubles* our physical memory > usage -- which limits the possibility of increasing the number of content > processes beyond a small number. Now I've done some follow-up > measurements to find out what is causing the per-content-process overhead. > > I did this by measuring memory usage with four trivial web pages open, first > with a single content process, then with four content processes, and then > getting the diff between content processes of the two. (about:memory's diff > algorithm normalizes PIDs in memory reports as "NNN" so multiple content > processes naturally get collapsed together, which in this case is exactly > what > we want.) I call this the "small processes" measurement. > > If we divide the memory usage increase by 3 (the increase in the number of > content processes) we get a rough measure of the minimum per-content process > overhead. > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
> I've been assuming that Firefox would use the Rust "stable" channel > compiler on the Firefox "release" channel, but I don't know if that > has ever been really *decided* anywhere. Has it been decided? Your assumption is correct. For gecko integration related work we use the stable version of the language. However, this is explicitly not a policy of Servo, where we make use of all the new features we can get our hands on. These are not in conflict today*, but may someday be. We'll handle that issue if we get there. jack. * rust-url can actually compile either way for example. We often handle this by supporting both through the use of Cargo features (amounting to conditional compilation). ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: An analysis of content process memory overhead
On Thu, Mar 17, 2016 at 9:50 AM, Nicolas B. Pierron < nicolas.b.pier...@mozilla.com> wrote: > Source compressions should already be enabled. I think we do not do it > for small sources, and for Huge sources, as the compression would either be > useless, or it would take a noticeable amount of time. > I think Luke suggested that we could compress larger JS sources off the main thread if we implemented this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1001231 Its been in my queue for 2 years, unfortunately. If anyone wants to make that happen, please feel free to steal it. :-) Ben ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Include pdb files for firefox-sdk, and remove redudnt firefox-sdk/sdk/lib folder.
The firefox-sdk/sdk/lib is duplicate of firefox-sdk/lib That's was not necessary. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: The convention to pass a parameter when ownership transfer is optional
Just piling on here - I think that the conditional-move idiom is super confusing and we should forbid it. If some specific class needs such an idiom, they can build a special container that can be passed by lvalue-reference. On Thu, Mar 17, 2016 at 11:14 PM, Xidorn Quan wrote: > On Fri, Mar 18, 2016 at 11:07 AM, wrote: > > > > > > https://hg.mozilla.org/mozilla-central/file/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/mfbt/UniquePtr.h#l182 > > > > "To unconditionally transfer ownership of a UniquePtr into a method, use > a > > |UniquePtr| argument. To conditionally transfer ownership of a resource > > into a method, should the method want it, use a |UniquePtr&&| argument." > > > > Does that also apply to already_AddRefed<>&& or we stick to > > https://bugzilla.mozilla.org/show_bug.cgi?id=1247972#c19? > > > > Btw, we have some code like > > > https://hg.mozilla.org/mozilla-central/file/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/dom/base/WebSocket.cpp#l2790 > > where it really should be just already_AddRefed<> since the ownership is > > always transferred. > > > > I used already_AddRefed<>&& (and even UniquePtr<>&&) for parameters for > unconditional transfer for quite a while. But I'm recently convinced that > we should use already_AddRefed<> and UniquePtr<> in almost all cases, > because compilers actually have more chance to optimize code with them > rather than &&. > > I believe conditional move is rare, and could be confusing in most cases, > and you may actually want to use T& rather than T&& for conditional move. > The only use of && is for move-constructor and functions we want people to > reconsider when they try to pass in lvalue. > > - Xidorn > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
Hi, This is a matter that concerns the Rust team a lot. From the Rust side we want to make it possible for Debian and Fedora (as a proxy for all distros) to be able to bootstrap off of their own rustc's and keep up with the Rust release process. We have a [good idea][1] about how to accomplish that part, though I haven't followed through on the [implementation][2]. In particular, as noted in this thread, you can't bootstrap off of either the current version of Rust or the previous. This is actually a problem that could be fixed in about one release cycle. There's little excuse not to do it yesterday, though some on the team have lingering worry about having to wait 6 weeks to complete unstable language changes. We have less understanding of what it will take to get the major distros to a) actually deploy rust in a stable release, b) keep rust updated every 6 weeks. > Do we have a plan being executed to make sure that when Gecko components written in Rust ship on the release channel in Mozilla-produced builds the also ship in the Firefox packages of various Linux distributions? On the Rust side we have a plan to unblock the distros ability to package Rust [1]. As far as getting Rust into distros, we continue to go through cycles of finding blockers on the Rust side, fixing them, and then discovering what else is blocking the distros Unfortunately, after the initial push in the above threads, this has been on the backburner. I intend to push on it again next quarter. > When is rustc expected to reach that level of bootstrapping predictability? It should be feasible to make this change soon, but we'll need to discuss at the next core team meeting. I'll get back to you on that. > I think it's unacceptable to limit our ability to leverage Rust in Gecko by forgoing the ability to co-evolve Rust and Gecko at a rapid pace. Even after hurdles of getting stable rustc into distros is solved I think this is one of the most difficult issues. In particular, Firefox needs to be buildable on the *stable* Rust compiler in order for distros to build it. Rust's nightly compiler contains unstable features that we don't want used generally; it's the stable compiler that we promote and want distros to package. If Firefox requires nightly unstable features, then distros will be forced to package nightly Rust. If those distros users learn to expect that the nightly compiler is available then that could severely compromise Rust's strategy for evolving the language. Note that there's a big diference between Firefox needing to continually update to a bleeding edge Rust, and Firefox using *unstable* Rust features. It's fine to update to a new Rust compiler every few weeks, even Rust nightlies, as long as Firefox doesn't turn on unstable features (which must be done explicitly). > I think we must not allow ourselves to wait for a Debian or Ubuntu LTS cycle before Rust improvements can be used in Gecko. > I agree with Brian Smith's sentiment at https://internals.rust-lang.org/t/perfecting-rust-packaging/2623/67 I agree that Rust being stuck on the distro LTS cycle is impracticle for real software. If Rust produces 'universal' debs and rpms as suggested by @briansmith that would be enough for distro *users* to build Firefox, but it's not sufficient for the distro's themselves to keep their Firefox's up to date. I think Rust should do this but it isn't clear that it solves a critical problem. Firefox's release model also has this tension with distros, and even LTS distros *do* update Firefox, right? Is there any chance Rust can recieve updates like Firefox? Again [this thread][1] contains the most up to date status. [1]: https://internals.rust-lang.org/t/perfecting-rust-packaging-the-plan/2767 [2]: https://internals.rust-lang.org/t/perfecting-rust-packaging-the-plan/2767/41 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On 03/17/2016 08:30 PM, Henri Sivonen wrote: On Thu, Mar 17, 2016 at 4:11 PM, Nicolas B. Pierron wrote: I guess one of the thing we could do is use an alternative solution, such as an external package manager which can work side-by-side with the host, such as zero-install, Portage, or Nix. I don't see how that would be any better from the distro policy perspective than using Mozilla-shipped rustc. This is not better from the point of view of distributions policy. This is better because you remove a lot of unknown variables from the equation, and rely on a real package manager to package and distribute software with its dependencies. If you think that doing it manually is better, I will ask you the following questions: - What libraries of the system do you depend on, which version? - What versions of python, cargo, …? - Is sh implemented as dash or bash ? - Do you need a custom version for Linux(es), Mac, BSD? - … - Do you have deterministic builds of rustc? A portable package manager should be able to hide such variability for you, and reduce the burden of packaging. Also note that these package managers can be installed in user-land, which is convenient if you have no root access on your computer. -- Nicolas B. Pierron ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: An analysis of content process memory overhead
On Fri, Mar 18, 2016 at 2:29 AM, David Rajchenbach-Teller < dtel...@mozilla.com> wrote: > > I seem to remember that our ChromeWorkers (SessionWorker, > PageThumbsWorker, OS.File Worker) were pretty memory-hungry, but I don't > see any workers there. Does this mean that they have negligible overhead > or that they are only in the parent process? I checked the data again: they are only in the parent process, so they don't affect content process scaling. And they're not *that* big -- here's the biggest I saw in my data (from the Mac "large processes" data): > 6.33 MB (04.00%) -- workers/workers(chrome) > ├──2.15 MB (01.36%) ++ worker(resource://gre/modules/osfile/osfile_async_worker.js, 0x113881800) > ├──2.11 MB (01.33%) ++ worker(resource:///modules/sessionstore/SessionWorker.js, 0x1297e7800) > └──2.06 MB (01.30%) ++ worker(resource://gre/modules/PageThumbsWorker.js, 0x1169c1000) Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Linux distro readiness for Rust in Gecko
On 03/17/2016 11:59 AM, Henri Sivonen wrote: Do we have a plan being executed to make sure that when Gecko components written in Rust ship on the release channel in Mozilla-produced builds the also ship in the Firefox packages of various Linux distributions? If not, we really need to put such a plan in place, since it would be bad to have a conflict between our ability to reach Linux users and our ability to benefit from Rust quickly. My concern is that Linux distributions tend to have a policy that binary packages have to be buildable from source packages using the tools that are available in the package repository of the distro. Considering the current state of rustc availability in distro repositories, it seems to me that we are headed towards a problem. As far as I can tell, of the well-known distributions only Arch and openSUSE have the current rustc in their package repositories (and the openSUSE packages are categorized as "unstable"). Debian has it in unstable and testing only. Gentoo is a couple of rustc releases out of date. Ubuntu and Fedora don't have any version of rustc anywhere at all. At least in the case of Fedora, just shipping a rustc binary alongside the Firefox source is explicitly not good enough for their policy: https://fedoraproject.org/wiki/Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries Yes, that's not possible in Fedora - we have similar problem with pre-built signed extensions from AMO. Is it possible to build Rust from sources before Firefox build executes? That may be a solution for now and it ensures that the Rust is always up to date. ma. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: An analysis of content process memory overhead
On 3/17/16 9:50 AM, Nicolas B. Pierron wrote: Note, this worked on B2G, but this would not work for Gecko. For example all tabs addons have to use toSource to patch the JS functions. Note that we do have the capability to lazily load the source from disk when someone does this, and we do use it in Gecko for some things. We could use it for more things -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to ship: navigator.hardwareConcurrency
On Tuesday, March 15, 2016 at 2:08:50 AM UTC-4, Boris Zbarsky wrote: > On 3/15/16 12:59 AM, Domenic Denicola wrote: > > Filed https://github.com/whatwg/html/issues/879; should be able to have it > > in the spec by sometime tomorrow. > > Thank you. That's a lot more useful than me snarking, for sure. Now at https://html.spec.whatwg.org/#navigator.hardwareconcurrency (plus https://html.spec.whatwg.org/#navigator and https://html.spec.whatwg.org/#workernavigator). ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
