Quantum Flow Engineering Newsletter #2

[Ehsan Akhgari]

Hi everyone,

This past week was another busy week chasing down performance issues in
Firefox.  We managed to knock out a few issues, get closer to close out a
couple of really high impact ones, and are making good progress on starting
to get performance data from telemetry which will hopefully allow us to
prioritize our efforts in a systematic way in order to focus on issues that
hurt our users the most in the wild first.

Another nice aspect that we are starting to get some traction on is scaling
up the engineering side of the project.  Jean Gong has started to help out
with the project management side of things, and we have started to triage
the list of bugs that we have, with the goal of identifying our highest
priority bugs to ensure that they all have assignees and are being worked
on and won’t fall into the cracks.  We appreciate your help if someone
approaches you asking for help with fixing, code reviews, or answering a
question about one of these bugs!

There is a work week for Quantum Flow on the week of March 27 here in
Toronto.  We’re preparing to meet face to face for the second time for this
project.  One of the things that I’m trying to have ready in time for this
work week is telemetry data about where Firefox is performing really badly
in the wild so that we can focus there first.  Right now we have Background
Hang Reports data that can collect a backtrace of hung threads in two
modes: if the thread is hung for more than 128ms, a backtrace using Gecko
Profiler pseudo stacks is captured, and if a thread is hung for more than 8
seconds, a backtrace using the full native stack is captured.  The pseudo
stack backtrace doesn’t include a lot of information, the backtrace only
consists of the manual annotations that we have added to the source code
using PROFILER_LABEL annotations.  I have already skimmed over the former
set of data and it’s really hard to gather much meaningful information from
this data.  The native stack traces would be much more useful, but while 8
seconds of a thread being hung is really bad, that’s more of a hang
scenario than a badly performing browser, so we’re trying to reduce this
threshold in bug 1346415 to gather better data here.  I hope to have some
more information to share about this next week.


Now, time for the performance story of this week, page navigations!  As web
browser makers, we talk about page load times a lot, and we all have heard
of what usually gets talked about in this context many times.  I’m going to
talk about what usually doesn’t get talked about though: what can happen in
the real life when you navigate from page A to B.  Firstly, with multiple
content processes, we may need to start a new content process for the
navigation.  Right now when a content process starts up, it sends a number
of synchronous IPC messages to the parent processes in order to initialize
various components (although we have removed all except for the last few
remaining ones.)  This is especially bad since at this time the parent
process is typically busy doing other work.  For example, since the kind of
navigation that results in a process switch typically happens in a new
tab/window, the parent process is typically busy opening a new tab/window,
and because of that, in really bad cases I have seen these synchronous
message take an overall time of over a second of the content process just
being paused doing no work whatsoever.  This can slow down navigations
significantly.  There is also a synchronous IPC that is on the path of all
navigations (bug 1337064) where we run this risk on all navigations.  We
also do some synchronous IPCs if the navigation results in an error page
under some situations, which is of less concern since those are less common
(well, one would hope at least.)

Fixing each one of these doesn’t mean that navigations suddenly become
faster of course, the logic works more against us than in our favour: not
fixing them means that we will always run the risk of page navigations
being slow in Firefox due to unpredictable factors.  What’s really worrying
is that in general it’s really hard to know what performance cliffs like
these are going to be on the path of any critical user interaction, and
these issues have a way of creeping in over time.  This is why a while ago
we decided to disallow the addition of new synchronous IPC messages by
default (bug 1336919) to avoid programmers adding more issues of this
nature to the code base.  We may still decide to add a few more of these
messages here and there, but only after really careful consideration and
measurement.  Like most other things in engineering, this requires careful
thought and balancing, but it’s good to have default practices that don’t
result in potentially disastrous performance cliffs.  Next week, I’m going
to give you another example of one of these cliffs showing how through an
unintended consequence of matters, code that was trying to avoid doing
main-thread I/O was ending up blocking 

Re: Tracking bug for removals after XPCOM extensions are no more?

[Kris Maglione]

On Wed, Mar 15, 2017 at 11:35:10PM +0200, Henri Sivonen wrote:

What's the current outlook on letting chrome JS read ArrayBuffers as
opposed to JS strings where the high 8 bits are zero and the low 8
bits are the byte values from XPCOM streams? (Or letting chrome JS
access things that are currently exposed as XPCOM streams via some
other thing that exposes bytes as ArrayBuffers?)


This is already possible via nsIBinaryInputStream:

http://searchfox.org/mozilla-central/rev/571c1fd0ba0617f83175ccc06ed6f3eb0a1a8b92/xpcom/io/nsIBinaryInputStream.idl#71-82
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Tracking bug for removals after XPCOM extensions are no more?

[Bobby Holley]

On Thu, Mar 16, 2017 at 11:23 AM, R Kent James  wrote:

> On 3/15/2017 3:51 PM, Benjamin Smedberg wrote:
>
>> after Firefox 57 when
>> Webextensions are the only extensions, if our product no longer needs some
>> functionality (and it's "API"), let's remove it. Quickly and ruthlessly.
>> We
>> need to actively work to maintain less code.
>>
>
> I'm struggling to have a big picture overview of what is going to be
> removed. Let me state what I think the issue is, and see if I understand
> correctly.
>
> Certain APIs remain in the platform only because they were used by XUL
> addons. Once XUL addons are disabled in Firefox, those will be removed.
>
> The core XUL overlay feature however is used extensively throughout the
> platform, and is not going to be removed as a result of disabling XUL
> extensions (though it might be removed in the future as a part of
> eliminating XUL from the platform). Examples are the overlay features in
> XULDocument.cpp
>
> Is that reasonable correct?
>

This sounds about right. In practice, I think the goal is that a green run
on treeherder should be the primary criterion of what is landable, rather
than a bunch of downstream consumers whose breakage is only discovered when
we release.


>
> :rkent
>
>
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Tracking bug for removals after XPCOM extensions are no more?

[R Kent James]

On 3/15/2017 3:51 PM, Benjamin Smedberg wrote:

after Firefox 57 when
Webextensions are the only extensions, if our product no longer needs some
functionality (and it's "API"), let's remove it. Quickly and ruthlessly. We
need to actively work to maintain less code.


I'm struggling to have a big picture overview of what is going to be 
removed. Let me state what I think the issue is, and see if I understand 
correctly.


Certain APIs remain in the platform only because they were used by XUL 
addons. Once XUL addons are disabled in Firefox, those will be removed.


The core XUL overlay feature however is used extensively throughout the 
platform, and is not going to be removed as a result of disabling XUL 
extensions (though it might be removed in the future as a part of 
eliminating XUL from the platform). Examples are the overlay features in 
XULDocument.cpp


Is that reasonable correct?

:rkent


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Will script implementations of nsIOutputStream be prohibited once XPCOM extensions are no more?

[Jörg Knobloch]

On 16.03.17 09:08, Boris Zbarsky wrote:
We should try marking nsIOutputStream builtinclass on trunk right now. 
As far as I can tell, this should just work. 


No adverse effects on Thunderbird, I hope:

https://dxr.mozilla.org/comm-central/search?q=nsIOutputStream=false

As far as I can see we don't implement this in JS (which is what 
"builtinclass" would disable).


Jörg.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Tracking bug for removals after XPCOM extensions are no more?

[Boris Zbarsky]

On 3/13/17 11:36 AM, Steve Fink wrote:

There's a whitelist of "safe" things that we assume will never be
overridden. But now I can remove everything that treats nsISupports
specially.


Things can still be implemented in chrome JS via XPCWrappedJS, right?

Assuming the analysis will consider all known overrides of the relevant 
virtual method, we need to also restrict to cases in which the relevant 
class is builtinclass, I would think.


-Boris

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Will script implementations of nsIOutputStream be prohibited once XPCOM extensions are no more?

[Henri Sivonen]

Do we need to keep caring about
https://bugzilla.mozilla.org/show_bug.cgi?id=170416 once XPCOM
extensions are no more?

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to ship: IntersectionObserver API

[Patrick Brosset]

>
> 1)  Is there devtools support for this (e.g. to be able to see what
> intersection observers are registered where)?  If not, are there at least
> bugs tracking it?


I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1347849 for this.
MutationObserver would be a good one to add support for too!

On Thu, Mar 16, 2017 at 8:39 AM, Anne van Kesteren  wrote:

> On Wed, Mar 15, 2017 at 7:38 PM, Boris Zbarsky  wrote:
> > On 3/15/17 1:32 PM, Tobias Schneider wrote:
> >> 2.3) Platform tests are in the process of being upstreamed by Google (
> >> https://github.com/w3c/web-platform-tests/pull/4384).
> >
> > That seems to be in limbo for well over a month now.  jgraham just poked
> it
> > in hopes of at least getting automated testing going so we find out
> whether
> > Firefox passes the tests... but that will test release, afaik. Have you
> > checked whether we pass these tests?
>
> They would be run against Nightly.
>
>
> --
> https://annevankesteren.nl/
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Will script implementations of nsIOutputStream be prohibited once XPCOM extensions are no more?

[Boris Zbarsky]

On 3/16/17 3:35 AM, Henri Sivonen wrote:

Do we need to keep caring about
https://bugzilla.mozilla.org/show_bug.cgi?id=170416 once XPCOM
extensions are no more?


A first glance, no.

We should try marking nsIOutputStream builtinclass on trunk right now. 
As far as I can tell, this should just work.


-Boris

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to ship: IntersectionObserver API

[Anne van Kesteren]

On Wed, Mar 15, 2017 at 7:38 PM, Boris Zbarsky  wrote:
> On 3/15/17 1:32 PM, Tobias Schneider wrote:
>> 2.3) Platform tests are in the process of being upstreamed by Google (
>> https://github.com/w3c/web-platform-tests/pull/4384).
>
> That seems to be in limbo for well over a month now.  jgraham just poked it
> in hopes of at least getting automated testing going so we find out whether
> Firefox passes the tests... but that will test release, afaik. Have you
> checked whether we pass these tests?

They would be run against Nightly.


-- 
https://annevankesteren.nl/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Heads Up: /storage upgraded from version 1.0 to 2.0

[Benjamin Smedberg]

I apologize for the delays in bug 1246615. It fell off my radar with a
series of trips. I've set up a meeting tomorrow to at least identify who is
responsible for this decision, because it's not exactly me. I analyzed some
data in the bug which I'll repeat here.

Here is an analysis of the patterns of people downgrading:
https://gist.github.com/bsmedberg/1af70857106bfe29128a0e8d0b656804 - this
analysis was reviewed by chutten

users that switched channels at all: 0.48%
users that reverted to an older version: 2.55%
users that reverted to an older version, staying on the release channel:
2.19%

Downgrades are more prevalent than I thought, and channel-switching is not
a primary factor in downgrades. So the ideas that we would mostly solve
this problem by giving channels separate profiles appears to be false.

I still believe that we should do our best to make some set of downgrades
seamless for the user (if perhaps with some reversion to prior data or
duplicating). And that showing people a banner warning is user-hostile in
the sense that if they downgraded it was probably for a reason and so they
aren't going "back". What I'd like is some clearer guidance about how much
we should support downgraders and how much engineering effort we should put
into making downgrade as painless as possible.

--BDS


On Wed, Mar 8, 2017 at 9:40 AM, Ehsan Akhgari 
wrote:

> On 2017-03-07 8:02 PM, Ben Kelly wrote:
> > On Tue, Mar 7, 2017 at 6:09 PM, Xidorn Quan  wrote:
> >
> >>> This major version change is downgrade-incompatible, so IndexedDB and
> >>> DOM cache won't work in an older version if their profile has been
> >>> upgraded.
> >>> IndexedDB is also used internally, so stuff that depends on it likely
> >>> won't work too.
> >>> There's bug 1246615 [2] where you can find a discussion related to this
> >>> issue.
> >>
> >> It would probably be a good idea to backup the old files when upgrading,
> >> so that old version can at least pick the backup file to use.
> >>
> >
> > Maintaining downgrade compatibility is a lot more complex than people
> think
>
> Yes.
>
> > and we have zero test support for it.
>
> This part isn't entirely true.
>
> The full picture is a bit more complex.  Some components have supported
> full downgrades with automated tests running on the infrastructure for
> years.  Examples are the cookie manager and the permission manager.
> Other components have intentionally decided that it's OK to not maintain
> backwards compatibility, for example, IndexedDB.  In other places,
> people (myself included) have just not been careful enough and have
> (either intentionally or unintentionally) landed code that isn't
> downgrade compatible.
>
> It's certainly true that in general there is no tests that would catch
> you if you land code that breaks a downgrade scenario, and over the
> years it has become quite clear that maintaining a downgrade-compatible
> browser is impractical.
>
> I agree with Ben, Jan and others that this situation is unsustainable,
> and we have to do something about it.  I also agree with Xidorn about
> how bad it is that we risk destroying data from the very subset of our
> user population who go out of their way to help us by testing changes on
> newer builds and risk losing their data when going back to an older build.
>
> To be perfectly honest, the situation in bug 1246615 is quite
> frustrating.  I no longer understand what we are waiting for in that
> bug.  It seems like that bug is being scope creeped into a larger plan
> (based on comment 29 there) and still unclear what that exactly means,
> and here we are now: while Firefox 55 rides the trains, we *will*
> destroy our community's browsing data as they help us test Firefox.
>
> Benjamin, can we please address this with the urgency it deserves?
> Thank you for your attention.
>
> Cheers,
> Ehsan
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Editing vendored crates

[Simon Sapin]

On 27/02/17 19:47, Simon Sapin wrote:

On 27/02/17 19:30, Henri Sivonen wrote:

On Mon, Feb 27, 2017 at 8:00 PM, Bobby Holley  wrote:

FWIW, |cargo tree| is a very helpful tool to figure out who's pulling in a
crate.


Thanks, but what I'm trying to figure out isn't whose pulling it in
(the style crate is) but whether it is actually used beyond an
always-None Option in a way that would result in the data
tables actually getting included in the build as oppose to (hopefully)
getting discarded by LTO.

(Motivation: If we are taking on the data tables, I want to argue that
we should include encoding_rs instead even before the "replace uconv
with encoding_rs" step is done.)


As an aside, I have a plan to remove rust-encoding entirely from Servo
(including Stylo) and use encoding_rs instead. But doing this the way I
want to has a number of prerequisites, and I’d prefer to switch
everything at once to avoid having both in the build. At the moment I’m
prioritizing other Stylo-related work, but I’m confident it’ll happen
before we start shipping Stylo.


Henri, you’re correct that the parts of the style crate using 
rust-encoding are never used in Stylo. Gecko always passes UTF-8 input 
(after decoding on the C++ side I assume) to Stylo for parsing 
stylesheets, which we unsafely view as  with 
std::str::from_utf8_unchecked.


I’ve submitted https://github.com/servo/servo/pull/15993 to remove the 
rust-encoding dependency from stylo. This doesn’t need to wait on 
switching the rest of Servo.


--
Simon Sapin
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Quantum Flow Engineering Newsletter #2

[Ehsan Akhgari]

On Thu, Mar 16, 2017 at 11:07 AM, Ehsan Akhgari 
wrote:

>
> Now, on to the credits section.  I’d like to take a moment to recognize
> the work of the following individuals who have helped with various aspects
> of the Quantum Flow project.  Thank you very much for your help this past
> week!  (Apologies to those who I’m probably forgetting to name here.)
>
> * Kan-Ru Chen’s patches for bug 1194751 (moving PScreenManager off of sync
> IPC) are still under review.
> * Amy Chung submitted a first iteration patch for bug 1331680 (moving
> document.cookie off of sync IPC) for feedback.
> * Kearwood (kip) Gilbert has been helping with removing various sync IPC
> messages used in the WebVR implementation (bug 1344216 dependencies).
> * Boris Zbarsky made various input/textarea selection management APIs
> faster in many cases (bug 1343275, bug 1332036, bug 1342197).
> * Greg Tatum and Markus Stange’s improvements to https://perf-html.io/
> profiler UI significantly improve the responsiveness of the interface,
> making it much easier to look at profiles.
> * Nicholas Nethercote has been helping by fixing various threading, race
> and deadlock issues in the Gecko Profiler backend.
> * Michael Layzell has been helping with various telemetry data collection
> work.
> * Mike Conley has been teaching me how to use our telemetry analysis
> infrastructure.
>

I knew I was forgetting someone!  Better late than never:

* Miko Mynttinen made sorting of display lists according to the z-index of
the display items faster in bug 1343596
.

-- 
Ehsan
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Quantum Flow Engineering Newsletter #2

[Kris Maglione]

On Thu, Mar 16, 2017 at 11:07:19AM -0400, Ehsan Akhgari wrote:
This is why a while ago we decided to disallow the addition of new 
synchronous IPC messages by default (bug 1336919) to avoid programmers adding 
more issues of this nature to the code base.


Are there any plans to do something similar for sync message manager messages 
from JS? It seems like it would make sense to have a whitelist for those too, 
and require review from an IPC peer when making changes to it.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Switching to async/await from Task.jsm/yield

[Dave Townsend]

For a long time now we've been writing JS code that waits for promises
using Task.jsm and generator functions. Recently though the JS team added
support for the JS standard way of doing this, async/await:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

Writing code in standard JS is always better for the web, makes it easier
to onboard new engineers and allows for better support in developer tools.
So I'd like to propose that we switch to the standard way of writing these
functions immediately. New code should use async/await instead of Task.jsm
going forwards.

Florian has some rough plans to automatically rewrite existing usages of
Task.jsm to the standard JS forms so for now don't worry much about going
and submitting patches to fix up existing code. Once that is done we can
remove Task.jsm from the tree.

Does anyone object to any of this?
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

FYI: git-cinnabar and a "Intermittent IndexError: tuple index out of range" error

[Andrew McCreight]

If you are seeing this error when you try to do |git mozreview push|,
update your version-control-tools. Then you'll see a different error. Work
related to that is being looked at in
https://bugzilla.mozilla.org/show_bug.cgi?id=1338530

Andrew
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Switching to async/await from Task.jsm/yield

[Kris Maglione]

On Thu, Mar 16, 2017 at 03:29:15PM -0700, Dave Townsend wrote:

Writing code in standard JS is always better for the web, makes it easier
to onboard new engineers and allows for better support in developer tools.
So I'd like to propose that we switch to the standard way of writing these
functions immediately. New code should use async/await instead of Task.jsm
going forwards.


+1

I've already started doing this in places where using Task.jsm was 
unwieldy, and it's improved things tremendously.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Switching to async/await from Task.jsm/yield

[J. Ryan Stinnett]

For modules that have already converted, is there any performance change
(good or bad) between async / await vs. Task?

- Ryan

On Thu, Mar 16, 2017 at 5:33 PM, Kris Maglione 
wrote:

> On Thu, Mar 16, 2017 at 03:29:15PM -0700, Dave Townsend wrote:
>
>> Writing code in standard JS is always better for the web, makes it easier
>> to onboard new engineers and allows for better support in developer tools.
>> So I'd like to propose that we switch to the standard way of writing these
>> functions immediately. New code should use async/await instead of Task.jsm
>> going forwards.
>>
>
> +1
>
> I've already started doing this in places where using Task.jsm was
> unwieldy, and it's improved things tremendously.
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Switching to async/await from Task.jsm/yield

[Kris Maglione]

On Thu, Mar 16, 2017 at 05:39:06PM -0500, J. Ryan Stinnett wrote:

For modules that have already converted, is there any performance change
(good or bad) between async / await vs. Task?


I haven't noticed any differences either way, but I also haven't 
done any explicit profiling. There's definitely a difference in 
the way we collect async stacks in async/await code vs. with the 
Promise.jsm promises that Task.jsm uses, but that shouldn't show 
up much on release.



On Thu, Mar 16, 2017 at 5:33 PM, Kris Maglione 
wrote:


On Thu, Mar 16, 2017 at 03:29:15PM -0700, Dave Townsend wrote:


Writing code in standard JS is always better for the web, makes it easier
to onboard new engineers and allows for better support in developer tools.
So I'd like to propose that we switch to the standard way of writing these
functions immediately. New code should use async/await instead of Task.jsm
going forwards.



+1

I've already started doing this in places where using Task.jsm was
unwieldy, and it's improved things tremendously.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Quantum Flow Engineering Newsletter #2

[Ehsan Akhgari]

On Thu, Mar 16, 2017 at 4:27 PM, Kris Maglione 
wrote:

> On Thu, Mar 16, 2017 at 11:07:19AM -0400, Ehsan Akhgari wrote:
>
>> This is why a while ago we decided to disallow the addition of new
>> synchronous IPC messages by default (bug 1336919) to avoid programmers
>> adding more issues of this nature to the code base.
>>
>
> Are there any plans to do something similar for sync message manager
> messages from JS? It seems like it would make sense to have a whitelist for
> those too, and require review from an IPC peer when making changes to it.
>

Great question -- I recently found out that navigator.userAgent suffers
from a sync IPC issue in JS
 that was hidden in
our existing telemetry probes.  I filed bug 1348116
 for the whitelist
you suggested.  I also filed bug 1348113
 to get better
telemetry for sync IPC caused by JS.

-- 
Ehsan
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Heads Up: /storage upgraded from version 1.0 to 2.0

[Justin Dolske]

It would be useful to know if most of those users are just downgrading the
previous version (N-1 for release, N-7 for ESR), and how quickly they
return to a current release (assuming they do!).

This would help inform to what degree we need to support (and test!) the
myriad combinations of upgrade-downgrade-reupgrade combinations.

I also wonder how much of user-reversion is driven by addon incompatibility
(which will improve in Firefox 57 with WebExtensions), but I'm not sure how
we would answer that.

Justin

On Thu, Mar 16, 2017 at 12:15 PM, Benjamin Smedberg 
wrote:

> I apologize for the delays in bug 1246615. It fell off my radar with a
> series of trips. I've set up a meeting tomorrow to at least identify who is
> responsible for this decision, because it's not exactly me. I analyzed some
> data in the bug which I'll repeat here.
>
> Here is an analysis of the patterns of people downgrading:
> https://gist.github.com/bsmedberg/1af70857106bfe29128a0e8d0b656804 - this
> analysis was reviewed by chutten
>
> users that switched channels at all: 0.48%
> users that reverted to an older version: 2.55%
> users that reverted to an older version, staying on the release channel:
> 2.19%
>
> Downgrades are more prevalent than I thought, and channel-switching is not
> a primary factor in downgrades. So the ideas that we would mostly solve
> this problem by giving channels separate profiles appears to be false.
>
> I still believe that we should do our best to make some set of downgrades
> seamless for the user (if perhaps with some reversion to prior data or
> duplicating). And that showing people a banner warning is user-hostile in
> the sense that if they downgraded it was probably for a reason and so they
> aren't going "back". What I'd like is some clearer guidance about how much
> we should support downgraders and how much engineering effort we should put
> into making downgrade as painless as possible.
>
> --BDS
>
>
> On Wed, Mar 8, 2017 at 9:40 AM, Ehsan Akhgari 
> wrote:
>
> > On 2017-03-07 8:02 PM, Ben Kelly wrote:
> > > On Tue, Mar 7, 2017 at 6:09 PM, Xidorn Quan  wrote:
> > >
> > >>> This major version change is downgrade-incompatible, so IndexedDB and
> > >>> DOM cache won't work in an older version if their profile has been
> > >>> upgraded.
> > >>> IndexedDB is also used internally, so stuff that depends on it likely
> > >>> won't work too.
> > >>> There's bug 1246615 [2] where you can find a discussion related to
> this
> > >>> issue.
> > >>
> > >> It would probably be a good idea to backup the old files when
> upgrading,
> > >> so that old version can at least pick the backup file to use.
> > >>
> > >
> > > Maintaining downgrade compatibility is a lot more complex than people
> > think
> >
> > Yes.
> >
> > > and we have zero test support for it.
> >
> > This part isn't entirely true.
> >
> > The full picture is a bit more complex.  Some components have supported
> > full downgrades with automated tests running on the infrastructure for
> > years.  Examples are the cookie manager and the permission manager.
> > Other components have intentionally decided that it's OK to not maintain
> > backwards compatibility, for example, IndexedDB.  In other places,
> > people (myself included) have just not been careful enough and have
> > (either intentionally or unintentionally) landed code that isn't
> > downgrade compatible.
> >
> > It's certainly true that in general there is no tests that would catch
> > you if you land code that breaks a downgrade scenario, and over the
> > years it has become quite clear that maintaining a downgrade-compatible
> > browser is impractical.
> >
> > I agree with Ben, Jan and others that this situation is unsustainable,
> > and we have to do something about it.  I also agree with Xidorn about
> > how bad it is that we risk destroying data from the very subset of our
> > user population who go out of their way to help us by testing changes on
> > newer builds and risk losing their data when going back to an older
> build.
> >
> > To be perfectly honest, the situation in bug 1246615 is quite
> > frustrating.  I no longer understand what we are waiting for in that
> > bug.  It seems like that bug is being scope creeped into a larger plan
> > (based on comment 29 there) and still unclear what that exactly means,
> > and here we are now: while Firefox 55 rides the trains, we *will*
> > destroy our community's browsing data as they help us test Firefox.
> >
> > Benjamin, can we please address this with the urgency it deserves?
> > Thank you for your attention.
> >
> > Cheers,
> > Ehsan
> >
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Switching to async/await from Task.jsm/yield

[Kris Maglione]

On Thu, Mar 16, 2017 at 09:08:24PM -0400, Michael Layzell wrote:

Can we pass async functions everywhere where we use `add_task` in tests
already? If we can't, is there a tracking bug for converting all of the
test systems to support it?


Yes, add_task accepts any function which returns a promise, including async 
functions.



Can you `await` a function created with Task.async such as
`BrowserTestUtils.withNewTab` during the transition period?


Yes, you can await on any object with a Promise-like `then` method.


How about in ESR52? Will uplifting tests to ESR52 be a painful experience
due to having to use different tools to write the tests?


ESR52 supports async functions.


Can I pass an async function as the callback argument to things like
ContentTask.spawn and BrowserTestUtils.withNewTab? How about on ESR52?


Yes, they stringify in a way that's compatible with ContentTask.spawn, and 
withNewTab doesn't care what kind of function you pass it, so long as it 
returns a promise or generator.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Tracking bug for removals after XPCOM extensions are no more?

[Henri Sivonen]

On Thu, Mar 16, 2017 at 7:34 AM, Boris Zbarsky  wrote:
> On 3/15/17 5:35 PM, Henri Sivonen wrote:
>>
>> What's the current outlook on letting chrome JS read ArrayBuffers as
>> opposed to JS strings where the high 8 bits are zero and the low 8
>> bits are the byte values from XPCOM streams?
>
>
> I see no reason not to allow that.  We'd just add this on
> nsIScriptableInputStream, I assume, so we don't have to modify every single
> stream impl

OK. Thanks. Turns out this is already on file:
https://bugzilla.mozilla.org/show_bug.cgi?id=923017

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: windows build anti-virus exclusion list?

[Ben Kelly]

On Thu, Mar 16, 2017 at 11:26 PM, Ben Kelly  wrote:

>   - mozilla-build install dir
>   - visual studio install dir
>   - /users/bkelly/appdada/local/temp
>   - /users/bkelly (because temp dir was not enough)
>

FWIW, adding all these extra exclusions dropped my build time from ~22
minutes to ~14 minutes.

I'm hoping I can narrow my home directory exclusion down to things like
.bash_profile, .cargo, etc.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: windows build anti-virus exclusion list?

[Michael Hoye]

Depending on your AV, if you don't exempt mozilla-central some of our tests
will get quarantined and you won't be able to build at all.

- mhoye


On Mar 16, 2017 20:34, "Ben Kelly"  wrote:

On Thu, Mar 16, 2017 at 11:26 PM, Ben Kelly  wrote:

>   - mozilla-build install dir
>   - visual studio install dir
>   - /users/bkelly/appdada/local/temp
>   - /users/bkelly (because temp dir was not enough)
>

FWIW, adding all these extra exclusions dropped my build time from ~22
minutes to ~14 minutes.

I'm hoping I can narrow my home directory exclusion down to things like
.bash_profile, .cargo, etc.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

windows build anti-virus exclusion list?

[Ben Kelly]

Hi all,

I'm trying to configure my new windows build machine and noticed that
builds were still somewhat slow.  I did:

1) Put it in high performance power profile
2) Made sure my mozilla-central dir was not being indexed for search
3) Excluded my mozilla-central directory from windows defender

Watching the task monitor during a build, though, I still saw MsMpEng.exe
(antivirus) running during the build.

I ended up added some very broad exclusions to get this down close to
zero.  I am now excluding:

  - mozilla-central checkout
  - mozilla-build install dir
  - visual studio install dir
  - /users/bkelly/appdada/local/temp
  - /users/bkelly (because temp dir was not enough)

I'd like to narrow this down a bit.  Does anyone have a better list of
things to exclude from virus scanning for our build process?

Thanks.

Ben
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: windows build anti-virus exclusion list?

[David Major]

Try using Sysinternals Process Monitor to see what files MsMpEng.exe is
reading.

On Fri, Mar 17, 2017, at 04:26 PM, Ben Kelly wrote:
> Hi all,
> 
> I'm trying to configure my new windows build machine and noticed that
> builds were still somewhat slow.  I did:
> 
> 1) Put it in high performance power profile
> 2) Made sure my mozilla-central dir was not being indexed for search
> 3) Excluded my mozilla-central directory from windows defender
> 
> Watching the task monitor during a build, though, I still saw MsMpEng.exe
> (antivirus) running during the build.
> 
> I ended up added some very broad exclusions to get this down close to
> zero.  I am now excluding:
> 
>   - mozilla-central checkout
>   - mozilla-build install dir
>   - visual studio install dir
>   - /users/bkelly/appdada/local/temp
>   - /users/bkelly (because temp dir was not enough)
> 
> I'd like to narrow this down a bit.  Does anyone have a better list of
> things to exclude from virus scanning for our build process?
> 
> Thanks.
> 
> Ben
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: windows build anti-virus exclusion list?

[Ben Kelly]

On Thu, Mar 16, 2017 at 11:40 PM, Michael Hoye  wrote:

> Depending on your AV, if you don't exempt mozilla-central some of our
> tests will get quarantined and you won't be able to build at all.
>

I guess I was hoping someone familiar with our build might know the answer.
:-)

In any case, I managed to narrow my exclusions down to:

c:\devel
c:\mozilla-build
c:\program files\microsoft visual studio 12.0
c:\users\bkelly\.bash_profile
c:\users\bkelly\.cargo
c:\users\bkelly\.multirust
c:\users\bkelly\.rustup

And then in my .bash_profile I did this to avoid having to exclude all of
%TEMP%:

export TMP=/c/devel/.tmp
export TEMP=$TMP
export TMPDIR=$TMP

I couldn't figure out how to re-map /tmp in msys.  The fstab was not
working for me.

I guess if that all seems reasonable I can try to add some text to the wiki
suggesting it:

https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Windows_Prerequisites

Thanks.

Ben


>
> - mhoye
>
>
> On Mar 16, 2017 20:34, "Ben Kelly"  wrote:
>
> On Thu, Mar 16, 2017 at 11:26 PM, Ben Kelly  wrote:
>
> >   - mozilla-build install dir
> >   - visual studio install dir
> >   - /users/bkelly/appdada/local/temp
> >   - /users/bkelly (because temp dir was not enough)
> >
>
> FWIW, adding all these extra exclusions dropped my build time from ~22
> minutes to ~14 minutes.
>
> I'm hoping I can narrow my home directory exclusion down to things like
> .bash_profile, .cargo, etc.
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: windows build anti-virus exclusion list?

[Ben Kelly]

On Fri, Mar 17, 2017 at 12:05 AM, Ben Kelly  wrote:

> On Thu, Mar 16, 2017 at 11:40 PM, Michael Hoye  wrote:
>
> Depending on your AV, if you don't exempt mozilla-central some of our
>> tests will get quarantined and you won't be able to build at all.
>>
>
> I guess I was hoping someone familiar with our build might know the
> answer. :-)
>

Sorry.  I think I replied to the wrong person here.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: windows build anti-virus exclusion list?

[Mike Hoye]

On 2017-03-17 12:06 AM, Ben Kelly wrote:
On Fri, Mar 17, 2017 at 12:05 AM, Ben Kelly > wrote:


On Thu, Mar 16, 2017 at 11:40 PM, Michael Hoye > wrote:

Depending on your AV, if you don't exempt mozilla-central some
of our tests will get quarantined and you won't be able to
build at all.


I guess I was hoping someone familiar with our build might know
the answer. :-)


Sorry.  I think I replied to the wrong person here.


Hah! I thought "yeah, harsh but fair."

- mhoye
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform