Bulk Closing Intermittents

[Emma Humphries]

As discussed earlier, Joel and I have kicked off a process to close
intermittent test failures in Bugzilla.

If a test associated with a bug does not fail in 21 days, we'll close the
bug as RESOLVED:INCOMPLETE.

The first batch of intermittent bugs to close has 5,130 tickets. I have a
script to close these, but to close these without bug spam requires DBA
intervention.

I'd like to run the closures over the weekend but that's going to create a
non-trivial amount of bug spam for some of you.

There is a way to get rid of the bug spam!

Every bug we close will have the keyword `bulk-close-intermittents` added
to it.

If you search for messages from `bugzilla-dae...@mozilla.org` containing
`bulk-close-intermittents` you can isolate, review, and remove those
messages.

Thank you for your patience while we all work to get the noise out of
Bugzilla so we can find the strong signals on what we must focus on to
deliver Firefox 57 in November.

-- Emma
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Linting for common causes of oranges in mochitests (need ideas)

[Gijs Kruitbosch]

IME panels/popups/context-menus are flaky generally, and especially on 
Linux. I don't know how to meaningfully lint for this.


The other thing is various incantations that try to do variations on the 
"load x in a tab" theme. It turns out that it's not trivial to not race 
with an expectant page load, especially if you don't have full control 
over triggering it (ie the test is not the callsite of loadURI or 
equivalent) and they involve new tabs, and e10s is on as well. Worse if 
you also need to avoid hitting the network (see e.g. 
waitForDocLoadAndStopIt for a demonstration of this).


Finally, the most common description of orange I see is some variation 
of "test timed out". It would be helpful if we had more fluid rules 
about test durations, so that mostly-unrelated patches don't "suddenly" 
cause tests to cross the magical 45/90 second barrier (more often) and 
then get backed out.


~ Gijs

On 06/07/2017 19:47, Andrew Halberstadt wrote:
I'm looking into creating custom eslint rules that aim to avoid common 
causes of oranges in tests. We have an MDN page 
 
containing some of these already. Some of those patterns might be 
pretty hard to catch with a linter, but others look do-able. So far, 
I'm thinking of adding rules for:


  * No arbitrary setTimeouts
  * No synthesizeKey without waiting for focus
  * No deleting original tab
  * ???

These rules would only be configured to run on test files. I have a 
few questions:


 1. Would this cause too much of a burden for legitimate uses of those
patterns?
 2. Do any of the other patterns on that page look feasible to
implement as linters?
 3. Are there additional things not listed on that page that we could
lint for?

I'd love to find volunteer(s) who are more familiar with writing 
mochitests than myself (and who also care about this issue) to help 
brainstorm and provide more in-depth feedback. I want to strike a 
balance between preventing oranges and not over-burdening test 
authors, but I'm not sure what that balance looks like.


Thanks in advance for your input,
Andrew


___
firefox-dev mailing list
firefox-...@mozilla.org
https://mail.mozilla.org/listinfo/firefox-dev



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Linting for common causes of oranges in mochitests (need ideas)

[Andrew Halberstadt]

On Fri, Jul 7, 2017 at 12:15 PM, Ehsan Akhgari 
wrote:

> FWIW years ago I decided to act on trying to detect some of the patterns
> indicated on that page automatically, see https://bugzilla.mozilla.org/s
> how_bug.cgi?id=649012.  It didn't use linting (because we didn't really
> have any linting support back then) but the idea is the same, just with a
> different syntax.  That project never really received much (well, any!)
> traction since I always was hoping that people would triage the initial set
> of "opt-out" annotations that it landed with, and do something about those
> tests, but that never happened.  But hopefully this helped a bit by
> plugging one of the holes in terms of the unintended sources of
> intermittent oranges.
>
> In this case the way to opt out was adding a single line to the test.
> This wasn't hard per se, but if my memory serves it used to trip people for
> quite a while, even though the failure message you would get if you forgot
> to do it would indicate how to opt out of the extra check.  In hindsight, I
> think some people may have found the check more annoying than helpful,
> because in the cases where it helps, you'd get green tests, so nobody would
> consciously know that this is helping and in the cases where it would get
> in your way, people would be rightfully unhappy since their workflow got
> disturbed.
>

I forgot mochitest already checks for flaky timeouts! Maybe people would be
less opposed to a lint rule as many teams (especially frontend) have
already ingrained eslint into their workflows and know how to deal with
enabling/disabling rules. I guess in the setTimeout case we'd need to
choose one or the other, either eslint or requestFlakyTimeout. Doing both
at once would be a pain for developers. Fwiw, the eslint rule for
no-arbitrary-setTimeout turned out to be pretty easy to implement.



>
>>  1. Do any of the other patterns on that page look feasible to
>> implement as linters?
>>
>> I always struggled with this one.  Not many of them are, in my opinion.
> The example of using Date()s is, for example, but in my experience that is
> a super rare cause of intermittent failures...  A lot of these require some
> kind of dynamic check at runtime, and aren't easily detectible by just
> looking at the code of the test without running it.
>
>>
>>  1. Are there additional things not listed on that page that we could
>> lint for?
>>
>> Almost certainly.  Ideally when people fix an intermittent orange, they
> would think about whether this was a pattern that could reappear in other
> places, and whether it could be detected using a linter.  It is unclear to
> me how many of these new unique patterns we encounter though.  (For
> example, *many* intermittent oranges boil down to code expecting the wrong
> ordering of events, and that's not easily detectible using a linter
> But there may be a long tail of super rare error cases that are detectible
> using linters.)
>
> Hope this helps,
> Ehsan
>

Thanks for the insight. If we can block maybe 1 or 2 major causes of
intermittents with this, I'd be pretty happy. Maybe the bigger benefit of
this would be laying a foundation for adding new such linters in the
future, even if they only avoid minor causes of intermittents.

-Andrew
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Linting for common causes of oranges in mochitests (need ideas)

[Ehsan Akhgari]

FWIW years ago I decided to act on trying to detect some of the patterns 
indicated on that page automatically, see 
https://bugzilla.mozilla.org/show_bug.cgi?id=649012.  It didn't use 
linting (because we didn't really have any linting support back then) 
but the idea is the same, just with a different syntax.  That project 
never really received much (well, any!) traction since I always was 
hoping that people would triage the initial set of "opt-out" annotations 
that it landed with, and do something about those tests, but that never 
happened.  But hopefully this helped a bit by plugging one of the holes 
in terms of the unintended sources of intermittent oranges.


Based on that experience, here are some answers:


On 07/06/2017 02:47 PM, Andrew Halberstadt wrote:
I'm looking into creating custom eslint rules that aim to avoid common 
causes of oranges in tests. We have an MDN page 
 
containing some of these already. Some of those patterns might be 
pretty hard to catch with a linter, but others look do-able. So far, 
I'm thinking of adding rules for:


  * No arbitrary setTimeouts
  * No synthesizeKey without waiting for focus
  * No deleting original tab
  * ???

These rules would only be configured to run on test files. I have a 
few questions:


 1. Would this cause too much of a burden for legitimate uses of those
patterns?

In this case the way to opt out was adding a single line to the test.  
This wasn't hard per se, but if my memory serves it used to trip people 
for quite a while, even though the failure message you would get if you 
forgot to do it would indicate how to opt out of the extra check.  In 
hindsight, I think some people may have found the check more annoying 
than helpful, because in the cases where it helps, you'd get green 
tests, so nobody would consciously know that this is helping and in the 
cases where it would get in your way, people would be rightfully unhappy 
since their workflow got disturbed.


 1. Do any of the other patterns on that page look feasible to
implement as linters?

I always struggled with this one.  Not many of them are, in my opinion.  
The example of using Date()s is, for example, but in my experience that 
is a super rare cause of intermittent failures...  A lot of these 
require some kind of dynamic check at runtime, and aren't easily 
detectible by just looking at the code of the test without running it.


 1. Are there additional things not listed on that page that we could
lint for?

Almost certainly.  Ideally when people fix an intermittent orange, they 
would think about whether this was a pattern that could reappear in 
other places, and whether it could be detected using a linter.  It is 
unclear to me how many of these new unique patterns we encounter 
though.  (For example, *many* intermittent oranges boil down to code 
expecting the wrong ordering of events, and that's not easily detectible 
using a linter  But there may be a long tail of super rare error 
cases that are detectible using linters.)


Hope this helps,
Ehsan
I'd love to find volunteer(s) who are more familiar with writing 
mochitests than myself (and who also care about this issue) to help 
brainstorm and provide more in-depth feedback. I want to strike a 
balance between preventing oranges and not over-burdening test 
authors, but I'm not sure what that balance looks like.


Thanks in advance for your input,
Andrew


___
firefox-dev mailing list
firefox-...@mozilla.org
https://mail.mozilla.org/listinfo/firefox-dev


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to unship: moz*Frames attributes of HTMLVideoElement

[Ehsan Akhgari]

On 07/07/2017 03:41 AM, Anne van Kesteren wrote:

On Fri, Jul 7, 2017 at 8:51 AM, Jet Villegas  wrote:

Depending on what you find, we may have to keep this API around :-/

Yeah, I would expect us to follow the "normal" deprecation and remove
path to be more confident in the eventual unshipping:
Yes indeed.  (I just commented the same about unshipping another feature 
in a different thread, so I won't repeat literally all of the same 
points here, but they all apply here too!)

1. Gather telemetry on how often these APIs are used to measure feasibility.
2. Indicate in the console when these APIs are used that they are deprecated.
3. Bonus points for indicating in the console by which Firefox release
they are expected to be gone. (This might require more knowledge on
usage though.)
FWIW I don't think we have ever included a version number (at least 
according to my knowledge) and I don't think that is a good idea, since 
the specific version number depends on the data and our guesses often 
end up being wrong.  But we usually don't consider these messages as 
"ultimatums", rather we try to monitor usage and err on the side of not 
breaking content.

(Perhaps this is not documented anywhere currently?)
To the best of my knowledge, it isn't documented anywhere besides the 
archives of this list.  :-/

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to unship: tree pseudo-element selectors from web content (::-moz-tree-*)

[Ehsan Akhgari]

On 07/07/2017 12:29 AM, Xidorn Quan wrote:

On Fri, Jul 7, 2017, at 02:24 PM, Jet Villegas wrote:

It looks like there may be a lot more of these:
https://stackoverflow.com/a/25397485

How about we just stub out the Stylo impls instead of unshipping the pseudo in 
content?

I'm not unshipping this for Stylo... I just don't feel this is something we 
should keep at all... And I hope websites don't rely on this kind of 
browser-specific hacks :(
Sure, but it seems even more important for websites that people use and 
rely on to continue to work properly in Firefox, especially given that 
in practice websites certainly do rely on all sorts of browser-specific 
hacks for the better or worse.  :-/


Do we have any data on how commonly these selectors are used on the 
Web?  Usually the process for unshipping features like this is to land 
telemetry to detect their usage in the wild, find a way to figure out 
which sites do use it and figure out a plan to evangelize them off of 
using it (for example, adding a web console warning indicating the 
deprecation of the feature), waiting for a number of releases while 
monitoring the usage to ensure that at the removal time we certainly 
won't be breaking web pages for our users.  Any reason why we shouldn't 
go through the normal process here?


Thanks,
Ehsan
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to unship: moz*Frames attributes of HTMLVideoElement

[Mike Taylor]

On 7/7/17 2:41 AM, Anne van Kesteren wrote:
> On Fri, Jul 7, 2017 at 8:51 AM, Jet Villegas  wrote:
>> Depending on what you find, we may have to keep this API around :-/
> 
> Yeah, I would expect us to follow the "normal" deprecation and remove
> path to be more confident in the eventual unshipping:
> 
> 1. Gather telemetry on how often these APIs are used to measure feasibility.
> 2. Indicate in the console when these APIs are used that they are deprecated.
> 3. Bonus points for indicating in the console by which Firefox release
> they are expected to be gone. (This might require more knowledge on
> usage though.)
> 
> (Perhaps this is not documented anywhere currently?)

It would be good to document this, if it isn't currently.

Removing Moz vendor-prefixed stuff is cool in theory, but in reality we
risk breaking the web for our user base.

-- 
Mike Taylor
Web Compat, Mozilla
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Embedding Firefox or Gecko in 2017

[Henri Sivonen]

Positron isn't real Win32 embedding, though, right? That is, Positron
runs the whole app instead of a view inside an otherwise Win32 app.

To put a Gecko view inside a larger Win32 app, resurrecting EmbedLite
might the more appropriate way to go. The actively-maintained user of
EmbedLite code is the Sailfish Browser (not Win32 but Qt5), so it
probably makes sense to start at
https://git.merproject.org/mer-core/qtmozembed/activity and discover
the repo dependencies to find the latest EmbedLite code, which seems
to be at 
https://git.merproject.org/mer-core/gecko-dev/tree/nemo_embedlite/embedding/embedlite
.

On Fri, Jul 7, 2017 at 12:23 AM, Ralph Giles  wrote:
> You're right that this isn't currently easy to do.
>
> A more recent project, now also abandoned, was
> https://github.com/mozilla/positron You might see if that's a place to
> start.
>
>  -r
>
> On Thu, Jul 6, 2017 at 12:47 PM, cnico7  wrote:
>
>> Hi,
>>
>> I would like to embed gecko or firefox in a native windows application.
>> The need is to be able to use gecko's rendering and javascript engines in
>> order to display web pages that are not correctly displayed with other
>> browsers (mainly for legacy reasons of the intranet web site). Those pages
>> are part of a desktop application deployed on windows 7.
>>
>> So I did some searches around this subject and I discovered it could be a
>> complex path to achieve it with recent gecko versions.
>>
>> Here is what I found :
>>  * around 2000, an embedding solution was available through XPCOM as
>> described here : https://developer.mozilla.org/en-US/docs/Gecko/Embedding_
>> Mozilla
>> Unfortunately, this is obsolete and does not work any more from my
>> understanding.
>>  * this blog post http://chrislord.net/2016/03/08/state-of-embedding-in-
>> gecko/ lists some old embedding possibilities.
>> Unfortunately, none of them seems to still be available.
>>  * the servo project seems interesting but is to be shipped in years from
>> now.
>>
>> So here are my questions :
>> If I had the technical capabilities to contribute to a gecko embedding
>> solution, from what should I start investigating ?
>> Is EmbedLite (aka IPCLite) the good starting point or is there a better
>> path to embed gecko ?
>> If yes, is there a description of the API to use ?
>>
>> I had the idea of using firefox with marionette protocol in order to
>> interact with the engine and to use a custom plugin in order to hide all
>> the design (menus, tab bars,...). This idea has many drawbacks : it is slow
>> at launching time, it requires to improve marionette protocol in order to
>> intercept browsing events and hiding all ui elements with webextensions is
>> no more possible.
>> So my idea is clearly not a the good way.
>>
>> Any help would be appreciated. Even if I have not all the technical
>> knowledge of firefox internal, I am ready to work on it but I need the good
>> entry points to start.
>>
>> Thank you for reading me and for your answers,
>>
>> Regards,
>> ___
>> dev-platform mailing list
>> dev-platform@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform



-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

PSA: If using npm 5.0/5.0.1, please update to 5.1.0

[Mark Banner]

If you've got npm 5.0 or 5.0.1 installed, please update to 5.1.0 
. 
The latest version has a lot of fixes for its package handling when 
package.json is manually changed.


I'll be doing some ESLint dependency updates soon which might show up 
the issues if you have 5.0/5.0.1 installed.


Versions earlier than 5.0 shouldn't be affected.

Most npm installs can be updated with npm install -g npm

Mark

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to unship: moz*Frames attributes of HTMLVideoElement

[Anne van Kesteren]

On Fri, Jul 7, 2017 at 8:51 AM, Jet Villegas  wrote:
> Depending on what you find, we may have to keep this API around :-/

Yeah, I would expect us to follow the "normal" deprecation and remove
path to be more confident in the eventual unshipping:

1. Gather telemetry on how often these APIs are used to measure feasibility.
2. Indicate in the console when these APIs are used that they are deprecated.
3. Bonus points for indicating in the console by which Firefox release
they are expected to be gone. (This might require more knowledge on
usage though.)

(Perhaps this is not documented anywhere currently?)


-- 
https://annevankesteren.nl/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform