Re: disabled non-e10s tests on trunk

>you could edit the taskcluster/ci/test/tests.yml file and add >|linux64/debug: both| in the places we have |windows7-32/debug: both|, as >seen here: >http://searchfox.org/mozilla-central/source/taskcluster/ci/test/tests.yml#138 > >If there are concerns with windows build times, please ask for

Re: disabled non-e10s tests on trunk

you could edit the taskcluster/ci/test/tests.yml file and add |linux64/debug: both| in the places we have |windows7-32/debug: both|, as seen here: http://searchfox.org/mozilla-central/source/taskcluster/ci/test/tests.yml#138 If there are concerns with windows build times, please ask for those to

Re: disabled non-e10s tests on trunk

Joel, Is there an easy way for me to run non-e10s tests on linux? We often use "t-style" try pushes where we only run tests on one platform. Restricting non-e10s to win7-debug means I either need to run tests on multiple platforms or use windows for the "t-style". I don't want to use windows

Re: disabled non-e10s tests on trunk

Yesterday I landed bug 1391371 which enabled non-e10s unittests on windows 7 debug. A few tests had to be disabled in order to do this. To keep track of what we did and each of the test suites to evaluate, I have filed bug 1391350. As a note we now have the following non-e10s tests running:

Re: disabled non-e10s tests on trunk

On Wednesday, August 16, 2017 at 4:03:20 PM UTC-4, Nils Ohlmeier wrote: > > On Aug 16, 2017, at 07:23, James Graham wrote: > > > > On 16/08/17 01:26, Nils Ohlmeier wrote: > >> I guess not a lot of people are aware of it, but for WebRTC we still have > >> two distinct

Re: disabled non-e10s tests on trunk

> On Aug 16, 2017, at 07:23, James Graham wrote: > > On 16/08/17 01:26, Nils Ohlmeier wrote: >> I guess not a lot of people are aware of it, but for WebRTC we still have >> two distinct implementations for the networking code. >> So if I understand the impact here right

Re: disabled non-e10s tests on trunk

On 16/08/17 19:36, Ben Kelly wrote: My only thought about windows7-debug is that android is a variant of linux. Running a linux platform might be closer to android behavior. But I don't have a known specific difference in mind. Right it seems like there are two use cases here: 1) Tests that

Re: disabled non-e10s tests on trunk

On Wed, Aug 16, 2017 at 2:32 PM, Joel Maher wrote: > Thanks everyone for chiming in here. I see this isn't as simple as a > binary decision and to simplify things, I think turning on all non-e10s > tests that were running for windows7-debug would give us reasonable >

Re: disabled non-e10s tests on trunk

Thanks everyone for chiming in here. I see this isn't as simple as a binary decision and to simplify things, I think turning on all non-e10s tests that were running for windows7-debug would give us reasonable coverage and ensure that users on our most popular OS (and focus for 57) have a stable

Re: disabled non-e10s tests on trunk

On 08/15/2017 04:29 PM, jma...@mozilla.com wrote: I would propose running these above tests on windows7-opt (we don't have these running yet in windows 10, although we are close), and only running specific tests which are not run in e10s mode, turning them off December 29th, 2017. Keep in

Re: disabled non-e10s tests on trunk

On 15/08/17 21:39, Ben Kelly wrote: On Tue, Aug 15, 2017 at 4:37 PM, Joel Maher wrote: All of the above mentioned tests are not run on Android (well mochitest-media is to some degree). Is 4 months unreasonable to fix the related tests that do not run in e10s? Is there

Re: disabled non-e10s tests on trunk

On 16/08/17 01:26, Nils Ohlmeier wrote: I guess not a lot of people are aware of it, but for WebRTC we still have two distinct implementations for the networking code. So if I understand the impact here right we just lost test coverage for probably a couple of thousand lines of code. […]

Re: disabled non-e10s tests on trunk

I guess not a lot of people are aware of it, but for WebRTC we still have two distinct implementations for the networking code. So if I understand the impact here right we just lost test coverage for probably a couple of thousand lines of code. And yes you can potentially blame it on WebRTC

Re: disabled non-e10s tests on trunk

I think Ben's argument has merit: 1. Even after Firefox 57, we will still be shipping a product in non-e10s mode: Firefox for Android 2. If WPT (and potentially other suites) aren't being run in non-e10s mode, it increases risk because we are shipping untested code paths to our users Someone

Re: disabled non-e10s tests on trunk

On Tue, Aug 15, 2017 at 4:43 PM, Joel Maher wrote: > This is a discussion about tests in e10s mode, not WPT on Android. > Yes. And android is our last tier 1 platform that requires non-e10s. I think that makes it relevant to the discussion. > > What specific coverage are

Re: disabled non-e10s tests on trunk

This is a discussion about tests in e10s mode, not WPT on Android. What specific coverage are we missing by not running WPT in non-e10s mode on desktop? When can we ensure we have that coverage in e10s mode? I assume this is just making sure the tests that we have disabled on e10s mode need to

Re: disabled non-e10s tests on trunk

On Tue, Aug 15, 2017 at 4:37 PM, Joel Maher wrote: > All of the above mentioned tests are not run on Android (well > mochitest-media is to some degree). Is 4 months unreasonable to fix the > related tests that do not run in e10s? Is there another time frame that > seems

Re: disabled non-e10s tests on trunk

All of the above mentioned tests are not run on Android (well mochitest-media is to some degree). Is 4 months unreasonable to fix the related tests that do not run in e10s? Is there another time frame that seems more reasonable? On Tue, Aug 15, 2017 at 4:34 PM, Ben Kelly

Re: disabled non-e10s tests on trunk

On Tue, Aug 15, 2017 at 4:29 PM, wrote: > While there are many tests which individually are disabled or lacking > coverage, these test suites have no non-e10s coverage: > * web-platform-tests > * browser-chrome > * devtools > * jsreftests > * mochitest-webgl, mochitest-gpu,

Re: disabled non-e10s tests on trunk

Thanks everyone for commenting on this thread. As a note, we run many tests in non-e10s mode: * android mochitest, reftest, crashtest, marionette, * mochitest-chrome * xpcshell * gtest/cpptest/jittest * mochitest-a11y * mochitest-jetpack (very few tests remain) While there are many tests which

Re: disabled non-e10s tests on trunk

On Tue, Aug 8, 2017 at 5:18 PM, Ben Kelly wrote: > On Tue, Aug 8, 2017 at 5:12 PM, wrote: > >> In bug 1386689, we have turned them off. There was some surprise in >> doing this and some valid concerns expressed in comments in the bug. Given >> that, I

Re: disabled non-e10s tests on trunk

I ran some scripts that I had to find out tests that are *fully* disabled on e10s, and posted the results to https://bugzilla.mozilla.org/show_bug.cgi?id=1376934 In summary: mochitest-plain: 49 tests browser-chrome: 15 tests devtools: 86 tests Note that the script evaluates the skip-if condition

Re: disabled non-e10s tests on trunk

On Wed, Aug 9, 2017 at 3:36 AM, Boris Zbarsky wrote: > > Hmm. Do we load about:blank from the url bar in a content process? > > Yes. I agree, I find it annoying too that we have to rely on MOZ_DEBUG_CHILD_PROCESS or MOZ_DEBUG_CHILD_PAUSE and that I have to be clever all the

Re: disabled non-e10s tests on trunk

On 8/8/17 6:40 PM, Blake Kaplan wrote: What part of the current set-up is rocket science? Debugging pageload. Especially pageload with no session history. In multi, there's definitely a problem figuring out which process is the active one (though tooltips when hovering over tabs can help).

Re: disabled non-e10s tests on trunk

On 08/08/2017 06:51 PM, Blake Kaplan wrote: L. David Baron wrote: Has there been any additional effort to deal with tests that have been disabled under e10s? This change means we've essentially turned off a decent number of tests. IIRC, the last time we talked about this,

Re: disabled non-e10s tests on trunk

L. David Baron wrote: > Has there been any additional effort to deal with tests that have > been disabled under e10s? This change means we've essentially > turned off a decent number of tests. IIRC, the last time we talked about this, there was interest in either running

Re: disabled non-e10s tests on trunk

On Wed, Aug 9, 2017 at 9:31 AM, Boris Zbarsky wrote: Something as simple as "debug something that happens during pageload" is > currently fairly rocket science to do in e10s mode; doubly so in > e10s-multi. I haven't seen any concrete proposals for improving this rr

Re: disabled non-e10s tests on trunk

On Tuesday 2017-08-08 14:12 -0700, jma...@mozilla.com wrote: > As Firefox 57 is on trunk, we are shipping e10s by default. This means that > our primary support is for e10s. As part of this, there is little to no need > to run duplicated tests in non-e10s and e10s mode. > > In bug 1386689,

Re: disabled non-e10s tests on trunk

On 8/8/17 5:12 PM, jma...@mozilla.com wrote: In bug 1386689, we have turned them off. There was some surprise in doing this and some valid concerns expressed in comments in the bug. Indeed. Given how often non-e10s mode needs to get used for debugging, it's a little concerning to see the

Re: disabled non-e10s tests on trunk

On Tue, Aug 8, 2017 at 5:18 PM, Ben Kelly wrote: > On Tue, Aug 8, 2017 at 5:12 PM, wrote: > >> While we get some advantages to not running duplicated tests (faster try >> results, less backlogs, fewer intermittent failures) there might be >> compelling

Re: disabled non-e10s tests on trunk

On Tue, Aug 8, 2017 at 5:12 PM, wrote: > As Firefox 57 is on trunk, we are shipping e10s by default. This means > that our primary support is for e10s. As part of this, there is little to > no need to run duplicated tests in non-e10s and e10s mode. > We still run android