Re: Intent to deprecate - linux32 tests starting with Firefox 69

On 2019-04-09 11:00 a.m., Gian-Carlo Pascutto wrote: On 5/04/19 15:35,jma...@mozilla.com wrote: Currently linux32 makes up about .77% of our total users. This is still 1M+ users on any given week. I asked jmaher what percentage of our Linux users this is. It's 21%. This doesn't seem small.

Re: Shippable Builds Incoming...

New version of mozregression released which should have support for shippable builds (and also fixing bisection after the opt->pgo transition). Thanks to :Kwan and :Callek for making this happen. Will On 2019-03-26 3:08 p.m., Justin Wood wrote: Yes it does, I'm currently looking at a good

Re: Removing tinderbox-builds from archive.mozilla.org

On 2018-05-09 2:58 PM, Andrew Halberstadt wrote: Going back to the original question, it looks like mozregression doesn't use the builds that Nick wants to remove anyway. So regardless of our retention policies, it looks like removing these builds would have no impact on mozregression's

Re: Removing tinderbox-builds from archive.mozilla.org

On 2018-05-09 2:01 PM, Ted Mielczarek wrote: It's useful for tracking down regressions no matter how old the regression is; I pretty regularly see mozregression finding useful data on bugs that regressed multiple years ago. To be clear here--we still have an archive of nightly builds dating

Re: Removing tinderbox-builds from archive.mozilla.org

On 2018-05-09 11:48 AM, Botond Ballo wrote: Good question. I checked and it seems that the answer is no (yay). For nightly builds in mozregression, we fetch stuff out of: https://archive.mozilla.org/pub/firefox/nightly/ For inbound builds, we've been using taskcluster for a while. What about

Re: Removing tinderbox-builds from archive.mozilla.org

On 2018-05-09 4:59 AM, Xidorn Quan wrote: Would removing those files affect the ability of mozregression to locate pushes of old regressions? Good question. I checked and it seems that the answer is no (yay). For nightly builds in mozregression, we fetch stuff out of:

Re: CPU core count game!

I don't believe these counts take into account the number of usage hours of each client, so people with 4+ cores may actually account for a larger amount of relative Firefox usage than their absolute numbers would suggest. I am sure some people on the data / analyst / product side of Firefox

Re: ./mach try fuzzy: A Try Syntax Alternative

On 2017-09-02 11:04 PM, Andrew Halberstadt wrote: - --env: You can now set environment variables in your tasks directly from the command line, e.g: - ./mach try fuzzy --env XPCOM_DEBUG_BREAK=stack --env MOZ_LOG="example_logger:3"| This is*awesome*; I've been wanting this for

Re: C++ performance test harness?

On 2017-07-05 8:36 AM, Emilio Cobos Álvarez wrote: On 07/05/2017 10:19 AM, Henri Sivonen wrote: Do we have a gtest analog for local performance testing? That is, something that makes it easy to microbenchmark libxul methods? For CSS parsing there is a benchmark using MOZ_GTEST_BENCH[1].

Re: Proposal for informing Developers and Sheriffs about new Testsuites

Isn't this the sort of thing m.d.tree-management is for? I would say that + sheriffs@ should be enough. Will On 2017-06-14 10:37 AM, Carsten Book wrote: i guess sheriffs list (sheriffs @ m.o ) and newsgroup dev.platform, fx -dev but i'm also open for suggetions from others :) - tomcat On

Re: JSBC: JavaScript Start-up Bytecode Cache

On 2017-06-13 4:36 PM, Chris Peterson wrote: Nicolas, when JSBC is enabled by default, should we change our test procedure for our various page load tests (Talos and Softvision's manual testing)? Since the first page load will be slower than subsequent page loads (as you noted in the bug [1]),

Re: Linux builds now default to -O2 instead of -Os

On 2017-06-06 1:33 PM, Boris Zbarsky wrote: On 6/1/17 9:04 PM, Mike Hommey wrote: Ah, forgot to mention that. No, it doesn't affect *our* shipped builds (because PGO uses a different set of optimization flags). But it does affect downstream builds that don't PGO. Based on the jump I see on

Re: Improving visibility of compiler warnings

On 2017-05-19 2:44 PM, Gregory Szorc wrote: The Perfherder alerts and tracking are informational only: nobody will be backing you out merely because you added a compiler warning. While the possibility of doing that now exists, I view that decision as up to the C++ module. I'm not going to

Turning off platform microbenchmarks

Hi all, Last year, Benoit Girard added "platform microbenchmarks" to our test suite, and we started tracking the data in Perfherder: https://groups.google.com/d/msg/mozilla.dev.platform/uVA3bEvhUg0/g9WEY817BwAJ It seemed like a promising idea but almost a year later, no one seems to be

Some recent Treeherder improvements

Just wanted to highlight some recent Treeherder changes of interest to sheriffs and platform developers: * As of today, we have a completely rewritten logviewer which (among other things) has vastly improved scrolling behaviour, making it much faster to diagnose the cause of a failing job.

Perfherder back up (was Re: Perfherder is currently broken)

ge mistake. It just duplicates what a database like MySQL was designed to do, but poorly. Looking forward, I am going to do some work to make sure this never happens again: https://bugzilla.mozilla.org/show_bug.cgi?id=1310386 Will On 2016-10-13 3:14 PM, William Lachance wrote: Update: I have a f

Re: Perfherder is currently broken

talos jobs, please go ahead. :) I'll make another announcement when everything is ok. Will On 2016-10-11 9:59 PM, William Lachance wrote: Hey all, Due to an unfortunate bug that went undiscovered until we migrated to Heroku last week, Perfherder currently isn't giving reliable results. I hope

Perfherder is currently broken

Hey all, Due to an unfortunate bug that went undiscovered until we migrated to Heroku last week, Perfherder currently isn't giving reliable results. I hope to have this fixed within the next couple of days (https://bugzilla.mozilla.org/show_bug.cgi?id=1309294). No data should be lost, so

RFC: Implementing tools/process for tracking push end-to-end times and other Mozilla CI metrics

Hey all, I've been prototyping a new tool for monitoring the performance of our infrastructure called "Infraherder" (in particular focusing on tracking the end-to-end time from push submission to a full set of test/build results being available) and I'm at the point where I'd like to get some

Re: Making try faster for debugging intermittents

On 2016-07-01 6:10 AM, Mike Hommey wrote: > > One thing that might be helpful is enabling running only tests on > > try with a designated build that has already been created. > > > > Often tests are modified to add logging, after which the same > > build could be run with the new version of the

Making try faster for debugging intermittents

As part of a larger effort to improve the experience around debugging intermittents, I've been looking at reducing the time it takes for common "try" workloads for developers (so that e.g. retriggering a job to reproduce a failure can happen faster). Of course, the common advice of "profile

Re: PSA: Cancel your old Try pushes

On 2016-04-18 2:46 PM, William Lachance wrote: Treeherder did trigger a rather large memory leak which got fixed in the browser a while back (Dec 2015), so please consider revisiting it if you gave up around then: ... If anyone feels like profiling and submitting patches, we'd welcome the help

Re: PSA: Cancel your old Try pushes

On 2016-04-16 5:50 AM, Gijs Kruitbosch wrote: On 16/04/2016 01:24, Steve Fink wrote: Doesn't everyone keep a tab open to their try page? eg I have https://treeherder.mozilla.org/#/jobs?repo=try=sf...@mozilla.com open all the time. No. Treeherder is too resource-intensive to keep open for long

New talos e10s comparison mode: e10s-vs-non-e10s on a single push

Hey all, I added a new mode to the talos e10s comparison view, that lets you compare e10s-vs-non-e10s numbers on the same push. For example: https://treeherder.mozilla.org/perf.html#/e10s?repo=try=77c805ba9c18 Currently there is no UI wired up, the expectation is that this would be mostly

Re: Skia Content on OS X

On 2016-03-22 11:21 AM, Mike de Boer wrote: I was also quite curious, so I started clicking up the hierarchy of that bug and ended up at: https://bugzilla.mozilla.org/show_bug.cgi?id=1154825#c1 (comment 2 is also useful info)

Platform engineering project of the month: Perfherder

Hello from Platform Engineering Operations! Once a month we highlight one of our projects to help the Mozilla community discover a useful tool or an interesting contribution opportunity. This month’s project is Perfherder! What is Perfherder? Perfherder is a generic system for visualizing

Re: Improving Platform quality

Hi Gabor! Thanks for starting this thread. On 2016-03-10 5:07 AM, Gabor Krizsanits wrote: While the other thread about fuzzing friendly Gecko is an interesting option I would like to go back to the original topic, and start another thread to collect other ideas too, that might help getting

Re: Talos e10s dashboard

On 2016-02-29 1:52 AM, Chris Peterson wrote: Will, this dashboard looks great! The current e10s release criteria allows regressions up to 5% on the following Talos tests. Is it possible to configure your e10s dashboard to display acceptable (<= 5%) regressions on these particular tests using

Re: Talos e10s dashboard

On 2016-02-27 10:09 AM, Jim Mathies wrote: On Friday, February 26, 2016 at 5:58:54 PM UTC-6, William Lachance wrote: Hey, I wrote up a dashboard for tracking the performance delta between non-e10s and e10s on the Talos tests on nightly: https://treeherder.allizom.org/perf.html#/e10s

Talos e10s dashboard

Hey, I wrote up a dashboard for tracking the performance delta between non-e10s and e10s on the Talos tests on nightly: https://treeherder.allizom.org/perf.html#/e10s (sometime next week, https://treeherder.mozilla.org/perf.html#/e10s will work too) Note that the tests are not always

Re: Presto: Comparing Firefox performance with other browsers (and e10s with non-e10s)

On 2016-02-14 8:30 PM, Valentin Gosu wrote: >Great! It'll be interesting to track how these change over time as well >(or as versions get added to the list). Again, medians/means/etc may >help with evaluating and tracking this (or automating notices, ala Talos) > I thought about doing this,

Re: Memory Usage on Perfherder & Memory Reduction

Incidentally, the automatic regression detection dashboard has been coming together recently with Perfherder, which should let you track such things as this much more easily (as well as providing a convenient interface for filing bugs). For example, you can see all recently detected

Re: Proposed changes to Talos (performance) alerting

On 2016-01-18 4:42 AM, Nicolas B. Pierron wrote: I agree, this should be the part of the developer to work that out, but the TS Paint benchmark is out of the knowledge base of JS developers. I feel that the problem is reaching developers with a wording known by the developers. So we have a

Re: any concerns with dropping the talos test v8 and using AWFY Octane instead?

On 2016-01-11 4:12 PM, Eric Rahm wrote: On Monday, January 11, 2016 at 8:42:11 AM UTC-8, William Lachance wrote: It seems like another alternative might be to run Octane in Talos, instead of v8_7. It seems like Talos has two advantages over AWFY (correct me if I'm wrong): 1. Easy

Re: any concerns with dropping the talos test v8 and using AWFY Octane instead?

On 2016-01-11 9:08 AM, jma...@mozilla.com wrote: Currently we run a very outdated version of V8 (version 7) in Talos. This has since been replaced with Octane in the world of benchmarks. AWFY (arewefastyet.com), has been running Octane and catching regressions faster than Talos. There is

Proposed changes to Talos (performance) alerting

I'd like to propose some changes in how we report and triage Talos alerts. Over the past couple years, Joel Maher (with occasional assistance from myself and others) has taken over the job of triaging and responding to ("sheriffing") Talos regressions. He's done this through a bunch of

Re: Now measuring Firefox size per-commit. What else should we be tracking?

On 2015-11-06 5:56 PM, Mark Finkle wrote: I also think measuring build times, and other build related stats, would be useful. I'd like to see Mozilla capturing those stats for developer builds though. I'm less interested in build times for the automation. That data is already looked at by the

Now measuring Firefox size per-commit. What else should we be tracking?

Hey, so as described here: http://wrla.ch/blog/2015/11/perfherder-onward/ ... I recently added tracking for Firefox installer size inside Perfherder. This should let us track how bloated (or not) Firefox is on our various supported platforms, on a per-commit basis:

Re: Now measuring Firefox size per-commit. What else should we be tracking?

On 2015-11-04 11:15 AM, Chris AtLee wrote: This is really great, thanks for adding support for this! I'd like to see the size of the complete updates measured as well, in addition to the installer sizes. Where are the updates generated? As part of the build process? If so, can you self-serve

Re: Now measuring Firefox size per-commit. What else should we be tracking?

On 2015-11-04 10:55 AM, William Lachance wrote: 1. Relatively deterministic. 2. Something people actually care about and are willing to act on, on a per-commit basis. If you're only going to look at it once a quarter or so, it doesn't need to be in Perfherder. Anyway, just thought I'd open

Re: mach mozregression command

On 2015-09-14 1:17 PM, Benjamin Smedberg wrote: Does this command work by downloading nightly/hourly builds (the way mozregression typically works) or by doing local builds at various changesets and getting a local source regression range? The former. It's basically just a convenience wrapper

Proposal: Turning off Eideticker perf testing

The Eideticker project (http://eideticker.mozilla.org) was started almost 4 years ago, in an effort to better measure the performance of Firefox for Android by directly capturing video of the browser performing various actions. It later evolved to encompass testing FirefoxOS performance. It's

Re: Proposal: Turning off Eideticker perf testing

On 2015-08-17 7:21 PM, Kartikaya Gupta wrote: FWIW one of the original driver behind Eideticker (tuning Fennec for checkerboarding during scrolling) will become relevant again in the next couple of months as we transition Fennec to using the C++ APZ code and off the old Java pan/zoom code. While

Talos changes: moving in tree, EOL for Android

Joel hinted at these changes in his post earlier in the week (https://groups.google.com/d/msg/mozilla.dev.platform/PaJFBtvc3Vg/91Mb5DEwTGEJ), but these are going to be major updates and some impact to developer workflow is immediate, so I'm going to call them out here so people are informed

Treeherder Hacking Sessions @ Whistler

The Treeherder devs will be on-hand for a hacking session in Whistler Thursday next week. If you've curious to get your hands into the Treeherder code, or have a workflow you'd like to improve, please stop by. This session will be two-track: One for the Treeherder UI, the other aimed at data

Re: what is new in talos, what is coming up

Hi Karl, On 2015-06-04 12:30 AM, Karl Tomlinson wrote: jma...@mozilla.com writes: 2) compare-talos is in perfherder (https://treeherder.mozilla.org/perf.html#/comparechooser), other instances of compare-talos have a warning message at the top indicating you should use perfherder. We will

Re: Change coming to ftp.mozilla.org

On 2015-04-21 1:29 PM, Kevin Brosnan wrote: This form has some issues. There are required sections of Downloading with scripts and other programs that only make sense for developers of the script or download tool. - Which protocols do you use ? (no idea whatever mozregression uses) -

Talos on e10s is broken. Do we care?

Last fall, we got Talos working on e10s and scheduled e10s-specific jobs on mozilla-central and holly. It looks like sometime in the last few weeks something broke talos-on-e10s on an integration branch, and now all the jobs are broken: https://bugzilla.mozilla.org/show_bug.cgi?id=1134238

Testing infrastructure change: ManifestDestiny - manifestparser

Just wanted to make a quick announcement that ManifestDestiny, the python package we use internally here at Mozilla for declaratively managing lists of tests in Mochitest and other places, has been renamed to manifestparser. We kept the versioning the same (0.6), so the only thing you should

Modifying files in `testing/mozbase`

Recently we've started moving our in-tree harnesses (mochitest, reftest) to use a set of python libraries called mozbase (https://wiki.mozilla.org/Auto-tools/Projects/MozBase). I just wanted to bring up several things about this: 1. MozBase is developed on github

Python device manager code for Android + B2G moved

Just a heads up: In an effort to help facilitate code mirroring between mozilla-central and the mozbase project (https://wiki.mozilla.org/Auto-tools/Projects/MozBase), I've moved most of the python device manager files used for mobile testing (both for Android and B2G) from `build/mobile` to

Re: Increase in mozilla-inbound bustage due to people not using Try

On 08/15/2012 07:08 PM, Gregory Szorc wrote: When I was working on this project last year, I designed a build charts view to help visualize which parts were taking the longest (you can see implicit dependencies between build/test tasks by seeing when certain jobs run), which proved very helpful