Re: Reminder on Try usage and infrastructure resources

On 15/09/17 18:45, Dan Mosedale wrote: I wonder if this isn't (in large part) a design problem disguised as a behavior problem. The existing try syntax (even with try chooser) is so finicky and filled with abbreviations that even after years of working with it, I still regularly have to look up

Re: Reminder on Try usage and infrastructure resources

I wonder if this isn't (in large part) a design problem disguised as a behavior problem. The existing try syntax (even with try chooser) is so finicky and filled with abbreviations that even after years of working with it, I still regularly have to look up stuff and sometimes when I've been in a

Re: Reminder on Try usage and infrastructure resources

Masayuki, your try push had trouble because you requested "mochitest-2" instead of "mochitest-e10s-2". Non-e10s mochitests only run on Android and Windows now. You probably wanted something like: https://treeherder.mozilla.org/#/jobs?repo=try=d68382f17d63f0674c62acc7242a9e406793895f This is a

Re: Reminder on Try usage and infrastructure resources

Even when I got the chunk numbers, specifying chunk numbers of mochitests wouldn't work, see this log: https://treeherder.mozilla.org/#/jobs?repo=try=c09c7046ed0664e89f7224e1de5219c39c94c948 After that, I needed to rerun mochitests with |-u mochitests|. IIRC, I tried to kick the specific chunks

Re: Reminder on Try usage and infrastructure resources

I tried to say different point. See the treehearder log, mochitests didn't run except on Win7 Debug, Android 4.3 API16+ Opt/Debug. So, try syntax parser or something is really broken. I often meet this kind of bug. On 9/15/2017 10:07 AM, Kris Maglione wrote: Your best bet is probably to use

Re: Reminder on Try usage and infrastructure resources

On 15/09/17 00:53, Dustin Mitchell wrote: 2017-09-14 18:32 GMT-04:00 Botond Ballo : I think "-p all" is still useful for "T pushes" (and it sounds like build jobs aren't the main concern resource-wise). Correct -- all builds are in AWS. I'd like to steer this away from

Re: Reminder on Try usage and infrastructure resources

Your best bet is probably to use `mach try` with a specific set of test directories. It will generate a set of --try-test-paths flags to restrict tests to those paths, and only run the first chunk of any group. Without that, groups shift around too much to be reliable. On Fri, Sep 15, 2017

Re: Reminder on Try usage and infrastructure resources

2017-09-14 18:32 GMT-04:00 Botond Ballo : > I think "-p all" is still useful for "T pushes" (and it sounds like > build jobs aren't the main concern resource-wise). Correct -- all builds are in AWS. I'd like to steer this away from "What legacy syntax should we use instead?"

Re: Reminder on Try usage and infrastructure resources

On Thu, Sep 14, 2017 at 4:54 PM, Mike Hommey wrote: > Maybe it's time to kill the `all` flag, at least for -p. Why? For the > combined reason that you're saying we shouldn't be using it, and that > it's actually *not* running every platform. I think "-p all" is still useful

Re: Reminder on Try usage and infrastructure resources

On Thu, Sep 14, 2017 at 11:35:53AM -0400, Stuart Philp wrote: > Hello all, > > As we near 57 the Firefox CI group felt it was important to send out a bit > of a reminder regarding infrastructure usage when you push. > > *tl;dr* There is a real cost (both time and $) to using the 'all' flags in >

Re: Reminder on Try usage and infrastructure resources

You can try ActiveData, which stores all test results from the past few weeks.  Here is an example query that shows the chunk number for each run/build combo in the past day.  ActiveData is sometimes more than a day behind https://activedata.allizom.org/tools/query.html#query_id=4HHuBgDu {    

Re: Reminder on Try usage and infrastructure resources

On Thu, Sep 14, 2017 at 6:13 PM, Andrew Halberstadt wrote: > Yes, all mochitests except Android restart between manifests (which is > usually the same as folders). Thank you very much, that's really useful to know and will allow me to save some infra time!

Re: Reminder on Try usage and infrastructure resources

Yes, all mochitests except Android restart between manifests (which is usually the same as folders). On Thu, Sep 14, 2017 at 12:03 PM Marco Bonardo wrote: > On Thu, Sep 14, 2017 at 5:56 PM, James Graham > wrote: > > On 14/09/17 16:48, Marco Bonardo

Re: Reminder on Try usage and infrastructure resources

That’s correct, yeah. If you don’t have a push where it’s failed already, then it won’t show in the Test Centric UI. Though I’ll write up a bug to explore adding this functionality. Perhaps there’s a way to mine Active Data to get this. -Cam > On Sep 14, 2017, at 9:05 AM, Gijs Kruitbosch

Re: Reminder on Try usage and infrastructure resources

This only works once you have a run that failed the test you're interested in, right? There's no way to tell the test-centric UI "find me the chunk for test with name X". ~ Gijs On 14/09/2017 16:55, Cameron Dawson wrote: Marco— I don’t know of a way to do exactly that yet. But that is in

Re: Reminder on Try usage and infrastructure resources

There's sort of a way to do this with try syntax. I say sort of because it doesn't support all suites and there seems to be a few bugs with it. But you can try: ./mach try -b o -p linux64 -u none path/to/dir/or/test This should only run the directory or test you specified (it'll always show up

Re: Reminder on Try usage and infrastructure resources

On Thu, Sep 14, 2017 at 5:56 PM, James Graham wrote: > On 14/09/17 16:48, Marco Bonardo wrote: > mach try -p linux64 Afaict, that runs a single folder, but the intermittent may be caused by interactions across different tests in different folders. I'm not up-to-date to

Re: Reminder on Try usage and infrastructure resources

On 14/09/17 16:48, Marco Bonardo wrote: When I need to retrigger a mochitest-browser test multiple times (to investigate an intermittent), often I end up running all the mochitest-browser tests, looking at every log until I find the chunk where the test is, and retrigger just that chunk. The

Re: Reminder on Try usage and infrastructure resources

Marco— I don’t know of a way to do exactly that yet. But that is in the roadmap for the Test-based UI in Treeherder. And the existing UI may help you there. On any push, click the down arrow (Action Menu) at the far right of the push status line and select “Experimental: Test-Centric UI”

Re: Reminder on Try usage and infrastructure resources

> On 14 Sep 2017, at 17:48, Marco Bonardo wrote: > > When I need to retrigger a mochitest-browser test multiple times (to > investigate an intermittent), often I end up running all the > mochitest-browser tests, looking at every log until I find the chunk > where the test

Re: Reminder on Try usage and infrastructure resources

When I need to retrigger a mochitest-browser test multiple times (to investigate an intermittent), often I end up running all the mochitest-browser tests, looking at every log until I find the chunk where the test is, and retrigger just that chunk. The chunk number changes based on the platform

Reminder on Try usage and infrastructure resources

Hello all, As we near 57 the Firefox CI group felt it was important to send out a bit of a reminder regarding infrastructure usage when you push. *tl;dr* There is a real cost (both time and $) to using the 'all' flags in pushes. They are there if you need them, but please remember to think about