On Mon, 2015-08-17 at 15:31 -0400, Matthew Miller wrote: > On Mon, Aug 17, 2015 at 09:08:31AM -0700, Adam Williamson wrote: > > Hi folks! So I was wondering: what do folks think about running > > openQA's 'all' mode nightly (on the BOS deployment) instead of the > > 'current' mode? > > +1; do it! > > > compose. I think it'd be nice to have an idea every day if Rawhide > > and/or Branched are broken. We could then actually write some kind > > of > > 'israwhidebroken.com' thing. Ideally we'd be doing this with fedmsg > > How do we get from "it's broken today" to "and someone should do > something about it?"
Electric shock collars? No, seriously, I'm kinda working on some stuff to do some sort of daily compose status report, which should help. openQA doesn't do automatic bug submission and we probably don't want to do it (because of cases where the failures are more to do with openQA than Fedora, and avoiding potential excessive duplication when libreport dupe detection fails), so the first thing that needs to happen is someone (i.e. one of us) goes and looks at the failures and submits bugs if they're actually valid problems. If the bugs are significant enough they'll wind up as release blockers, which is probably an adequate short-term mechanism. We're still a reasonably long distance from true CI in the sense of '...and if it's broken, we block the change that broke it', because daily is still far too coarse granularity for that. For that we'd need Dennis' pet project that somehow knows all the things that can possibly influence the behaviours of the images, and we'd need to run the tests every time any of those things changed, and we'd need to do it fast enough to gate the changes. Which is all fairly challenging stuff. This is more 'observe and report' level stuff now and for the medium-term foreseeable future, I'd say. What we could (probably) do right now is hack up a dumb lil' thing with a hardcoded list of packages which would re-spin an image every time a build of one of those packages happened, and run maybe a subset of the most critical tests on it (to avoid the tests taking too long). But it would probably be a bit of a duct-tape project for now, and it'd be a few days' work at least (and that's an estimate with my 'pretend developer' hat on, so I'm now going to switch into my 'QA' hat and revise that idiot's estimate to 'two weeks at least'). -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ qa-devel mailing list [email protected] https://admin.fedoraproject.org/mailman/listinfo/qa-devel
