Top posting FTW! (sorry) Excellent news Adam, this is awesome!
Simo. On Fri, 2017-04-28 at 17:07 -0700, Adam Williamson wrote: > Hi folks! I thought this might be of interest to the FreeIPA > community, > so I thought I'd write it up here in case anyone missed it elsewhere. > > I work on the Fedora QA team, and we have been using the openQA > automated test system (developed by our friends at SUSE) to run > various > functional tests on Fedora composes for the last couple of years. > > As FreeIPA is considered a critical part of Fedora Server, we run a > few > tests that exercise FreeIPA. The tests set up a FreeIPA server, run > some basic checks on it, and also enrol two systems as clients of the > domain, one using the 'realm join' command directly, one using > Cockpit. > The client tests do some basic client functionality testing (getent, > logging in as a domain user, changing passwords, etc.) and also test > the web UI to some extent. > > Until recently we ran these tests only on Fedora's nightly > development > release distribution composes. Recently, though, we deployed some > enhancements to our openQA setup that let us run tests on Fedora > distribution updates as well, and have the results made visible > through > the Fedora update system (Bodhi). The tests are automatically run on > any critical path package, and as of today, they are also run on any > update containing any of a manually-tended list of FreeIPA-related > packages: > > 389-ds > 389-ds-base > bind > bind-dyndb-ldap > certmonger > ding-libs > freeipa > krb5-server > pki-core > sssd > tomcat > cockpit > > This means that for any Fedora update containing one of these or any > critical path package, Fedora's openQA FreeIPA tests should run, and > you should see the results in the Fedora update system (Bodhi). You > can > see the results in Bodhi by clicking the Automated Updates tab for > any > update. For instance, here's a recent 389-ds-base update for Fedora > 26: > > https://bodhi.fedoraproject.org/updates/FEDORA-2017-15e2a038b2 > > If you look at the Automated Tests tab, you can see passes for: > > update.server_role_deploy_domain_controller > update.realmd_join_cockpit > update.realmd_join_sssd > > indicating that this update didn't cause any problems for FreeIPA. > Clicking on any test result will take you to the openQA page for the > test, where you can diagnose failures and so on (explaining how to do > this is a bit beyond the scope of this mail, please do ask me if > you're > interested!) > > I hope this stuff will help us avoid shipping updates that break > FreeIPA (and other key components). If you have any questions, > concerns, comments, or suggestions, please do ask! > > To anticipate one question: you can cause *all* the tests for an > update > to be re-run by editing the update in any way (you don't have to > change > the package loadout, just changing a single character in the > description or something will do). If you think just one test result > is > bogus and want it re-run, currently, you'll have to ask someone with > the necessary power - either me or Jan Sedlak (garretraziel on IRC). > I'm in North America and he's in Europe, so we should have most > timezones covered between us. We're hoping to set up a better > mechanism > for this in future. > > Note, if you're interested in the results for the nightly Fedora > distribution composes, an email summary of the results for those is > sent each time they're run to the Fedora test@ and devel@ lists, look > for mails with "compose check report" in the subject. Any time any of > the FreeIPA tests fails, the failure will be listed in the mail > (passed > tests are not specifically listed, just a count of them). I usually > keep an eye on those results and analyze failures and file bugs, > though. > -- > Adam Williamson > Fedora QA Community Monkey > IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . > net > http://www.happyassassin.net > -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
