On 11/9/18 11:32 AM, Alexander Bokovoy via FreeIPA-devel wrote:
On pe, 09 marras 2018, Florence Blanc-Renaud via FreeIPA-devel wrote:
Hi Developers,
Currently our test suite contains 3 different test sets:
- gating: executed on each pull request
- nightly_master: executed monday, wednesday, friday
- nightly_rawhide: not executed
Some tests are part of both gating and nightly, but that is an
exception. The majority of gating tests are not part of nightly, and
this was a choice done earlier in the project.
Christian raised a concern with this decision (see
https://github.com/freeipa/freeipa/pull/2540#issuecomment-437078271)
and would like nightly to also contain all the gating tests.
So what's your opinion on this? Should we keep gating and nightly
disjoint or make gating a subset of nightly?
I do think that having all gating tests included in nightly does not
hurt... I vote +1 for this. A different story would be if we would be in
serious lack of resources, but I do not think that's the case.
I think we can try to figure out what is our resources capacity and
based on that decide if we run nightly once per day and mix/alternate
different flavors: f28, f29, and rawhide. Also, having a dashboard to
show the last results/statistics will help.
I don't have any strong opinion on this, except that we should be
consistent...
I would have nightly running all tests if we can guarantee that at least
one nightly run is using up to date Fedora repositories, including
updates-testing.
With such configuration we would be testing:
- FreeIPA as in the git repo against a known Fedora configuration
("stable Fedora for FreeIPA development")
This is basically our nightly_master.
- FreeIPA as in the git repo against an up-to-date Fedora configuration
("stable FreeIPA for Fedora development")
and IIUC this one is the nightly_rawhide.
The latter one is very important to have to allow us automated upgrade
of our container images. Right now we fix Fedora container image in time
and then gate/run nightlies against that image. An update of the image
is almost always causing a breakage to us and require to find out what's
wrong, usually month or so after the actual breakage happened in Fedora.
I totally agree with this.
If we would have at least one weekly run that uses an updated container
image, we can automatically switch to that image as a base for our tests
in case of a success. We also will have a better way to investigate the
differences if there are nightly runs with the same code base (git
committish) against two different container images.
+1
I think if everybody agree, then Eagle Team can start planning these
changes. Taking into account that we are working on migrating to f29, it
would be the right moment to apply these changes.
-Triviño.
_______________________________________________
FreeIPA-devel mailing list -- freeipa-devel@lists.fedorahosted.org
To unsubscribe send an email to freeipa-devel-le...@lists.fedorahosted.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-devel@lists.fedorahosted.org
