I think we can do silly automation in one specific plugin, after some trial, expand the idea or not:
pulpcore on 6844 [$] via 🐍 v3.8.0 (venv) ❯ git diff --name-only master CHANGES/6844.feature functest_requirements.txt setup.py pulpcore on 6844 [$] via 🐍 v3.8.0 (venv) took 2s ❯ git diff --name-only master | grep -E 'feature|bugfix' CHANGES/6844.feature pulpcore on 6844 [$] via 🐍 v3.8.0 (venv) ❯ git diff --name-only master | grep -E '^test_' Best regards, Fabricio Aguiar Software Engineer, Pulp Project Red Hat Brazil - Latam <https://www.redhat.com/> +55 11 999652368 On Tue, Aug 18, 2020 at 11:08 AM David Davis <davidda...@redhat.com> wrote: > +1 from me. > > One of the problems I foresee though is that this could make cherry > picking difficult if we have tests that depend on other tests. For example, > suppose you have change A that adds some tests and then change B adds some > tests on top of A's tests. It'll make cherry picking B without A tricky. > > We'll need to be careful to avoid such situations. > > David > > > On Tue, Aug 18, 2020 at 4:48 AM Matthias Dellweg <mdell...@redhat.com> > wrote: > >> +1 >> >> On Tue, Aug 18, 2020 at 10:39 AM Pavel Picka <ppi...@redhat.com> wrote: >> > >> > Big +1 to require (at least) one basic/positive functional test if >> possible. >> > Maybe we can set up a review checklist (contains 'check for basic >> test'). >> > >> > We already have some docs which we can update a bit [0] to be yet more >> plugin writer friendly and point back to it from plugins' docs. >> > Like extending it with the way the test can be run when pulp is >> installed e.g. by the rpm package. >> > >> > [0] https://docs.pulpproject.org/contributing/tests.html >> > >> > On Mon, Aug 17, 2020 at 10:07 PM Brian Bouterse <bmbou...@redhat.com> >> wrote: >> >> >> >> I have two goals with this: >> >> >> >> 1. Improve the stability of pulpcore and it's plugins >> >> >> >> 2. Enable all downstreams and packagers with tests and information on >> how to run those tests, so they can run the automated tests even as the >> downstream is curated with a series of backports. This closely follows the >> model of a downstream maintainer who backports a feature/fix from upstream >> into a RPM and runs the upstream tests to make sure their patch didn't >> break functionality. >> >> >> >> # Proposal >> >> Upstream could require a basic functional test for each feature or a >> test for each bug fix. This test would be in the same commit as the feature >> or fix causing it to "follow the code" so when it's cherry picked >> downstream or into a package, the test goes with it. I believe we don't >> change the test during cherry pick time because although the code >> implementation may change what the feature or fix provides will not. >> >> >> >> Additionally, upstream should document how to run these tests easily >> and have the goal of making that easy. >> >> >> >> This policy change would ultimately be decided by each plugin and >> pulpcore separately. I don't want to make it a requirement to be part of >> the github.com/pulp/ organization in any way, so if a plugin can't, or >> won't, make this change that's ok, they can still host at >> github.com/pulp/. >> >> >> >> As a pulpcore and pulp_file maintainer, I'm proposing this policy for >> both of those repos specifically. >> >> >> >> In terms of enforcement, I'm pitching manual enforcement by review as >> a start. We can automate it later, but I hope that's a separate discussion >> to focus on the policy change and keep it simple. >> >> >> >> # Brian's take >> >> This is a non-trivial contribution requirement so we need to really >> think about it. My personal take is that at this point in the 3.y release >> line it's time to trade feature/fix velocity for stability. Also >> backporting is about to become a regular activity, so I think we have a >> practical motivation to adopt this, even though it will slow down the >> future features and bug fix velocity. >> >> >> >> What's your perspective? >> >> >> >> _______________________________________________ >> >> Pulp-dev mailing list >> >> Pulp-dev@redhat.com >> >> https://www.redhat.com/mailman/listinfo/pulp-dev >> > >> > >> > >> > -- >> > Pavel Picka >> > Red Hat >> > _______________________________________________ >> > Pulp-dev mailing list >> > Pulp-dev@redhat.com >> > https://www.redhat.com/mailman/listinfo/pulp-dev >> >> >> _______________________________________________ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> >> _______________________________________________ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
