On Mon, 17 Oct 2016 17:18:25 +0000
Zbigniew Jędrzejewski-Szmek <zbys...@in.waw.pl> wrote:

> On Mon, Oct 17, 2016 at 12:45:30PM -0400, Matthew Miller wrote:
> > On Mon, Oct 17, 2016 at 04:38:28PM +0000, Zbigniew
> > Jędrzejewski-Szmek wrote:  
> > > It's a good principle to require both tests and fixes required for
> > > those tests to pass to be submitted and merged as a single pull
> > > request. I'd love to see a PR that adds a test for one of my
> > > packages, exposes some bugs, but immediately fixes any fallout. I
> > > would be less thrilled to have tests committed which will fail on
> > > the next rebuild, leaving me to fix the package (or manually
> > > override the tests).  
> > 
> > As I'm imaginging it, tests added in this way would be non-blocking,
> > and would be expected to pass initially.  
> "feel good" tests ;) But seriously, why should people write only tests
> which pass?

I think this starts getting into the question about what we want to do
with the results.

If we are going to start gating builds based on results from
automation, we will need some way to indicate which results contribute
to a failure or a pass as part of that gating process.

The simple way to do this is to say "every test needs to pass for
gating to happen without manual override" but I'm not sure that's the
best solution here and I'm not aware of any existing proposals on how
to manage that in a more granular way for build gating.

I'm of the opinion that it would be best to get the checks/tests
running and once we have enough of those to justify gating builds, we
can start figuring out what kinds of things should fail a build.

Even if we're not gating builds right away, more testing is being done
and forward progress is being made.


Attachment: pgp8oNXRS46Do.pgp
Description: OpenPGP digital signature

devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Reply via email to