On Tue, Jul 10, 2018 at 10:16:37AM +0100, Chris Dent wrote: > On Mon, 9 Jul 2018, Matthew Treinish wrote: > > > It's definitely a bug, and likely a bug in stestr (or one of the lower > > level > > packages like testtools or python-subunit), because that's what's generating > > the return code. Tox just looks at the return code from the commands to > > figure > > out if things were successful or not. I'm a bit surprised by this though I > > thought we covered the unxsuccess and xfail cases because I would have > > expected > > cdent to file a bug if it didn't. Looking at the stestr tests we don't have > > coverage for the unxsuccess case so I can see how this slipped through. > > This was reported on testrepository some years ago and a bit of > analysis was done: https://bugs.launchpad.net/testrepository/+bug/1429196 >
This actually helps a lot, because I was seeing the same issue when I tried writing a quick patch to address this. When I manually poked the TestResult object it didn't have anything in the unxsuccess list. So instead of relying on that I wrote this patch: https://github.com/mtreinish/stestr/pull/188 which uses the output filter's internal function for counting results to find unxsuccess tests. It's still not perfect though because if someone runs with the --no-subunit-trace flag it still doesn't work (because that call path never gets run) but it's at least a starting point. I've marked it as WIP for now, but I'm thinking we could merge it as is and leave the --no-subunit-trace and unxsuccess as a known issues for now, since xfail and unxsuccess are pretty uncommon in practice. (gabbi is the only thing I've seen really use it) -Matt Treinish > So yeah, I did file a bug but it fell off the radar during those > dark times. >
signature.asc
Description: PGP signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
