On 10/09/2015 09:01 AM, Milan Kubík wrote:
On 10/08/2015 02:50 PM, Martin Basti wrote:



On 10/08/2015 02:39 PM, Martin Kosek wrote:
On 10/08/2015 02:08 PM, Oleg Fayans wrote:
Hi,

On 10/08/2015 11:18 AM, Jan Pazdziora wrote:
On Thu, Oct 08, 2015 at 11:12:37AM +0200, Oleg Fayans wrote:
When the ticket is addressed and these workarounds are no longer
needed -- what is our process for finding these workarounds and
reverting them, so that the tests test the real, expected behaviour?
As per discussion with Martin Basti, it was decided that this workaround will only be applied to the current 4-2 branch, not to the upstream. In
That sounds like a reasonable plan for this issue.

upstream the issue itself will (supposedly) be solved
Except currently it's not, so (IIUIC) you will keep having
nondeterministic failures in master.

I was mostly interested in the general approach that we have to
workarounds -- how do we track them, how do we make sure they don't
stick in tests forever, even after the issue was already properly
addressed.

That's actually a great point. I personally would like tickets to have one more field: "workaround" containing the address of a workaround in the format "path_to_the_file:line_number" or better even - a commit id of this workaround, so that once the ticket is resolved, we could easily find what to reverse.

Please don't add any more trac fields, there is too many of them already :-)
Keyword may serve better for now...

+1

new trac field for a few workarounds per year is not worth it.

Perhaps we could use pytest's expected fail (xfail) or skip marker. [1] It would prevent test from failing in the report and once the underlying issue is fixed, it will raise as an unexpected pass.

It could be used as a temporary solution, once the issue is fixed, we would remove the mark from the test. This would probably need some workflow to be defined for these cases.

[1]: https://pytest.org/latest/skipping.html

I write faster than I read. The issue at hand here is diffeerent use case. :)

--
Milan Kubik

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Reply via email to