Farid Zaripov wrote:
Perhaps we need to define the process for when a failure should
go in xfail.txt? I'd be inclined to mark a failure expected on
a given branch when it can't be fixed on that branch given the
release criteria (e.g., because it's incompatible). Would that
work for everyone?
As I understand the expected failure is failure, for which we have
unresolved JIRA issue and no matter in which release this issue is
planned to fix. When someone fix the issue he should remove
corresponding entry in xfail.txt. In ideal case the all failures at the
test results page should be an expected. Of course if some failure
can be fixed in one day (before the next nightly test results will be
obtained) there no needs to mark this failure as expected. IMHO.
So this definition doesn't doesn't distinguish between test failures
that can't be fixed due to compatibility restrictions from those that
can be fixed in a compatible way but that we haven't had time to fix
yet.
It occurs to me that tests that exercise bugs that can't be fixed in
a compatible way would be better dealt with by disabling them in code
using a suitable #if conditional. I.e., we don't need xfails for those.
The only purpose of xfails is then to mark up test failures that we
haven't gotten around to, like you said :)
Unless someone has a different opinion I'll update STDCXX-683 with
a link to Farid's definition in this discussion.
Martin
