On Thu, Jul 31, 2008 at 11:05:42PM +0100, Mel Gorman wrote:
> There are regression tests that fail for known reasons such as the binutils
> or kernel version being too old. This can be confusing to a user who reports
> a FAIL from "make func" only to find out it is expected. This patchset can
> be used to tell a user when a FAIL is an expected fail. It is broken up
> into three patches.
>
> The first patch introduces the means for calling a verification script. The
> second patch adds verification that an ELFMAP failure is due to an old
> version of binutils. The final patch runs a verification script of the
> version of ld for linkhuge tests instead of skipping them.
>
> There are other known failures that could be accounted for now such as
> mprotect() failing due to an old kernel. These can be handled over time.
Hrm. I certainly agree that we need better handling of the various
expected failures. But I'm finding this implementation kinda
confusingly complicated. So here's a counter-proposal for the design.
In fact, we already have a rudimentary mechanism for handling expected
failures: the CONFIG() macro basically means "failed because
something in the environment isn't suitable to run the test". It's
not very helpful about what's wrong with the environment, of course,
nor have I been terribly consistent with what's a CONFIG() and what's
a FAIL() in some cases.
Nonetheless, I think handling expected vs. unexpected failures within
the testcases themselves will be a better option. In cases where the
distinction is easy we can have separate FAIL_EXPECTED() - which would
give a reason - and FAIL_UNEXPECTED() macros. In other cases we could
simply have a FAIL() that branches off to some postprocessing code to
determine whether the failure is expected or not. This mechanism
would subsume the current handling of CONFIG().
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libhugetlbfs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
