Note: This reply is being sent to all 4 mailing lists that the original was, though I am only subscribed to the first 2 myself, for the benefit of completeness to the readers of the original.


As my initial reply, I'll bring up 2 recent discussions or Pugs changes that affect this, so please keep them in mind when continuing this thread.

1. First, I started a "making v6 test suite its own distro" thread on just perl6-compiler which overlapped a lot with the intent of your email, though went into some other details, and there were 4 replies:

* - me
* - Gaal
* - me
* - Gaal
* - Jerry

Aside from proposing v6::tests as a separate distro that implementation distros have as an external dependency, ...

In summary, I had proposed that unilateral SKIPs/TODOs etc would be eliminated, and that a language-level feature, eg, a global %FEATURES could be queryable at run time by programs, including the test suite, to see if the current implementation claims to correctly implement some features they want to use or test. A test suite would query this for a feature, and output a SKIP if it returns false, and tests the feature if it returns true, with PASS or FAIL.

At the very least, some language tests would have to be unconditional, such as the ability to reference a predefined hash, use basic conditionals, and strings, but once those are passing, it is possible to use %FEATURES to conditionalize everything else.

We realized that, while this should work well long-term, and should certainly work well for a framework written in a language that has interchangeable backends and common tests for all, like DBI (or my Rosetta), this may be too difficult to implement short-term for Perl 6 itself.

2. And so, within the last week, some commits to Pugs added the ability for a SKIP/TODO to hardcode in the tests that it is just for a particular version or version range of a particular implementation, so it would either skip or not skip based on the declared version of the interpreter, which would increment at release time. That is, unconditional TODOs are conditional TODOs.

It seems that this is the current plan to do what you suggest in the short term.

The short term would have the tests referencing specific implementation versions, and the long term would have the tests completely agnostic to what implementations or versions they run under.

-- Darren Duncan

Reply via email to