Comment #2 on issue 4164 by [email protected]: Make the status file
support filtering on variants or flags
https://code.google.com/p/v8/issues/detail?id=4164
Design 1 (continuation of #1 - but agreed with jkummerow that we shouldn't
implement this to keep status evaluation simple - keeping this for the
record only):
We add a lazy evaluation notion to the status file rules. E.g. a "lazy"
function that returns a lazily evaluated object. The evaluation is then
postponed until the flags of the matching tests are defined.
This might unfortunately create some unintuitive behavior, e.g.:
PASS, ['arch == x64 and lazy(\'\\'--use-strict\\' in flags\')', FAIL]
evaluates to False if arch == ia32.
PASS, [lazy(\'\\'--use-strict\\' in flags\') and arch == x64', FAIL]
evaluates to a lazy object.
An extra 'lazy' keyword wouldn't be required if we overwrite "in" and "not
in" in flags directly. That would simplify expressions to, e.g.:
PASS, ['arch == x64 and \'--use-strict\' in flags', FAIL]
We could also make aliases for some standard cases like:
strict ~= \'--use-strict\' in flags'
Dependent on when exactly we evaluate the lazy objects in the rule, we can
use different modifiers. E.g. if variants are already decided, we can't use
the variants keywords like NO_VARIANTS or FAST_VARIANTS. Also SKIP would
require an extra round of filtering in the end.
---------------------
Design 2 (simpler, but the scope would be restricted to strict/sloppy):
Add two new outcomes:
STRICT_FAIL ~= fails only in strict mode
SLOPPY_FAIL ~= fails only in sloppy mode
The status file evaluation and filtering wouldn't require any changes. The
test262 suite would check the flags in HasUnexpectedOutput after a test has
run and check if any of the new keywords is in the expected outcome.
Examples:
test1: [PASS, STRICT_FAIL]
Run sloppy and fails = failure
Run sloppy and passes = pass
Run strict and fails = pass
Run strict and passes = failure
Small drawbacks: This couldn't be combined with other test outcomes like
TIMEOUT only in strict mode. We could only add a STRICT and SLOPPY outcome,
but the semantics would be confusing. E.g. in FAIL, SLOW, STRICT, it
wouldn't be clear if the test is slow in strict mode only or if it fails in
strict mode only or both.
Maybe only SLOPPY_FAIL is required, looking at
https://codereview.chromium.org/1136553008/#ps40001
--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings
--
--
v8-dev mailing list
[email protected]
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
