2006/12/28, Alexander Kleymenov <[EMAIL PROTECTED]>:
> BTW, Mikhail, are we going to integrate IR-level JIT regression tests
> ([1]) into the *DRLVM regression testing infrastructure* in the
> nearest future?
Stop, guys, let's think. Regression test is a test checking for
particular problem reported and fixed before. So regression test and
JIRA report have one-to-one correspondence. What I see at HARMONY-1531
is a set of tests for Jitrino.OPT marked as "improvement", not as a bug.
So, I think, these tests should go to different place, not to regression
test suite. Right?
Alexander,
I'm not expert in testing taxonomy, could you please share the full
picture - which tests go where and when/how intended to be run, etc?
> Are there any pending tasks to reach that point? IMHO, we should
> integrate/commit it and let go for constant improvement. That's quite
> enough for the start. (Not the highest priority, but gonnabe pretty
> useful). Please, tell me if I am missing something here.
I'm agree with you. Tests first! Any tests increase the quality
of the product (if they do not do some intentional harm of course).
I got an impression that great diversity of test suites may come into
conflict with this manifested goal - more frameworks to maintain, more
steps to launch/track various tests, scattered reports, more complex
desicion tree to add new tests, etc.
Leaving aside official certification tests, I personally do not really
care where from a test case is taken - from spec perseption of code
implementer or from real-life user problem, the essence point is that
it properly exercises particular bit of functionality. And further
grading IMO should be practical tradeoff between
importance/coverage/time to test, dividing pre-commit/pre-contribute
and nightly or weekly CC runs.
Ideally there should be just one entry point which takes in a
config/testbase to be tested and a single consolidated test status and
report.
--
Alexey
