On the 0x24D day of Apache Harmony Alexey Varlamov wrote: > 2006/12/28, Alexander Kleymenov <[EMAIL PROTECTED]>: > > > BTW, Mikhail, are we going to integrate IR-level JIT regression tests > > > ([1]) into the *DRLVM regression testing infrastructure* in the > > > nearest future? > > > > Stop, guys, let's think. Regression test is a test checking for > > particular problem reported and fixed before. So regression test and > > JIRA report have one-to-one correspondence. What I see at HARMONY-1531 > > is a set of tests for Jitrino.OPT marked as "improvement", not as a bug. > > So, I think, these tests should go to different place, not to regression > > test suite. Right? > > Alexander, > I'm not expert in testing taxonomy, could you please share the full > picture - which tests go where and when/how intended to be run, etc? > > > > Are there any pending tasks to reach that point? IMHO, we should > > > integrate/commit it and let go for constant improvement. That's quite > > > enough for the start. (Not the highest priority, but gonnabe pretty > > > useful). Please, tell me if I am missing something here. > > > > I'm agree with you. Tests first! Any tests increase the quality > > of the product (if they do not do some intentional harm of course). > > I got an impression that great diversity of test suites may come into > conflict with this manifested goal - more frameworks to maintain, more > steps to launch/track various tests, scattered reports, more complex > desicion tree to add new tests, etc. > Leaving aside official certification tests, I personally do not really > care where from a test case is taken - from spec perseption of code > implementer or from real-life user problem, the essence point is that > it properly exercises particular bit of functionality. And further > grading IMO should be practical tradeoff between > importance/coverage/time to test, dividing pre-commit/pre-contribute > and nightly or weekly CC runs. > Ideally there should be just one entry point which takes in a > config/testbase to be tested and a single consolidated test status and > report.
"one entry point" for all tests sounds good -- Egor Pasko
