On the 0x24D day of Apache Harmony Geir Magnusson, Jr. wrote: > On Dec 28, 2006, at 5:31 AM, Egor Pasko wrote: > > > > >> So, I think, these tests should go to different place, not to > >> regression test suite. Right? > > > > Good point. > > > > Initially the intention was to call it JIT Unit Tests. But what can > > stop us from tracking regressions with tests based on this idea? > > > > Let's look at the variants where to include the framework: > > 1. smoke > > 2. c-unit > > 3. kernel > > 4. regression > > 5. harmony unit > > 6. forgot something? > > So, for the moment (4) seems most natural. Maybe, some tests to > > include into (2) and some to (4). I do not insist, just want to know > > your opinion for now. > > TO be clear, 5 is "classlibrary tests", which I think should remain > different.
yep. I just tried to recall all test suites we have (and show that it's pretty much for my small small brain organ) > c-unit are also different, because it's a totally > different testing framework (c-based). Well, actually, JIT IR-level tests are C-based too. Then, probably, c-unit is the best place? I think, it makes sense to combine *all* types of tests in a single framefork with unified top-level running mechanisms as well as reporting mechanisms. When we have them all together we can cut them in pieces based on importance/time-to-run/catching-rate. Some pieces to run as pre-commit some in CC, some even more rare on snapshots, etc. > Now, that leaves smoke, kernel and regression. > > Smoke and kernel should be able to be combined. Smoke just seems to > be tests where someone wanted to write their own testing framework. I like that > Regresssion - I do understand where Tim is coming from in later > messages in this thread, but I don't actually mind slapping tests > that show reported bugs into a separate category called "regression", > because they probably are more complicated than unit tests or even > general functional tests, and a maybe poke at weird corner cases or > effects of multiple interacting systems. It would be interesting to > see if our regression tests are finding things that unit tests should > have... > > geir > -- Egor Pasko
