Re: [gem5-dev] Test infrastructure improvements

2016-06-10 Thread Andreas Sandberg
Steve, I like the idea of adding random testers. This is something we have had on our internal wish list for quite a while. I think it’s orthogonal to the test infrastructure we use for nightly regressions though. I would like to keep the regressions framework as predictable as possible, a random

Re: [gem5-dev] Test infrastructure improvements

2016-06-09 Thread Steve Reinhardt
Hi Andreas, For the most part this all sounds pretty good. Thanks for all the effort. Just a few comments: - The scons support is very useful to make sure we don't re-run tests unnecessarily when binaries don't change. This is particularly important as long as we're running periodic (nightly,

Re: [gem5-dev] Test infrastructure improvements

2016-05-27 Thread Andreas Sandberg
Hi Jason, I think we do still need scons support. The test run script wasn’t really designed for running tests in parallel. It’s mainly designed to run one test to completion and create a result summary. This works great for both scons and our cluster environment since they both take care of job

Re: [gem5-dev] Test infrastructure improvements

2016-05-26 Thread Jason Lowe-Power
Hi Andreas, If I haven't said it before, thanks for all the effort you've put in here updating the regressions! I'm glad to see things moving in a positive direction. For the scons changes, I think I'll just trust you that they work ;). The SConscript files are pretty much inscrutable to me. My

[gem5-dev] Test infrastructure improvements

2016-04-28 Thread Andreas Sandberg
Fellow gem5 Developers, As a part of a larger gem5 infrastructure refresh within ARM (switch to git among other things), we have reworked parts of the test infrastructure. One of the main issues we have with the current test framework is that it is hard to integrate with our cluster and CI