Hi Jason, Thanks for doing this. I haven't had a chance to look at the code in details myself, but from what I have seen it looks like a real improvement. Ideally, I'd like someone from our side (probably Giacomo T) to have a look at it. However, Giacomo is currently travelling and I have other pressing matters to tend to. :(
I have had a quick look at the code and it all looks reasonable. There are three things we need for CI integration: 1. Ability to separate build and test. 2. Ability to schedule individual (long-running) tests separately in a cluster environment. 3. Ability to get a JUnit XML file with per-test results. With the potential exception of 2, I think the new framework supports all of the above. We could, maybe, use the support for distributed test using the test server in the framework, but I'm a bit reluctant to deploy it for two reasons. 1) It requires credentials to be hard-coded. It would be much more convenient to use an auto-generated authentication file (with suitable permissions) in a shared file location. 2) Having a custom server is potentially very unreliable since the network tends to be noisy (e.g., port scanners, IDS software). The static allocation of a port will also be troublesome (e.g., when two servers run on the same node). In the current framework, we typically list the available tests and then create one cluster job per test. Once the tests complete, the CI framework aggregates all of the JUnit files (it also stores the pickle files for debugging) and generates a report. I'd like to be able to do something with the new framework. Is that possible? Cheers, Andreas On 05/07/2018 21:22, Jason Lowe-Power wrote: Hi all, I'm hoping that we can finalize the new testing framework, and I can push it in within the next week or so. All of the important changes have at least one +2, but I want to give everyone one last chance to object or suggest changes. I've been talking to Sean, and while there's more that could be cleaned up in ext/testlib, we think it's good as it is. At a minimum, this will give us a starting point. I'd like to get this in soon so I can start adding more tests to it in the coming weeks. The patches are linked below: https://gem5-review.googlesource.com/c/public/gem5/+/4880/7 https://gem5-review.googlesource.com/c/public/gem5/+/4881/7 https://gem5-review.googlesource.com/c/public/gem5/+/4421/12 https://gem5-review.googlesource.com/c/public/gem5/+/4882/13 https://gem5-review.googlesource.com/c/public/gem5/+/4883/13 https://gem5-review.googlesource.com/c/public/gem5/+/4422/13 https://gem5-review.googlesource.com/c/public/gem5/+/4423/13 https://gem5-review.googlesource.com/c/public/gem5/+/10121/4 Thanks, Jason _______________________________________________ gem5-dev mailing list [email protected]<mailto:[email protected]> http://m5sim.org/mailman/listinfo/gem5-dev IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
