Hi all, At the moment we run roughly 120 regressions, and divide them into quick and long somewhat arbitrarily. Anyone doing active development and using quick as their “quick” way of checking that nothing is broken has to wait more than 10 minutes for some of these regressions to finish, which seems a bit of a stretch. It turns out the actual regression run times follow an exponential distribution, ranging from a few seconds up to >10k seconds (almost 3 hours). I propose we also start using medium (mentioned in a few places), and use a slightly more structured approach in dividing them up into quick, medium and long.
Here is what I propose: Quick – anything below 180 seconds, resulting in roughly 40 regressions across all ISAs. The turn around for a quick regression run for NULL, ALPHA, ARM and X86 (what I would deem the minimum to run) should thus be below 5 minutes of wall-clock time. Note that there are plenty configurations not covered by this (o3, realview64 etc). Medium – anything above 180 seconds, but below 1800 seconds, also resulting in roughly 40 regressions. Long – anything >1800 seconds. With this split, quick could be used as part of any development, to get an indication that everything is ok. For a sensible coverage before posting any patch, quick and medium should do the job. The cronjobs we have running at the moment could thus do 'quick,medium' for the daily one, and 'quick,medium,long’ for the weekly one. Thoughts? Ideas? Additional comments? Thanks, Andreas -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590 ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782 _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
