Hello there, Damien and I uploaded soft/isocheck (http://svnweb.mageia.org/soft/isocheck/trunk/) several days ago. It tests for basic rule-based things in a built ISO (name, size, checksums, headers, and some contents so far) and can start to save a lot of time.
While it still needs improvements, it's ready for some action, right from the output of bcd. And it's up for review. Usage, dependencies are described in the README. Anyone can use it individually on its own copy of images, the person building ISOs first, as a task just after bcd (might be good to try it for alpha3). This could be used too as a filter in an automated build scenario: built ISOs are staged in a repository where a cron picks them one by one, runs isocheck tests, appends the log and stage them further for QA team (or a deeper automated test tool), unless there's a failure (and it notifies building person/team). Speaking of failure, should the script test fail at once at the first issue, or continue to test other potential issues? (for instance, an ISO may be too big and have misconfigured ISO headers and ..., etc.). So far, the script fails at the first failed assertion. The next step in automation would be misc's Viviane (https://www.mageia.org/pipermail/mageia-dev/2011-August/007539.html) and/or http://www.os-autoinst.org/ or other test assistant tool . That will help QA team and testers to focus on other, manual-only, categories of tests. Feel free to suggest additional tests for isocheck (and for the rest too)! Romain