Hi all, Lately we've had regressions in our toolchain which were discovered quite late (too late in my opinion). Luckily these regressions are solved now, but this got me thinking: Why don't we have a testsuite yet which is able to catch these type of regressions? Especially now that Fedora's AutoQA [1] is also gaining more and more attention these days.
So I decided to write a small tool in C using GLib's GTester framework to create a testsuite which can be used to automate testing of the various components in the Fedora MinGW toolchain. This Fedora MinGW Testsuite tool can execute testcases of the following types: * Parse RPM code and verify the output (using rpmlib) * Compile a .c / .cpp file containing source code and (optionally) verify the output using wine * Custom script where the exit code and output can be verified Testcases can be easily created by writing a small .ini-style file which describes what needs to be tested, how it needs to be tested and what the expected output is. The full details about this file format and all other documentation can be found at [2]. At the moment the testsuite tool supports the MinGW toolchain as currently available in Fedora, the MinGW toolchain as used in RHEL 6 / CentOS 6 and the mingw-w64 toolchain which is currently residing in my testing repo. While building the testsuite tool you'd have to indicate which toolchain needs to be used for running the testcases. There are currently 9 different testcases implemented (of which 3 are mingw-w64 specific) but I hope to get this number up in the coming days. This is also where you can help! As new testcases can be added really easy (no programming experience required) and all areas of the toolchain can be tested all Fedora MinGW package maintainers should be able to come up with testcases for the packages they maintain. Right now all code resides on a Subversion server of mine, but if there's enough interest in continuing developing it I'm willing to migrate it to Fedora Hosted so it can be more easily maintained. For now you can send testcases to me (either direct or through this mailing list) and I'll add them to the Subversion repository. Ideas for testcases are also welcome. Once the testsuite contains more testcases we can also integrate it in AutoQA so the testsuite will automatically be executed when a Fedora MinGW package maintainer tries to push an update so we can spot regressions before the package hits the Fedora trees. Kind regards, Erik van Pienbroek [1]: https://fedoraproject.org/wiki/AutoQA [2]: https://fedoraproject.org/wiki/MinGW/Testsuite _______________________________________________ mingw mailing list [email protected] https://admin.fedoraproject.org/mailman/listinfo/mingw
