Andrew C Aitchison <[email protected]> writes: > I have been reading The Not Rocket Science Rule Of Software Engineering > https://graydon2.dreamwidth.org/1597.html > which is: > automatically maintain a repository of code > that always passes all the tests > > Does https://github.com/OSGeo/gdal have a branch with this property ? > > If we do not yet, it should in principal be easy: > whenever HEAD of master passes all test, pull that to a current-working branch > (in practice we have 16 top-level tests, so would need a wrapper which says > whether they all passed or not).
In principle it's easy. In the real world, "passes all tests" is woefully underspecified because there are many of: operating systems operating system versions CPU types versions of each dependency (which themselves may or may not "pass all tests") whether or not grids are installed for proj probably something else I haven't thought of. In the general case, CI schemes cover a small fraction of the above. > A comment on the above blog post is: > Aegis has a delightful wrinkle on this: each changeset has to not just > pass all existing tests, but also come with a new test, independently > tracked and associated with the change, that must fail before the > change, and pass after it. > > That would be a useful addition too. It would lead to bugs not getting fixed because it becomes too much effort. Or maybe you're volunteering to write the tests for all the one-line bugfixes? And, it runs into the same issue: I recently submitted a PR -- which was merged -- that improved portability of mkgdaldist.sh. I don't think there were tests for that, and if there were CI doesn't happen on NetBSD anyway. Surely you don't mean it is reasonable to block such PRs on lack of testing. I do think that for changes that fix a real bug, it is nicer to have a new test case than not, and that changes shouldn't be merged if they cause test regressions on the set of systems CI runs on, plus reports. And if there is a bug, it's nice to add a new xfail test case to the tests, even before the fix. From watching gdal (and many other things as a packager), gdal is in very good shape this way. Tests seem to essentially always pass on GNU/Linux, and I have found some failures on NetBSD, now almost entirely resolved (a blend of test environment documentation and fixes to gdal). So I think things are actually in quite good shape, and the biggest issues are the proj grid-present issue and lack of official CI on minority platforms. Minority platform CI is important to everyone, because it tends to shake out bugs that remain latent on the platform most development happens, because of course bugs that manifest there are already fixed. I really don't mean to argue against "tests are a great thing and should exist" at all. Just that I think it's harder than it seems, and that gdal is already doing a great job.
signature.asc
Description: PGP signature
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
