On Mon, 2013-03-11 at 11:37 +0000, Simon McVittie wrote: > DEP8 tests can work like this with sufficient support from the package's > upstream developer (telepathy-gabble-tests 0.17.x from experimental > works like this).
I looked this up from here: http://packages.debian.org/experimental/telepathy-gabble So basically there's a manually-created subpackage named "-tests", and a .install file to install them, and a --enable-installed-tests configure option. But DEP8 doesn't cover this, so there's no established conventions, right? > Maemo's continuous integration system (CITA) enforced the approach you > advocate, and my experience from that system is that it requires > relatively intrusive changes to most packages to turn tests designed for > "make check" into something you can run like that. I hadn't heard of CITA before, so I did a bit of google searching, and the #3 result was this: http://maemo.gitorious.org/maemo-af/libcontentaction/blobs/a28157e74d0331f3b7388f5ca98f20ee2d84148f/t/lca-cita-test Which pretty clearly demonstrates what you're saying =) > That's fine if it's > something the package's upstream maintainer wants, but if they don't, > maintaining it as a distribution patch is pretty awkward. Right. That's obviously the tricky part - convincing a large enough subset of the audience here that they should support this. I don't want to carry patches (because that breaks continuous integration). While 'make check' is OK for "white box unit tests" (i.e. tests that call internal API), I'd say upwards of 80% of tests in GLib for example are black box, and could just as easily be run installed. If we can standardize conventions (--enable-installed-tests, install to /usr/share/$pkgname/tests), then jhbuild could be updated to be aware of them, and the new equivalent of "make && make check" could be: $ jhbuild make && jhbuild check glib Note that "jhbuild make" actually runs "make install". This is a bit alien to "package people" because it's ***really*** important to note that jhbuild operates on a sandbox, not your actual system. It'd obviously be very painful if you had to replace your system copy of glib to run the tests, and I'm sure the lack of this sandbox in the dpkg universe is why DEP8 is written the way it is. Anyways, now with installed tests we have advantages over "make check"; for example, the flexibility to run say the gjs tests after updating glib: $ jhbuild make && jhbuild check gjs Or in fact, just run all the tests against the current tree, without changing anything: $ jhbuild check Now, what would be quite tedious is supporting *both* uninstalled and installed, at least in the general case. So I do realize that I have an uphill battle here in some cases. But my plan still remains to just change say gjs to use installed tests at the start of the 3.10 cycle, implement a really basic "jhbuild check" (and also a more elaborate version in the gnome-ostree build system using qemu), and see how well it works. That said - I'm just going to add the authors of DEP8 to this discussion in the hopes of making it look more like "fully installed tests" (my proposal does kind of lack a name, and that's the best I can come up with at the moment...) _______________________________________________ desktop-devel-list mailing list [email protected] https://mail.gnome.org/mailman/listinfo/desktop-devel-list
