-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 skaller wrote: > I am curious to know Debian policy on testing. There is no official policy, but several people have been working on trying to put one together lately. See [1] and [2].
> Felix does include a test suite -- but it takes quite some > time to run. This will get worse as number of tests goes up, > and number of combinations of options to test goes up. Currently, I don't think its too long. For a compiler, I think there is a little more justification for running a lot of tests. Especially when it runs on so many architectures. > Furthermore, some new features require testing that begins to > look nasty -- for example testing sockets requires opening > sockets (we're using localhost .. but it isn't clear if even > that is really permitted), This should be ok. > and creation of pthreads > (which in event of a bug could run away on some platforms .. > and there is no point in tests that will never catch bugs, > so one has to assume all tests can and will fail). I believe the autobuilders are fairly good at cleaning up after packages... there are plenty of unusual things that can happen that I feel this is probably fairly well tested. For example, it watches the output and kills it if the package appears to be stuck. > Worse may be coming, for example testing killing of > an application may require launching another process > with sufficient authority which sends it a kill signal > (and could kill autobuilder by mistake .. :) Yet the > build scripts themselves are fairly arbitrary so there > is already a risk of this. Well, at least that would be detected quite quickly :) > > One technique might be a separate test package which > build-requires the binary package being tested. > However if the suite fails, the binary -- not the test > suite -- should be flagged as bugged: it isn't clear > the Debian autobuilder etc would handle all that properly. This kind of system is currently being discussed as a possibility for Debian packages. See the links below for the actual threads. > Even trickier .. bootstrapped systems, which build > depend on themselves .. :) Fortunately, these are fairly rare. > Any advice on how far to go in original package would > be of interest, and how to cope with 'extra testing' > since the need for that will certainly arise: there > have to be *some* limits on testing in original package, > but there should also be a way to automate testing > and bugging out of packages. Ideally these would be your full-blown test suite which gets run before your release a new version of the software. This would test regressions and ensure the program is producing the correct output. The tests run by a Debian autobuilder should probably be considerably fewer and only test things like basic functionality (i.e., does the program segv at startup?) - -Mike 1 - http://lists.debian.org/debian-devel/2005/12/msg01034.html 2 - http://lists.debian.org/debian-project/2005/11/msg00072.html 3 - http://lists.debian.org/debian-project/2005/11/msg00073.html -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFDvFBu7ZPKKRJLJvMRAmn6AJkBonXutuEhzCSkwvx6+MzZHkcREgCfZ6D3 QJrVwh9xVYCtOVuUTuwhwXQ= =1ue8 -----END PGP SIGNATURE----- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
