ons 2009-10-07 klockan 16:46 +0200 skrev Gilles Dartiguelongue: > Le mardi 06 octobre 2009 à 20:38 -0600, Ryan Hill a écrit : > > Some packages, like dbus[1], have testing features that, while useful for > > developers and arch-testers, aren't something that should be foisted on > > users. Dbus' case is extreme, as it builds-in functions that are useful for > > unit testing, but result in an insecure and unstable package (I just > > "fixed" a > > bunch of testsuite failures i've been seeing in dbus-using packages by > > disabling USE=test). Other packages have testsuites that take an > > unreasonable > > amount of time to build/run (db, ppl, boost, that faad/faac one that takes > > six hours), are pretty much guaranteed to fail (gcc, binutils), have strange > > dependency quirks (can't run the tests unless the package is already > > installed, create circular dependencies), or a dozen other situations I > > can't > > think of right now. > > > > I'd like to propose a new USE flag, qa-test or a better name, to handle > > these > > cases in a consistent way. This would give us a way to differentiate > > between > > tests that everyone should run and tests that only devs and arch-testers > > would be interested in, making enabling FEATURES=test by default in a future > > EAPI a little more palatable. Use of this flag would be up to the > > maintainer, of course. > > while it might sound sane, I think this proposal covers too much cases, > most of which should actually be filled as bugs to the maintainers of > the packages for not fixing the testsuite (or not filling an upstream > bug) before commiting to the tree. > > For gnome ebuilds as someone commented out, the test failure rate is > quite stable, and we are slowly trying to get around them, or at least > not commiting ebuilds with new regressions in the testsuite. > > Use of RESTRICT="test" shouldn't be encouraged as it disables tests > completely while part of them might still work and be relevant. >
The problem comes with packages like dbus where the testsuit is pretty useful for ATs, but if dbus is merged with FEATURES="test" then other testsuits for other packages depending on dbus breaks, which AFAICS tells us that a dbus built with tests is pretty broken for general usage. Example: FEATURES="test" emerge -1q dbus && FEATURES="test" emerge -1q dbus-python and dbus-python will FAIL its testsuit. However FEATURES="-test" USE="-test" emerge -1q dbus && FEATURES="test" emerge -1q dbus-python and dbus-python will compile, test and install without problems. Now how do we ensure that the ATs still can test a package even under these conditions? Because as I see it RESTRICT is only for packages whose testsuits are broken by design and never meant to be run on a working system (yeah *drm*, I am looking at you). I think we first hand should decide over these packages, after a decision is made about these problems with MORE then a compile-time impact, we can start talking about less important problems (i.e. problems that do not break stuff) like how long time a testsuit is supposed to run. And FEATURES="test" IS sometimes a good thing, I learned this the hard way once when something broke grub for me, and one of the things that made my system unbootable was one of those things the grub testsuit checks for.
