On Sun, 12 Apr 2020 10:43:07 +0200 Agostino Sarubbo <a...@gentoo.org> wrote:
> In case of 'yes', the arch team member must compile with FEATURE="test" and > he > is allowed to block the stabilization in case of test-failure. > > In case there will be a test-failure there are two choices: > 1) The maintainer fixes the test failure; > 2) The maintainer does not want to take care, so he can simply remove the > blocker and set "src_test pass" to no. > > Both of the above are fine for me. > > Obviously, if there are already test-failure bugs open, the flag "src_test > pass" should be set to 'no' when the stabilization bugs is filled. > > I'm sure that this way would help to avoid wasting time on both sides. > > What do you think about? Agree mostly, but this problem is in two places. Ebuilds themselves need more mechanisms to communicate this, as if an ebuild has problematic tests, then they're likely to be problematic every time, and remembering to set the appropriate bugzilla flags becomes a weakness. In this vein, there's lots of similar inadequacies you find if you follow this thread. Like for instance, I maintain a lot of packages where the ability for the tests to work is *environment specific*, and there's ultimately different *kinds* of testing, and the audiences who run each varies. Take perl packages for instance. There's a few where the test suite *cannot* work, because they need either network IO or access to physical devices to function. But simply disabling tests because of this produces a gross weakness, because Perl code cannot even be considered to be compatible with the target perl until you actually execute it, because those incompatibilities are ultimately "compiler errors", and you only see them when you spin up a runtime, unlike code with a dedicated compile stage. Subsequently, I've taken to bolting on ebuild-internal hacks that simply enumerate the modules and forces a perl instance to try loading them, to at least provide *some* degree of assurance that the code *probably* works. But ultimately this means the way I see it, every ebuild could have *multiple* test targets, and could have a control variable indicating which targets to execute by default, and consumer knobs could expand, or reduce, the testing based on preference. We could even have something like say: TEST_TARGETS="+build integration" TEST_RESTRICT="network-sandbox? ( !root? ( integration ))" Things like say, mysql, which has a standard test suite that takes days to run, might be well served by having controls like this, so consumers who want to have *some* basic assurance code works, without paying the price for a comprehensive smoke, can do so, instead of opting to choose "tests are hard, lets not do them". Just I've been cautious about talking about this, because this is such a can of worms, and leads into the debate about TDEPEND, and how we stuff everything into USE flags... And I really would *hate* for testing control variables to be USE flags, where --changed-use rampantly introduces unnecessary havoc ( And the eventual fact will be there will be code that tries to depend on specific values of these test control flags, and code that wants to omit dependencies based on them, and portage complexity goes cray cray )
pgpctmi0XWqfF.pgp
Description: OpenPGP digital signature
