Bug#896698: deprecating needs-recommends
Control: tags -1 - wontfix Hi all, On 18-09-18 21:17, Antonio Terceiro wrote: > On Tue, Sep 18, 2018 at 08:53:05PM +0200, Paul Gevers wrote: >> On 18-09-18 17:21, gregor herrmann wrote: >>> On Mon, 17 Sep 2018 15:11:30 -0700, Antonio Terceiro wrote: Maybe we want autopkgtest to also support @recommends@ alongside @builddeps@? >>> >>> From a user's point of view, that sounds ok to me. >>> Updating a few bits of documentation and a handful of packages with >>> manual d/t/control shouldn't be a problem. >> >> On the one hand, I wonder what that is going to help. This is just >> changing semantics as it still means that autopkgtest needs to figure it >> out. The current situation is going to waste peoples time debugging why >> a test is a regression, just to find out that every tool is silent on >> not installing the recommends. On the other hand, it is a clearer >> statement and it means that we can explain and document what we really >> mean with it. >> >> @recommends@ or needs-recommends should mean that direct recommends of >> the binary packages from the package that provides the autopkgtest are >> test dependencies. Implementation wise, this would mean that we would >> add all recommends, whether the binary that recommends it is installed >> or not. >> >> If the feature to have recommends in the test dependencies isn't going >> away, how bad would it be if we change the behavior in a sustainable >> way? In my endeavors to fix this bug, I have missed the option to just >> add the sum of all the Recommends lines to the test dependencies, like >> we do with @builddeps@. What do people think? > > That would work as well, with the benefit that it would require changing > only autopkgtest and nothing else. And I think that adding the packages > in Recommends to test dependencies is compatible with the original > spirit of needs-recommends, which is "this test needs the recommended > packages installed", but does it in a way that actually works. dpkg is going to add all the test dependencies except ones own packages to the Testsuite-Triggers field [1]. That means that @recommends@ would enable britney2 to also be aware of this, so I actually like that route for that reason. For the record, I currently am of the opinion that we should fix this bug and move to a new declaration: @recommends@ in the test dependencies. Paul [1] https://bugs.debian.org/910734 signature.asc Description: OpenPGP digital signature
Bug#896698: deprecating needs-recommends
On Tue, Sep 18, 2018 at 08:53:05PM +0200, Paul Gevers wrote: > Hi, > > On 18-09-18 17:21, gregor herrmann wrote: > > On Mon, 17 Sep 2018 15:11:30 -0700, Antonio Terceiro wrote: > > > >>> So if needs-recommends vanishes, there will be at least dozens if not > >>> hundreds of perl packages which will immediately fail their > >>> autopkgtests. > >> > >> Yes. We will need to find a replacement before dropping needs-recommends > >> completely.> > > Thanks. > > I think autodep8 (which is doing the work for these perl packages) must > be fixed before we can remove needs-recommends. It should already add > the recommends to the test dependencies. (but see my comments below). > > >> Maybe we want autopkgtest to also support @recommends@ alongside > >> @builddeps@? > > > > From a user's point of view, that sounds ok to me. > > Updating a few bits of documentation and a handful of packages with > > manual d/t/control shouldn't be a problem. > > On the one hand, I wonder what that is going to help. This is just > changing semantics as it still means that autopkgtest needs to figure it > out. The current situation is going to waste peoples time debugging why > a test is a regression, just to find out that every tool is silent on > not installing the recommends. On the other hand, it is a clearer > statement and it means that we can explain and document what we really > mean with it. > > @recommends@ or needs-recommends should mean that direct recommends of > the binary packages from the package that provides the autopkgtest are > test dependencies. Implementation wise, this would mean that we would > add all recommends, whether the binary that recommends it is installed > or not. > > If the feature to have recommends in the test dependencies isn't going > away, how bad would it be if we change the behavior in a sustainable > way? In my endeavors to fix this bug, I have missed the option to just > add the sum of all the Recommends lines to the test dependencies, like > we do with @builddeps@. What do people think? That would work as well, with the benefit that it would require changing only autopkgtest and nothing else. And I think that adding the packages in Recommends to test dependencies is compatible with the original spirit of needs-recommends, which is "this test needs the recommended packages installed", but does it in a way that actually works. signature.asc Description: PGP signature
Bug#896698: deprecating needs-recommends
Hi, On 18-09-18 17:21, gregor herrmann wrote: > On Mon, 17 Sep 2018 15:11:30 -0700, Antonio Terceiro wrote: > >>> So if needs-recommends vanishes, there will be at least dozens if not >>> hundreds of perl packages which will immediately fail their >>> autopkgtests. >> >> Yes. We will need to find a replacement before dropping needs-recommends >> completely.> > Thanks. I think autodep8 (which is doing the work for these perl packages) must be fixed before we can remove needs-recommends. It should already add the recommends to the test dependencies. (but see my comments below). >> Maybe we want autopkgtest to also support @recommends@ alongside >> @builddeps@? > > From a user's point of view, that sounds ok to me. > Updating a few bits of documentation and a handful of packages with > manual d/t/control shouldn't be a problem. On the one hand, I wonder what that is going to help. This is just changing semantics as it still means that autopkgtest needs to figure it out. The current situation is going to waste peoples time debugging why a test is a regression, just to find out that every tool is silent on not installing the recommends. On the other hand, it is a clearer statement and it means that we can explain and document what we really mean with it. @recommends@ or needs-recommends should mean that direct recommends of the binary packages from the package that provides the autopkgtest are test dependencies. Implementation wise, this would mean that we would add all recommends, whether the binary that recommends it is installed or not. If the feature to have recommends in the test dependencies isn't going away, how bad would it be if we change the behavior in a sustainable way? In my endeavors to fix this bug, I have missed the option to just add the sum of all the Recommends lines to the test dependencies, like we do with @builddeps@. What do people think? Paul signature.asc Description: OpenPGP digital signature
Bug#896698: deprecating needs-recommends
On Mon, 17 Sep 2018 15:11:30 -0700, Antonio Terceiro wrote: > > So if needs-recommends vanishes, there will be at least dozens if not > > hundreds of perl packages which will immediately fail their > > autopkgtests. > > Yes. We will need to find a replacement before dropping needs-recommends > completely. Thanks. > Maybe we want autopkgtest to also support @recommends@ alongside > @builddeps@? From a user's point of view, that sounds ok to me. Updating a few bits of documentation and a handful of packages with manual d/t/control shouldn't be a problem. Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- NP: The Cranberries: Zombie signature.asc Description: Digital Signature
Bug#896698: deprecating needs-recommends
On Mon, Sep 17, 2018 at 10:50:01PM +0200, gregor herrmann wrote: > On Sat, 15 Sep 2018 15:40:33 +0200, Michael Biebl wrote: > > > On Thu, 30 Aug 2018 12:03:31 +0200 Paul Gevers wrote: > > > I have just pushed commit 9fade8dcb501250f704da9c77af1f8e6165dc941 that > > > documents that we want to remove the needs-recommends option as we > > > discussed in https://lists.debian.org/debian-ci/2018/06/msg00016.html. > > So I have this specific case where I found needs-recommends quite > > convenient: > > I have a package (firewalld) which has optional dependencies like ipset > > or ebtables. If those are not installed, firewalld will log a warning > > and continue with the functionality disabled. > > We're also using needs-recommends for perl packages / in > pkg-perl-autopkgtest. > > % autodep8 > Test-Command: /usr/share/pkg-perl-autopkgtest/runner build-deps > Depends: @, @builddeps@, pkg-perl-autopkgtest > > Test-Command: /usr/share/pkg-perl-autopkgtest/runner runtime-deps > Depends: @, pkg-perl-autopkgtest > > Test-Command: /usr/share/pkg-perl-autopkgtest/runner > runtime-deps-and-recommends > Depends: @, pkg-perl-autopkgtest > Restrictions: needs-recommends > > In practice, there's currently one test in the > "runtime-deps-and-recommends" department: syntax.t which, among other > things, does `perl -wc ' and is helpful to detect > syntax errors in code which is not used during build tests. > Cf. also https://perl-team.pages.debian.net/autopkgtest.html#syntax.t > > The reason we need the packages in Recommends is quite simple: there > are often modules in packages which are no core functionality, hence > their dependencies are not in Depends but in Recommends. > > Without needs-recommends we would > - need to exclude them from syntax.t (the machinery exists but it's > quite some work) > - lose the ability to test them. > > So if needs-recommends vanishes, there will be at least dozens if not > hundreds of perl packages which will immediately fail their > autopkgtests. > > (josch's count of 154 affected packages in > https://lists.debian.org/debian-ci/2018/06/msg00016.html > fails to take all the autodep8 packages into account.) Yes. We will need to find a replacement before dropping needs-recommends completely. Maybe we want autopkgtest to also support @recommends@ alongside @builddeps@? This way tests that use @recommends@ would *depend* on the packages in Recommends:, and thus fail if those become uninstallable. This is contrary to the current status quo, where `Restrictions: needs-recommends` doesn't actually ensure that the packages in Recommends are installed. signature.asc Description: PGP signature
Bug#896698: deprecating needs-recommends
On Sat, 15 Sep 2018 15:40:33 +0200, Michael Biebl wrote: > On Thu, 30 Aug 2018 12:03:31 +0200 Paul Gevers wrote: > > I have just pushed commit 9fade8dcb501250f704da9c77af1f8e6165dc941 that > > documents that we want to remove the needs-recommends option as we > > discussed in https://lists.debian.org/debian-ci/2018/06/msg00016.html. > So I have this specific case where I found needs-recommends quite > convenient: > I have a package (firewalld) which has optional dependencies like ipset > or ebtables. If those are not installed, firewalld will log a warning > and continue with the functionality disabled. We're also using needs-recommends for perl packages / in pkg-perl-autopkgtest. % autodep8 Test-Command: /usr/share/pkg-perl-autopkgtest/runner build-deps Depends: @, @builddeps@, pkg-perl-autopkgtest Test-Command: /usr/share/pkg-perl-autopkgtest/runner runtime-deps Depends: @, pkg-perl-autopkgtest Test-Command: /usr/share/pkg-perl-autopkgtest/runner runtime-deps-and-recommends Depends: @, pkg-perl-autopkgtest Restrictions: needs-recommends In practice, there's currently one test in the "runtime-deps-and-recommends" department: syntax.t which, among other things, does `perl -wc ' and is helpful to detect syntax errors in code which is not used during build tests. Cf. also https://perl-team.pages.debian.net/autopkgtest.html#syntax.t The reason we need the packages in Recommends is quite simple: there are often modules in packages which are no core functionality, hence their dependencies are not in Depends but in Recommends. Without needs-recommends we would - need to exclude them from syntax.t (the machinery exists but it's quite some work) - lose the ability to test them. So if needs-recommends vanishes, there will be at least dozens if not hundreds of perl packages which will immediately fail their autopkgtests. (josch's count of 154 affected packages in https://lists.debian.org/debian-ci/2018/06/msg00016.html fails to take all the autodep8 packages into account.) Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- NP: R.E.M.: Losing My Religion signature.asc Description: Digital Signature
Bug#896698: deprecating needs-recommends
Michael Biebl writes ("Re: deprecating needs-recommends"):
> I have a package (firewalld) which has optional dependencies like ipset
> or ebtables. If those are not installed, firewalld will log a warning
> and continue with the functionality disabled.
...
> Now, if needs-recommends is going to be deprecated/removed, I'll have to
> specify the Recommends twice: once in debian/tests/control and
> debian/control and I don't like this duplication as this is prone to get
> out-of-sync.
>
> Just wanted to provide a data point why it might be useful to keep
> needs-recommends.
I have found that in practice, debian/tests/control often has too much
stuff in it that needs to be kept in step with the rest of the
package, to handle this manually.
So instead, when things get nontrivial, I have a script to update
debian/tests/control. Well, really, I generate it from template(s)
and other information found in the source.
I think your case is just an example of this kind of problem.
Unlike d/control it is not a big problem d/t/control is not updated.
My approach to detecting a failure to rerun the d/t/control generator
is to have a test which runs the generator and fails if the output is
not identical to the current file. So not updating the test list is
itself a test failure.
I commit the resulting d/t/control to git. This is slightly ugly but
not a practical problem. In particular, any merge conflicts are
easily resolved by rerunning the generator.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Bug#896698: deprecating needs-recommends
Hi Paul On Thu, 30 Aug 2018 12:03:31 +0200 Paul Gevers wrote: > I have just pushed commit 9fade8dcb501250f704da9c77af1f8e6165dc941 that > documents that we want to remove the needs-recommends option as we > discussed in https://lists.debian.org/debian-ci/2018/06/msg00016.html. So I have this specific case where I found needs-recommends quite convenient: I have a package (firewalld) which has optional dependencies like ipset or ebtables. If those are not installed, firewalld will log a warning and continue with the functionality disabled. I want to ship two autopkgtests: One with recommends installed, in which case no errors and warnings are allowed in the log. One without recommends installed, in which case errors are not allowed in the log, but warnings are. Now, if needs-recommends is going to be deprecated/removed, I'll have to specify the Recommends twice: once in debian/tests/control and debian/control and I don't like this duplication as this is prone to get out-of-sync. Just wanted to provide a data point why it might be useful to keep needs-recommends. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#896698: deprecating needs-recommends
Control: tags -1 - patch Control: notforwarded -1 Control: clone -1 -2 Control: tags -1 wontfix Control: reassign -2 src:autodep8 Control: retitle -2 autodep8: don't use needs-recommends restriction Hi all, I have just pushed commit 9fade8dcb501250f704da9c77af1f8e6165dc941 that documents that we want to remove the needs-recommends option as we discussed in https://lists.debian.org/debian-ci/2018/06/msg00016.html. It also means I will not fix this bug and I hope in the foreseeable future we can drop this restriction. For autodep8, which uses this in examples and and the perl generator, this means that the script should get the list of Recommends and merge it with the test depends. That shouldn't be too hard and makes the current behavior more reliable already. I'll request a Lintian check to warn against the use of this restriction. Paul signature.asc Description: OpenPGP digital signature
