Bug#275379: apt-get: please support "satisfy"
On Sat, 24 Dec 2016 02:44:59 +0100 Johannes Schauer wrote: > theoretically, one should be able to set the apt configuration value > APT::Build-Essential to something other than build-essential or the empty > list. > Unfortunately, due to a bug in apt this is currently not possible DonKult is > working on it, though. Once this is fixed, we can used the "apt-get build-dep" > approach without installing build-essential. ... > But I think the best short term solution would be to have the > APT::Build-Essential variable fixed and then use "apt-get build-dep". FTR, this is fixed in apt 1.4~beta3 and stretch, shortened changelog: apt (1.4~beta3) unstable; urgency=medium [ David Kalnischkies ] * allow default build-essentials to be overridden -- Julian Andres Klode Thu, 05 Jan 2017 20:50:01 +0100 -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#275379: apt-get: please support "satisfy"
Hi pabs, Quoting Paul Wise (2016-12-24 02:39:35) > On Thu, 2016-12-22 at 15:01 +0100, Johannes Schauer wrote: > > this has the disadvantage of always pulling in the build-essential package. > For systems with build-essential already installed, that method would be > simpler and better IMO. theoretically, one should be able to set the apt configuration value APT::Build-Essential to something other than build-essential or the empty list. Unfortunately, due to a bug in apt this is currently not possible DonKult is working on it, though. Once this is fixed, we can used the "apt-get build-dep" approach without installing build-essential. > > This has the disadvantage that it installs a meta-package on the user's > > system. > > You could `dpkg --purge` it afterwards to avoid this. Right. > > We'll be using apt's EDSP interface. > > What are the apt version constraints for that? > > It would be great if this worked for apt from wheezy and later. The EDSP interface got introduced in 0.8.16~exp2 but there were multiple changes and fixes since then, so one would have to experiment to see where this solution works. > > What do you think? > > I'd really like to see this functionality integrated into apt but your > workaround sounds good. > > It is a bit too late for this work to reach stretch and I'd like to do > a last-minute upload to finalise cats for stretch, so please work on > this in the wip/install branch for now. Cool, thanks! > I'm also not sure if apt-satisfy functionality should be in apt or devscripts > or check-all-the-things or elsewhere? Ideally, we'd really get an "apt satisfy" command. But I think the best short term solution would be to have the APT::Build-Essential variable fixed and then use "apt-get build-dep". Thanks! cheers, josch signature.asc Description: signature
Bug#275379: apt-get: please support "satisfy"
On Thu, 2016-12-22 at 15:01 +0100, Johannes Schauer wrote: > this has the disadvantage of always pulling in the build-essential > package. For systems with build-essential already installed, that method would be simpler and better IMO. > This has the disadvantage that it installs a meta-package on the > user's system. You could `dpkg --purge` it afterwards to avoid this. > We'll be using apt's EDSP interface. What are the apt version constraints for that? It would be great if this worked for apt from wheezy and later. > What do you think? I'd really like to see this functionality integrated into apt but your workaround sounds good. It is a bit too late for this work to reach stretch and I'd like to do a last-minute upload to finalise cats for stretch, so please work on this in the wip/install branch for now. I'm also not sure if apt-satisfy functionality should be in apt or devscripts or check-all-the-things or elsewhere? -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#275379: apt-get: please support "satisfy"
Hi,
On Thu, 22 Dec 2016 15:01:44 +0100 Johannes Schauer wrote:
> On Mon, 17 Oct 2016 07:20:43 +0800 Paul Wise wrote:
> > * Use python-apt to do the resolution and emit `apt install pkg=1.2.3`
> > commands?
>
> I don't see a way to achieve this with python-apt. Even when manually
> iterating
> through the given dependency string and checking which package in the apt
> cache
> satisfies the dependency, the solution would not always be correct. Here is an
> example in which this approach would fail:
>
> Suppose there are the packages A, B, C and D. A depends on D and D conflicts
> with C. Now the wants the following resolved:
>
> 'A | B, C'
>
> With the algorithm sketched above, A and C would be picked for installation
> but
> the solution would be invalid because A and C conflict via D. To avoid this
> situation, the code has to traverse all dependency trees which would in turn
> lead to the implementation of a full-fledged resolver. Certainly this is not
> desired.
on IRC, juliank informed me that this approach is already implemented in
python-apt. Specifically, try to do:
$ python3 -c "import apt.debfile; a = apt.debfile.DebPackage();
a.satisfy_depends_str('nginx-light | apache2, nginx-full'); a.check();
print(a.required_changes)"
(['nginx-full'], [], [])
That result is clearly wrong because it failed to satisfy the first alternative
(nginx-light and nginx-full conflict).
On the other hand, if I hand the problem to the script I presented in my last
email, then I get a correct list including apache2 and nginx-full.
Thanks!
cheers, josch
signature.asc
Description: signature
Bug#275379: apt-get: please support "satisfy"
Hi, On Mon, 17 Oct 2016 07:20:43 +0800 Paul Wise wrote: > I too would like to see a satisfy subcommand that could be run like this: > > apt satisfy 'python3, pyflakes3 | pyflakes (>= 0.7.3), pyflakes3 | pyflakes > (<< 1.1.0-1)' > > The reason I want this is so that I can make check-all-the-things run apt > for checks that have missing commands and get the right packages installed > no matter what the distribution it is being run on. > > $ check-all-the-things --install apt --checks pyflakes3 > sudo aptÃÂ satisfy 'python3, pyflakes3 | pyflakes (>= 0.7.3), pyflakes3 | > pyflakes (<< 1.1.0-1)' > > Right now there are workarounds that can be used for this being missing: > > * create a debian/control file containing a single Build-Depends line (apt > > 1.2.2) this has the disadvantage of always pulling in the build-essential package. > * create a full Debian binary package containing the right Depends This has the disadvantage that it installs a meta-package on the user's system. > * Use python-apt to do the resolution and emit `apt install pkg=1.2.3` > commands? I don't see a way to achieve this with python-apt. Even when manually iterating through the given dependency string and checking which package in the apt cache satisfies the dependency, the solution would not always be correct. Here is an example in which this approach would fail: Suppose there are the packages A, B, C and D. A depends on D and D conflicts with C. Now the wants the following resolved: 'A | B, C' With the algorithm sketched above, A and C would be picked for installation but the solution would be invalid because A and C conflict via D. To avoid this situation, the code has to traverse all dependency trees which would in turn lead to the implementation of a full-fledged resolver. Certainly this is not desired. > The first two are easy but I'm not sure about the last one. I see no way to use python-apt for tapping into apt's own resolver. On IRC you also mentioned the possibility to use dose3. A solution using dose3 would look the following way: Use apt-get indextargets to get the list of Packages files apt uses and then use "apt-helper cat-file" to feed the information to dose-deb-coinstall as the background packages. Then as the foreground packages, pass a dummy binary package containing a Depends line that you want to satisfy. The tool will then output an installation set in deb822 format on standard output. While this will create a valid solution, it might not be a very good solution because the package selection does not take into account the packages that are already installed on the user's system and it also does not adhere to apt's pinning or its idea about installing real packages before virtual ones. In this email I want to create a fourth option. It takes the installed packages into account, uses the apt resolver and even respects pinning values. We'll be using apt's EDSP interface. Essentially the solution works by creating an EDSP file that represents the current state of the system and then adding a custom header and dummy binary package to it and feed all that to /usr/lib/apt/solvers/apt. The EDSP returned by the program is then the solution that can be mapped to package name/version/arch triplets which can be used to assemble an apt-get install command line. You can find my script attached. You use it like this: $ ./apt-satisfy "apache2 (>= 1.3.37) | httpd" apt install libaprutil1:amd64=1.5.4-3 libaprutil1-dbd-sqlite3:amd64=1.5.4-3 apache2-data:all=2.4.23-8 libaprutil1-ldap:amd64=1.5.4-3 apache2-bin:amd64=2.4.23-8 apache2:amd64=2.4.23-8 apache2-utils:amd64=2.4.23-8 apt-mark auto libaprutil1:amd64 libaprutil1-dbd-sqlite3:amd64 apache2-data:all libaprutil1-ldap:amd64 apache2-bin:amd64 apache2:amd64 apache2-utils:amd64 It prints two commands that you can directly execute. The first command installs the right packages and the second command marks the newly installed packages as automatically installed. The latter is to make sure that these packages which have no real origin (the user never directly requested a real package to be installed) can easily be autoremoved. In the context of check-all-the-things, marking the packages as auto will not even lead to an autoremoval because check-all-the-things Recommends them. What do you think? Thanks! cheers, josch #!/bin/sh set -eu depends=$@ edspfile=$(mktemp) solution=$(mktemp) trap "rm -r \"$edspfile\" \"$solution\"" EXIT # simulate installation of a random package - we choose apt because that has to exist # # we expect the dump solver to fail with exit 100 APT_EDSP_DUMP_FILENAME="$edspfile" apt-get --simulate install --solver dump apt >/dev/null 2>&1 && [ $? -eq 100 ] request=$(grep-dctrl -F Request '' -s Request -n "$edspfile") architecture=$(grep-dctrl -F Request '' -s Architecture -n "$edspfile") architectures=$(grep-dctrl -F Request '' -s Architectures -n "$edspfile") # generate the APT-ID of the dummy package from the largest id plus one
Bug#275379: apt-get: please support "satisfy"
Package: apt Followup-For: Bug #275379 Control: affects -1 + check-all-the-things I too would like to see a satisfy subcommand that could be run like this: apt satisfy 'python3, pyflakes3 | pyflakes (>= 0.7.3), pyflakes3 | pyflakes (<< 1.1.0-1)' The reason I want this is so that I can make check-all-the-things run apt for checks that have missing commands and get the right packages installed no matter what the distribution it is being run on. $ check-all-the-things --install apt --checks pyflakes3 sudo apt satisfy 'python3, pyflakes3 | pyflakes (>= 0.7.3), pyflakes3 | pyflakes (<< 1.1.0-1)' Right now there are workarounds that can be used for this being missing: * create a debian/control file containing a single Build-Depends line (apt > 1.2.2) * create a full Debian binary package containing the right Depends * Use python-apt to do the resolution and emit `apt install pkg=1.2.3` commands? The first two are easy but I'm not sure about the last one. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
