Bug#934657: apt: add interface between external downloaders (apt-offline/apt-zip) and apt
On Tue, 13 Aug 2019 13:12:26 +0200 Julian Andres Klode wrote: > ugh, no. verification is pretty complex, we don't want people to > reimplement it. FTR this is exactly what has happened with apt-offline due to: https://bugs.debian.org/871656 In that report David suggested `apt-helper import-lists-directory`, which sounds almost exactly what this bug report is about, except that it implies that it doesn't support importing .deb files. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#934657: apt: add interface between external downloaders (apt-offline/apt-zip) and apt
On Tue, 2019-08-13 at 13:12 +0200, Julian Andres Klode wrote: > I think it's the wrong solution though. The correct solution is to bundle > the offline system's state (/var/lib/apt, /var/lib/dpkg/status, /etc/apt), > copy it to the online system, and then calculate and download the > update/upgrade there, and copy /var/lib/apt and /var/cache/apt back. That means that you would basically have to port apt to various platforms where users could conceivably want to do the downloading. I'm assuming you aren't going to want to port apt to the BSDs, Windows, macOS, Apple OS9, Solaris, Android, iOS, Symbian, Huawei HarmonyOS etc, which could all conceivably be the downloader system for apt-offline. OTOH, all of these systems probably have some sort of wget-like command or other automatic way to download files (and maybe hash them). So if apt could export the things that need downloading, then apt-offline could convert that to the appropriate file formats and commands needed for downloading from almost arbitrary downloader platforms. > Hence I'd propose to have a bundle and an unbundle command or something. I wouldn't object to having this feature in addition to the one I propose, but I think bundle/unbundle is only going to be useful in situations where the downloader is running Debian/Ubuntu, which is probably rare in random Internet cafes in Pakistan for eg. > update is not possible. we don't know which files we'll have to > fetch, it depends on the server response. Hmm, apt-offline manages to do that just fine, it uses --print-uris > ugh, no. verification is pretty complex, we don't want people to > reimplement it. The verification would not be the final verification, that would be done by apt on the offline system. The verification done on the downloader system would mainly be a convenience, to avoid another round-trip in the case of a corrupted download etc. > Just stuff them into partial, and run install to make it recognize > the files are already downloaded. Does putting them into partial do verification? I think just stuffing into partial is the cause of bug #871656. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#934657: apt: add interface between external downloaders (apt-offline/apt-zip) and apt
Control: tag -1 -moreinfo On Tue, Aug 13, 2019 at 08:44:07AM +0800, Paul Wise wrote: > Package: apt > Severity: wishlist > X-Debbugs-CC: apt-offl...@packages.debian.org > Control: block 871656 by -1 > > For machines that are in a location with no Internet, apt-offline is a > semi-convenient way to perform updates, upgrades and installs. > > There are two situations where offline machines can occur: > > * systems in remote locations with no Internet access at all > * systems that are air-gapped and recieve only incoming data, no >outgoing data is allowed for security reasons. > > Unfortunately it was discovered that apt-offline does not check > signatures properly and the package was removed from Debian buster. > > https://bugs.debian.org/871656 > > In addition the interface that apt-offline uses for exporting the list > of files that should be downloaded is just the --print-uris option, > which I noticed only prints MD5 hashes when installing packages. > > It would be nice to resolve both of these issues properly by creating a > bidirectional interface between external downloaders and apt. I'm fine with having print uris fixed so it reproduces all hashes and/or exporting them via JSON hooks. I think it's the wrong solution though. The correct solution is to bundle the offline system's state (/var/lib/apt, /var/lib/dpkg/status, /etc/apt), copy it to the online system, and then calculate and download the update/upgrade there, and copy /var/lib/apt and /var/cache/apt back. Hence I'd propose to have a bundle and an unbundle command or something. > I suggest that such an interface should have these properties: > > * be usable with all commands, including update, install, upgrade etc > * allow the downloader to be run on any kind of system with Internet >access, including Windows/macOS/Android etc machines update is not possible. we don't know which files we'll have to fetch, it depends on the server response. > * allow the downloader to be as sophisticated or as dumb as needed > * tell the downloader what to download and what filenames to choose > * tell the downloader how to verify each download was correct, >including needed OpenPGP keys etc ugh, no. verification is pretty complex, we don't want people to reimplement it. > * optionally don't tell the downloader about local sources.list >transports like file:// cdrom:// copy:// since those probably won't >be available on the download system but in some circumstances they >could be if the sysadmins have set them up correctly > * some transports (mirror:// tor://) may need some special handling... That's pointless. We can't know what the downloader supports, so we can't ignore anything. We can tell it what to download, and that's about it. > > * allow imports of downloaded data from a directory, probably best to >leave it to apt-offline users to define how they transfer the data >to the import directory Just stuff them into partial, and run install to make it recognize the files are already downloaded. > * do verification twice, potentially once by the downloader (won't be >possible in all situations) and always by apt See above -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en signature.asc Description: PGP signature
Bug#934657: apt: add interface between external downloaders (apt-offline/apt-zip) and apt
Package: apt Severity: wishlist X-Debbugs-CC: apt-offl...@packages.debian.org Control: block 871656 by -1 For machines that are in a location with no Internet, apt-offline is a semi-convenient way to perform updates, upgrades and installs. There are two situations where offline machines can occur: * systems in remote locations with no Internet access at all * systems that are air-gapped and recieve only incoming data, no outgoing data is allowed for security reasons. Unfortunately it was discovered that apt-offline does not check signatures properly and the package was removed from Debian buster. https://bugs.debian.org/871656 In addition the interface that apt-offline uses for exporting the list of files that should be downloaded is just the --print-uris option, which I noticed only prints MD5 hashes when installing packages. It would be nice to resolve both of these issues properly by creating a bidirectional interface between external downloaders and apt. I suggest that such an interface should have these properties: * be usable with all commands, including update, install, upgrade etc * allow the downloader to be run on any kind of system with Internet access, including Windows/macOS/Android etc machines * allow the downloader to be as sophisticated or as dumb as needed * tell the downloader what to download and what filenames to choose * tell the downloader how to verify each download was correct, including needed OpenPGP keys etc * optionally don't tell the downloader about local sources.list transports like file:// cdrom:// copy:// since those probably won't be available on the download system but in some circumstances they could be if the sysadmins have set them up correctly * some transports (mirror:// tor://) may need some special handling... * allow imports of downloaded data from a directory, probably best to leave it to apt-offline users to define how they transfer the data to the import directory * do verification twice, potentially once by the downloader (won't be possible in all situations) and always by apt These properties should make it usable in these situations: * For remote locations, a bi-directional sneakernet can be used to perform arbitrary apt operations as needed. * For air-gapped locations, an online clone/chroot/container of the remote system can be used to download updates, apply them locally and then ship the tested updates to the remote system; maintaining two identical systems, one with the air-gapped data and one without. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
