Bug#871656: apt-offline: Does not validate Packages or .deb files in bundle
Hello David, On Sun, 2017-08-20 at 17:51 +0200, David Kalnischkies wrote: > On Fri, Aug 18, 2017 at 04:33:01PM +0530, Ritesh Raj Sarraf wrote: > > Currently, our approach has a flaw. It completely misses to > > validate > > the Packages files. Instead, just after verifying the Release file, > > it > > assumes everything is clean and blindly copies the Packages files. > > You are hardly the only one with this problem – and even if you would > do > it 100% secure we as apt developers would probably not be 100% happy > about it as it means that /var/lib/apt/lists must be handled like > a public interface as in no changes to the filenaming or even bigger > changes to the storage (like e.g. compressing the files). Perhaps > from > the apt side we should implement something like "apt-helper > import-lists-directory" to provide a way out of this mess in the > longterm. > Yes. A helper tool like this would be really useful. > > > > We may not need this validation for .debs. > > You need to do this for debs as well. The quick test just works as > expected because the deb file has a different filesize than what is > expected and apt checks the filesize as apt can do it for free while > checking for file existance and so deletes "obviously" bad files > silently. > > > As a workaround for this part, I think (= haven't tried) you can > place > the deb files in partial/ – the download methods should pick up the > partial file and notice that it is already completely downloaded > without > doing online requests. The files will then take there usual way > through > the verifcation of checksums and end up in archives/ if everything is > fine. > Thanks for this information. I'll try this out. This delegation will help a lot. > That doesn't work for lists/ as Release files are always requested > from > an online source (as apt can't know if its complete or outdated > already) > and the other files tend to be no longer compressed & you can't be > sure > that if you compress it again, that you would get the same hash (as > e.g. > different versions of a compressor can generate different compatible > files). > Let me check on it. We download Packages file in a compressed format. The Release file does list the checksums for all these files. So my plan right now is to validate the downloaded file's checksum against the details mentioned in the Release file. -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Bug#871656: apt-offline: Does not validate Packages or .deb files in bundle
Hi, (Input from apt devs was requested on IRC, so here you go – please CC me if there is something you think I could help with. Note that I am not an apt-offline user nor do I know how it works; I have just read the package description) On Fri, Aug 18, 2017 at 04:33:01PM +0530, Ritesh Raj Sarraf wrote: > Currently, our approach has a flaw. It completely misses to validate > the Packages files. Instead, just after verifying the Release file, it > assumes everything is clean and blindly copies the Packages files. You are hardly the only one with this problem – and even if you would do it 100% secure we as apt developers would probably not be 100% happy about it as it means that /var/lib/apt/lists must be handled like a public interface as in no changes to the filenaming or even bigger changes to the storage (like e.g. compressing the files). Perhaps from the apt side we should implement something like "apt-helper import-lists-directory" to provide a way out of this mess in the longterm. Interesting might be to implement a local (http) proxy as you can make that work with every apt version, but that of course gives the user the wrong impression that files are downloaded from "somewhere" while in reality the proxy would just serve files from the bundle on request. [I am thinking about implementing both more or less for a while, but haven't made any actual progress and somehow doubt I will in a reasonable timeframe on my own. If someone wanted to pick it up I could probably help with reviews through] > We may not need this validation for .debs. You need to do this for debs as well. The quick test just works as expected because the deb file has a different filesize than what is expected and apt checks the filesize as apt can do it for free while checking for file existance and so deletes "obviously" bad files silently. As a workaround for this part, I think (= haven't tried) you can place the deb files in partial/ – the download methods should pick up the partial file and notice that it is already completely downloaded without doing online requests. The files will then take there usual way through the verifcation of checksums and end up in archives/ if everything is fine. That doesn't work for lists/ as Release files are always requested from an online source (as apt can't know if its complete or outdated already) and the other files tend to be no longer compressed & you can't be sure that if you compress it again, that you would get the same hash (as e.g. different versions of a compressor can generate different compatible files). Best regards David Kalnischkies signature.asc Description: PGP signature
Processed: Re: Bug#871656: apt-offline: Does not validate Packages or .deb files in bundle
Processing control commands: > tag -1 +confirmed Bug #871656 [apt-offline] apt-offline: Does not validate Packages or .deb files in bundle Added tag(s) confirmed. > severity -1 serious Bug #871656 [apt-offline] apt-offline: Does not validate Packages or .deb files in bundle Severity set to 'serious' from 'normal' > tag -1 -moreinfo Bug #871656 [apt-offline] apt-offline: Does not validate Packages or .deb files in bundle Removed tag(s) moreinfo. -- 871656: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=871656 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Processed: Re: Bug#871656: apt-offline: Does not validate Packages or .deb files in bundle
Processing control commands: > severity -1 normal Bug #871656 [apt-offline] apt-offline: Does not validate Packages or .deb files in bundle Severity set to 'normal' from 'serious' > tag -1 +moreinfo Bug #871656 [apt-offline] apt-offline: Does not validate Packages or .deb files in bundle Added tag(s) moreinfo. -- 871656: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=871656 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#871656: apt-offline: Does not validate Packages or .deb files in bundle
Control: severity -1 normal Control: tag -1 +moreinfo Hello Stuart, On Thu, 2017-08-10 at 23:17 +1000, Stuart Prescott wrote: > apt-offline claims to do gpg validation of the contents of the zip > file and > claims that this is an important thing for it to do. > > --allow-unauthenticated > Don't verify GPG signatures for the data to be installed > to APT. > Usage of this option is highly discouraged. > > However, it appears that apt-offline only verifies the GPG signature > on the > Release file. If that check passes, then it is assumed that all > referenced > resources (Packages files) are OK and apt-offline does not check that > the > hashes for the Packages files are indeed correct. Yes. We only check the Release file, which contains the checksum details for the Packages file, which in turn contains the checksum details for all data (.debs). > These Packages files are > then fed directly to apt. Once apt has been fed a manipulated > Packages file, > it will then trust the .deb packages that it refers to. > No. They aren't fed directly. We follow the same process that apt does. We sync them to the partial location and then apt do the verification. > One can take a zip bundle, decompress it, alter the Packages file and > the altered > file was no rejected by "apt-offline install bundle.zip". > > It seems that the existing GPG check of the Release file is rather > pointless > and gives a false sense of security validation. Either the bundle.zip > has been > securely handled all along and the GPG check is unnecessary, or > bundle.zip has > not been securely handled and it is incorrectly trusted. Let's take a deb example here: rrs@priyasi:~$ sudo apt upgrade [sudo] password for rrs: Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages were automatically installed and are no longer required: snap-confine ubuntu-core-launcher Use 'sudo apt autoremove' to remove them. The following packages have been kept back: inkscape libgsl2 The following packages will be upgraded: linux-libc-dev 1 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/1,331 kB of archives. After this operation, 24.6 kB disk space will be freed. Do you want to continue? [Y/n] n Abort. 15:51 ♒♒♒☹ => 1 rrs@priyasi:~$ apt policy linux-libc-dev linux-libc-dev: Installed: 4.12.8+-45 Candidate: 4.13~rc5-1~exp1 Version table: 4.13~rc5-1~exp1 100 100 http://deb.debian.org/debian experimental/main amd64 Packages *** 4.12.8+-45 100 100 /var/lib/dpkg/status 4.12.6-1 500 500 http://deb.debian.org/debian unstable/main amd64 Packages 4.11.6-1 900 900 http://deb.debian.org/debian testing/main amd64 Packages 15:51 ♒♒♒ ☺ rrs@priyasi:~$ cd /var/cache/apt/archives/ 15:51 ♒♒♒ ☺ rrs@priyasi:/var/cache/apt/archives$ ls -lh linux-libc-dev_4.13~rc5- 1~exp1_amd64.deb -rw-r--r-- 1 root root 1.3M Aug 17 01:24 linux-libc-dev_4.13~rc5- 1~exp1_amd64.deb 15:51 ♒♒♒ ☺ rrs@priyasi:/var/cache/apt/archives$ su -c "echo abc > linux-libc- dev_4.13~rc5-1~exp1_amd64.deb " Password: 15:52 ♒♒♒ ☺ rrs@priyasi:/var/cache/apt/archives$ ls -lh linux-libc-dev_4.13~rc5- 1~exp1_amd64.deb -rw-r--r-- 1 root root 4 Aug 18 15:52 linux-libc- dev_4.13~rc5-1~exp1_amd64.deb 15:52 ♒♒♒ ☺ rrs@priyasi:/var/cache/apt/archives$ sudo apt upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages were automatically installed and are no longer required: snap-confine ubuntu-core-launcher Use 'sudo apt autoremove' to remove them. The following packages have been kept back: inkscape libgsl2 The following packages will be upgraded: linux-libc-dev 1 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. Need to get 1,331 kB of archives. After this operation, 24.6 kB disk space will be freed. Do you want to continue? [Y/n] ^C 15:58 ♒♒♒☹ => 130 -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Bug#871656: apt-offline: Does not validate Packages or .deb files in bundle
Package: apt-offline Version: 1.7.2 Severity: serious Tags: security Dear Maintainer, apt-offline claims to do gpg validation of the contents of the zip file and claims that this is an important thing for it to do. --allow-unauthenticated Don't verify GPG signatures for the data to be installed to APT. Usage of this option is highly discouraged. However, it appears that apt-offline only verifies the GPG signature on the Release file. If that check passes, then it is assumed that all referenced resources (Packages files) are OK and apt-offline does not check that the hashes for the Packages files are indeed correct. These Packages files are then fed directly to apt. Once apt has been fed a manipulated Packages file, it will then trust the .deb packages that it refers to. One can take a zip bundle, decompress it, alter the Packages file and the altered file was no rejected by "apt-offline install bundle.zip". It seems that the existing GPG check of the Release file is rather pointless and gives a false sense of security validation. Either the bundle.zip has been securely handled all along and the GPG check is unnecessary, or bundle.zip has not been securely handled and it is incorrectly trusted. regards Stuart -- System Information: Debian Release: 9.1 APT prefers proposed-updates APT policy: (550, 'proposed-updates'), (500, 'stable-debug'), (500, 'stable'), (60, 'unstable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores) Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8), LANGUAGE=en_AU:en (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages apt-offline depends on: ii apt1.4.7 ii less 481-2.1 ii libpython2.7-stdlib [python-argparse] 2.7.13-2 ii python 2.7.13-2 ii python-magic 1:5.30-1 Versions of packages apt-offline recommends: ii debian-archive-keyring 2017.5 ii python-lzma 0.5.3-3 ii python-soappy 0.12.22-1 apt-offline suggests no packages. -- no debconf information
