Package: software-properties-common Version: 0.96.20.2-2.1 Severity: important X-Debbugs-Cc: rossboy...@stanfordalumni.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 This is a bit involved; it took several days of debugging to get to the bottom of it. After discussing severity I'll describe the initial symptoms, then present the results of my analysis, and finally present a possible fix. **Summary** When adding a key, add-apt-repository creates a number of keys and keyrings. At the last step it tries to import the wrong key and fails; it should import the key it exported. **Severity and Scope** The package provides a single command, add-apt-repository, and it is unable to add a repository for a ppa with Ubuntu's keyserver. I don't see anything that would limit the impact to the one ppa I tried to add, or even to ppa's, though it may depend on the format of the downloaded key. It seems quite likely this is behind the unresolved bug 843946 (from 2016). **Initial Symptoms** My original goal was to build dosemu2 from source. It is available in Ubuntu but not Debian. https://github.com/dosemu2/dosemu2/wiki/Building-From-Source describes the process, the first step of which is to run the ci_preq.sh script (https://github.com/dosemu2/dosemu2/blob/devel/ci_prereq.sh). The first line of the script is `sudo add-apt-repository -y ppa:dosemu2/ppa`. I executed that command with additional debugging in a root shell, without running the rest of the script. <terminal> # date; add-apt-repository -m ppa:dosemu2/ppa Fri 03 Jun 2022 11:44:22 AM PDT More info: https://launchpad.net/~dosemu2/+archive/ubuntu/ppa Press [ENTER] to continue or ctrl-c to cancel adding it START SOURCES.LIST: ## most entries omitted to save space # deb-src http://ppa.launchpad.net/dosemu2/ppa/ubuntu kinetic main END SOURCES.LIST gpg: keybox '/tmp/tmpcjw8ab77/pubring.gpg' created gpg: keyserver receive failed: Server indicated a failure </terminal> This failure is likely unrelated to the subject of this bug. The failing command was on gpg --import from keyserver.ubuntu.com. That import was only erratically successful. Here's one that worked: gpg --verbose --debug-level 7 --no-default-keyring --no-options --homedir /tmp/tmpc20sx87y --secret-keyring /tmp/tmpc20sx87y/secring.gpg --keyring /tmp/tmpc20sx87y/pubring.gpg --keyserver hkp://keyserver.ubuntu.com:80/ --recv 6D9CD73B401A130336ED0A56EBE1B5DED2AD45D6 Finally we arrive at the relevant problem for this bug. The python scripts of software-properties-common (invoked by add-apt-repository) execute 'apt-key --keyring /etc/apt/trusted.gpg.d/dosemu2_ubuntu_ppa.gpg add /tmp/tmpc20sx87y/pubring.gpg' which executes a series of steps, the last of which is gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/apt-key-gpghome.xVHD35b6q9 -v --check-trustdb --keyring /tmp/apt-key-gpghome.xVHD35b6q9/empty.gpg --keyring '/etc/apt/trusted.gpg.d/dosemu2_ubuntu_ppa.gpg' "-v - --batch --import /tmp/tmppr9n1vry/pubring.gpg" (The directory names don't match previous command because I'm mixing different runs). The result: gpg: no valid OpenPGP data found. gpg: Total number processed: 0 The error was repeatable, assuming the keyserver was responsive. It often wasn't. **Analysis** Earlier steps export the key, but the step just above (with gpg) does not use the exported key. This is likely the result of the fact that apt-key is invoked with pubring.gpg rather than the exported key. In fact, it doesn't seem *anything* uses the exported key. I presume it was exported to get it into a format that was usable. This would mean the failure arises from attempting to import a key in the wrong format. E.g., 852473 refers to this, in turn referencing 844724. I've put a fuller analysis of the program flow below. **Solution** I imported the previously exported key, export-keyring.gpg, instead of pubring.gpg. It worked. <terminal> # date; gpg -v --ignore-time-conflict --no-options --no-default-keyring - --homedir /tmp/apt-key-gpghome.xwyHmcOcK3 --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg.d/dosemu2_ubuntu_ppa.gpg -v --batch - --import /tmp/tmpc20sx87y/export-keyring.gpg Wed 08 Jun 2022 05:34:04 PM PDT # off=0 ctb=99 tag=6 hlen=3 plen=525 :public key packet: version 4, algo 1, created 1475850017, expires 0 pkey[0]: [4096 bits] pkey[1]: [17 bits] keyid: EBE1B5DED2AD45D6 # off=528 ctb=b4 tag=13 hlen=2 plen=25 :user ID packet: "Launchpad PPA for Dosemu2" # off=555 ctb=89 tag=2 hlen=3 plen=568 :signature packet: algo 1, keyid EBE1B5DED2AD45D6 version 4, created 1475850017, md5len 0, sigclass 0x13 digest algo 2, begin of digest aa 4b hashed subpkt 2 len 4 (sig created 2016-10-07) hashed subpkt 27 len 1 (key flags: 03) hashed subpkt 11 len 5 (pref-sym-algos: 9 8 7 3 2) hashed subpkt 21 len 5 (pref-hash-algos: 8 2 9 10 11) hashed subpkt 22 len 3 (pref-zip-algos: 2 3 1) hashed subpkt 30 len 1 (features: 01) hashed subpkt 23 len 1 (keyserver preferences: 80) subpkt 16 len 8 (issuer key ID EBE1B5DED2AD45D6) data: [4095 bits] gpg: pub rsa4096/EBE1B5DED2AD45D6 2016-10-07 Launchpad PPA for Dosemu2 gpg: writing to '/etc/apt/trusted.gpg.d/dosemu2_ubuntu_ppa.gpg' gpg: key EBE1B5DED2AD45D6: public key "Launchpad PPA for Dosemu2" imported gpg: no running gpg-agent - starting '/usr/bin/gpg-agent' gpg: waiting for the agent to come up ... (5s) gpg: connection to agent established gpg: Total number processed: 1 gpg: imported: 1 </terminal> As further confirmation, aptitude stopped complaining about an untrusted repository after this step. **Comments** Although the one substitution solved my immediate problem, there may be other places in the code, supporting other operations, that require a similar change. The unreliability of the ubuntu keyserver seems like a problem, but it's distinct from this one. The code is extremely difficult to debug, and the failures difficult to understand. Starting with add-apt-repository, one has python code, switches in the output stream (I guess to assure proper locale, but the python debugger seemed allergic to it), invocation of subprocesses. apt-key continues, invoking gpg and other programs, and even writing temporary shell scripts that invoke other temporary shell scripts (gpg.1.sh invokes gpg.0.sh, both written by apt-key). You also have to guess whether -debug1 or -debug2 will be more useful for apt-key since you can't do both--a fact that is not documented. So when there is a failure it is completely unclear what command failed or for what reason. Effective error reporting would need to flow all the way down to the scripts written by apt-key, since they are the ones that perform the actual commands. add-apt-repository's -m switch threatens/promises --massive-debug, but fell well short of that. apt-key is going away; this package should not use it, as already noted in 995895. **Appendix: Control flow** Here are some of my notes on the flow of control from add-apt-repository on down, including capturing the key commands that were executed. Indentation is nesting level, except for the actual shell commands executed which tend to be flush left. file:NN means the function is on line NN of file. There are undoubtedly many mistakes. Sequence of commands for add-apt-repository: from addkey_func() at SoftwareProperties.py:691 add_key ppa.py:384 AddPPASigningKey ppa.py:160 add_ppa_signing_key ppa.py:224 note the cleanup function is defined inside this one get_ppa_info retrieves info from URL, including the signing fingerprint tmp_keyring_dir = tempfile.mkdtemp() ppa.py:247 makes temp dir self._recv_key actually gets key ->ppa.py:165 verify_keyid_is_v4 just an integrity check call subprocess ppa.py:176 gpg --recv gives sources list ValueError: underlying buffer has been detached but then I can continue?! here's the command gpg --no-default-keyring --no-options --homedir /tmp/tmpc20sx87y --secret- keyring /tmp/tmpc20sx87y/secring.gpg --keyring /tmp/tmpc20sx87y/pubring.gpg - --keyserver hkp://keyserver.ubuntu.com:80/ --recv 6D9CD73B401A130336ED0A56EBE1B5DED2AD45D6 which prints gpg: key EBE1B5DED2AD45D6: "Launchpad PPA for Dosemu2" not changed gpg: Total number processed: 1 gpg: unchanged: 1 onwards self._export_key() ppa.py:257 -> 185 gpg --no-default-keyring --no-options --homedir /tmp/tmpc20sx87y --keyring /tmp/tmpc20sx87y/pubring.gpg --output /tmp/tmpc20sx87y/export-keyring.gpg - --export 6D9CD73B401A130336ED0A56EBE1B5DED2AD45D6 asked if I wanted to overwrite an existing file self._verify_fingerprint() -> 211 self._get_fingerprints -> 196 gpg --no-default-keyring --no-options --homedir /tmp/tmpc20sx87y --keyring /tmp/tmpc20sx87y/export-keyring.gpg --fingerprint --batch --with-colons why are we verifying a fingerprint we just got?? subprocess.check_output() on the previous command, whose output was 'tru::1:1654359237:0:3:1:5\npub:-:4096:1:EBE1B5DED2AD45D6:1475850017:::-:::scSC::::::23::0:\nfpr:::::::::6D9CD73B401A130336ED0A56EBE1B5DED2AD45D6:\nuid:-::::1475850017::0BB83C1AF9D4B9B057D465F9DEEA0D44E3CF1CC5::Launchpad PPA for Dosemu2::::::::::0:\n' fingerprint is extracted and returned back to line 267 pf add_ppa_signing_key which issues 'apt-key --keyring /etc/apt/trusted.gpg.d/dosemu2_ubuntu_ppa.gpg add /tmp/tmpc20sx87y/pubring.gpg' which is the step that failed. It does prepare_gpg_home makes tmp dir, e.g. + mktemp --directory --tmpdir apt-key-gpghome.XXXXXXXXXX + GPGHOMEDIR=/tmp/apt-key-gpghome.xVHD35b6q9 + chmod 700 /tmp/apt-key-gpghome.xVHD35b6q9 and then + gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/apt-key-gpghome.xVHD35b6q9 -v --check-trustdb --keyring /tmp/apt-key-gpghome.xVHD35b6q9/empty.gpg (written to gpg.0.sh and exec with additional args --batch --import) There is no argument to --import and the comment saying this is just to force creation of some files. warn_on_script_usage (apt-key deprecated) setup_merged_keyring dearmor file name same as original since .gpg create_new_keyring if it doesn't exist make next file and set mode /etc/apt/trusted.gpg.d/dosemu2_ubuntu_ppa.gpg then make gpg.1.sh: gpg.0.sh --keyring '/etc/apt/trusted.gpg.d/dosemu2_ubuntu_ppa.gpg' "$@" Note gpg.1.sh is not executed before function returns. But GPG is redefined to invoke it. aptkey_execute then does execute: + sh /tmp/apt-key-gpghome.xVHD35b6q9/gpg.1.sh -v --batch --import /tmp/tmppr9n1vry/pubring.gpg which means (directory names inconsistent from multiple runs) + gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/apt-key-gpghome.xVHD35b6q9 -v --check-trustdb --keyring /tmp/apt-key-gpghome.xVHD35b6q9/empty.gpg --keyring '/etc/apt/trusted.gpg.d/dosemu2_ubuntu_ppa.gpg' "-v - --batch --import /tmp/tmppr9n1vry/pubring.gpg" Why did it create export-keyring.gpg and then not use it? Why are there 2 keyrings? gpg: no valid OpenPGP data found. gpg: Total number processed: 0 - -- System Information: Debian Release: 11.3 APT prefers stable-security APT policy: (500, 'stable-security'), (500, 'stable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.10.0-14-amd64 (SMP w/8 CPU threads) Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages software-properties-common depends on: ii ca-certificates 20210119 ii gir1.2-glib-2.0 1.66.1-1+b1 ii gir1.2-packagekitglib-1.0 1.2.2-2 ii python-apt-common 2.2.1 ii python3 3.9.2-3 ii python3-dbus 1.2.16-5 ii python3-gi 3.38.0-2 ii python3-software-properties 0.96.20.2-2.1 Versions of packages software-properties-common recommends: ii packagekit 1.2.2-2 software-properties-common suggests no packages. -----BEGIN PGP SIGNATURE----- iQFSBAEBCgA8FiEEreS674/HIyV9gBfdnAYPmOsbK2AFAmKkIgIeHHJvc3Nib3ls YW5Ac3RhbmZvcmRhbHVtbmkub3JnAAoJEJwGD5jrGytgjsUH/03lTIvjoVmKSE55 R6rKEVhb5gHy4pRkycwse0+VTWTjD5o/21Ox2FSfSnbd2hVafNdpLE97V8BzqhPw xhIzNGd/6KdppOjoW1Lq6+z3A75JqsJeYA6JCIH4E9VZ69oCUJQZqOT49OSy0mAR hvGyP+qxoiz0AKC6kzcdPEd/dv5Rvp+zlMMgiWXN9S9UJilt3bzi2SxKaEprSAbI R0aiEFzF3p7u57BgGcMXICPsJGvUeMYalSk5ghyPpIOZrQ476g9DCbCwN9K9xlWO E2hFLnNdgQuYmXWh+XxLJmfUBr8/pj/htjKu3RLLrilFmLnAt37M6gC1pXSikyAZ Q52pGfg= =aIEH -----END PGP SIGNATURE-----
