[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
This bug was fixed in the package command-not-found - 20.10.2 --- command-not-found (20.10.2) hirsute; urgency=medium * Correctly handle compressed APT index files (LP: #1876034) -- Julian Andres Klode Thu, 23 Sep 2021 15:51:19 +0200 ** Changed in: command-not-found (Ubuntu Hirsute) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
This bug was fixed in the package command-not-found - 20.04.5 --- command-not-found (20.04.5) focal; urgency=medium * Correctly handle compressed APT index files (LP: #1876034) -- Julian Andres Klode Thu, 23 Sep 2021 15:51:19 +0200 ** Changed in: command-not-found (Ubuntu Focal) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Tags removed: verification-needed verification-needed-hirsute ** Tags added: verification-done verification-done-hirsute -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
I checked and the fix also works on Hirsute. To reproduce the issue (version 20.10.1): docker run --rm ubuntu:hirsute bash -c 'export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends command-not-found && apt-get update ; apt-cache policy command-not-found' To test the fix (version 20.10.2): docker run --rm ubuntu:hirsute bash -c 'echo "deb http://archive.ubuntu.com/ubuntu/ hirsute-proposed restricted main multiverse universe" > /etc/apt/sources.list.d/ubuntu-hirsute-proposed.list && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends command-not-found && apt-get update ; apt-cache policy command-not-found' -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Tags removed: verification-needed-focal ** Tags added: verification-done-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
The fix works for me. It's relatively easy to check for yourself with Docker. To reproduce the issue (version 20.04.4): docker run --rm ubuntu:focal bash -c 'export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends command-not-found && apt-get update ; apt-cache policy command-not-found' To test the fix (version 20.04.5): docker run --rm ubuntu:focal bash -c 'echo "deb http://archive.ubuntu.com/ubuntu/ focal-proposed restricted main multiverse universe" > /etc/apt/sources.list.d/ubuntu-focal-proposed.list && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends command-not-found && apt-get update ; apt-cache policy command-not-found' -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
With regards to the multiple changes the Launchpad diff was incorrect in that it was comparing a version of the package that is not in Hirsute. ** Changed in: command-not-found (Ubuntu Hirsute) Status: Incomplete => Fix Committed ** Tags added: verification-needed-hirsute -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Changed in: command-not-found (Ubuntu Focal) Importance: Undecided => Medium ** Changed in: command-not-found (Ubuntu Hirsute) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
The upload in the hirsute queue shows multiple changes outside of the fix for compressed files in db/creator.py, including extensive changes to the debian/ directory. Have these been included by mistake? ** Changed in: command-not-found (Ubuntu Hirsute) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
Hello Michael, or anyone else affected, Accepted command-not-found into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/command- not-found/20.04.5 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-focal. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: command-not-found (Ubuntu Focal) Status: New => Fix Committed ** Tags added: verification-needed verification-needed-focal ** Also affects: command-not-found (Ubuntu Hirsute) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
This bug was fixed in the package command-not-found - 21.10.0 --- command-not-found (21.10.0) impish; urgency=medium * Correctly handle compressed APT index files (LP: #1876034) -- Julian Andres Klode Thu, 23 Sep 2021 15:32:30 +0200 ** Changed in: command-not-found (Ubuntu Impish) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Also affects: command-not-found (Ubuntu Focal) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Description changed: - On all Ubuntu releases since at least 19.10, enabling compressed APT - package list storage with command-not-found installed leads to this - crash in the cnf-update-db hook called by apt update: + [Impact] + On all Ubuntu releases since at least 19.10, enabling compressed APT package list storage with command-not-found installed leads to this crash in the cnf-update-db hook called by apt update: Traceback (most recent call last): - File "/usr/lib/cnf-update-db", line 26, in - col.create(db) - File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 93, in create - self._fill_commands(con) - File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 127, in _fill_commands - self._parse_single_commands_file(con, fp) - File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 165, in _parse_single_commands_file - suite=tagf.section["suite"] + File "/usr/lib/cnf-update-db", line 26, in + col.create(db) + File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 93, in create + self._fill_commands(con) + File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 127, in _fill_commands + self._parse_single_commands_file(con, fp) + File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 165, in _parse_single_commands_file + suite=tagf.section["suite"] KeyError: 'suite' (To enable compressed APT list storage, add Acquire::GzipIndexes "true"; to an apt.conf fragment.) This appears to be caused by apt downloading an lz4-compressed command list for command-not-found and cnf-update-db passing the /var/lib/apt/lists/*Commands-*.lz4 files to db/creator.py, which does not know about the compression and attempts to parse them as plain text. + + + [Test plan] + Enable the option, purge old lists, and update to populate lists directory with compressed files; this will run cnf-update-db for you. + + [Where problems could occur] + Replacing the open() call with subprocess.Popen for apt-helper seems like a minimal change. If apt-helper has problems, we check the return code and fail, keeping the old db in place. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Also affects: command-not-found (Ubuntu Impish) Importance: Medium Status: Confirmed ** Changed in: command-not-found (Ubuntu Impish) Status: Confirmed => Triaged ** Tags removed: rls-ii-incoming -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Tags added: fr-1673 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
FWIW, Docker images do not install command-not-found and are intended for machine users, so it's not a place where you'd install it and notice that. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Tags added: rls-ii-incoming -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
Standard Docker images contain a file /etc/apt/apt.conf.d/docker-gzip- indexes with Acquire::GzipIndexes "true"; Acquire::CompressionTypes::Order:: "gz"; in it. So c-n-f is broken there. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
Actually, upon a closer look, it appears like command-not-found has been entirely unmaintained for more than 2 years now, so efforts to get this fixed upstream, let alone backported to focal, are probably futile. We should probably just remove this package from our base image and let the end user install some fixed fork of it from a PPA, if they need it. Luckily there's no hard dependency to it from ubuntu-standard. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
I'm aware of that, but that workaround defeats the point of enabling compressed list storage. We're using the patch linked above to solve the real issue without that space overhead, but such local fixes are a maintenance issue. Sadly there appears to be no intention to accept the fix upstream. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
As a temporary workaround, you can add to an apt.conf file: Acquire::GzipIndexes "false"; And do a check to ensure no files have GzipIndexes: grep -r GzipIndexes /etc/apt And then delete apt list cache rm -rf /var/lib/apt/lists And then finally update apt-get update -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
** Changed in: command-not-found (Ubuntu) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: command-not-found (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876034 Title: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
