Bug#1070203: python3-pip: app silently lost in upgrade to Bookworm + pip3 lost track of the status (4 bugs)
Hi Manny (2024.05.02_19:34:03_+) > I’m a bit confused because “pip3 list” shows a list of 146 packages, > but not argostranslation. Why did all those other packages survive the > upgrade? I wonder if some of them are somehow managed by apt. Yes, probably. > > So, I'm afraid you're well out of the supported area of pip. > > Sorry. > > Is it necessary for aptitude full-upgrade to withhold information from > the user about package destruction or removal? Ideally users would > get a loud warning when actions are taken that are expected to impact > an installed package. If it’s a mission critical tool, users need to > be able to back out of the upgrade and assess the consequences. Anything you install without apt, in /usr/local, /opt, etc. is outside of apt's area of responsibility. It's up to you to manage these yourself. > I would also like to mention a fifth defect I just discovered: > > ⑤ argostranslate was only /partially/ removed. > > There are some big language files that were originally installed by > argostranslate. The argostranslate executable survived the upgrade but > not some of the modules it relies on, leaving it in a broken partially > existent state with no information given to the user. The language > packs remained in tact. I don’t know where on the filesystem they > live, but when I installed argostranslate again the previous language > packs were found and automatically available for use. They're probably there, just not importable in your new python. Have a look around in /usr/local. > The pip package manager has an uninstall procedure and since pip is > the manager of the argostranslate package, users rely on it to keep > track of the objects associated to the application. Pip is a far less expansive package manager than apt. It's only responsible for installing libraries and applications *within* a particular Python install. It doesn't try to do anything beyond that. Stefano -- Stefano Rivera http://tumbleweed.org.za/
Bug#1070203: python3-pip: app silently lost in upgrade to Bookworm + pip3 lost track of the status (4 bugs)
> If you upgrade from bullseye to bookworm, your python3 is upgraded from > 3.9 to 3.11. These are incompatible versions, and install libraries to > different paths (when you use pip3). > Anything installed with pip on 3.9 will not be importable in 3.11. Thanks for the explanation. That explains bug ① (“an app module was dropped/lost”). Apparently that bug cannot realistically be remedied, although perhaps the release notes should cover this topic. I’m a bit confused because “pip3 list” shows a list of 146 packages, but not argostranslation. Why did all those other packages survive the upgrade? I wonder if some of them are somehow managed by apt. There are still 3 other bugs. > So, I'm afraid you're well out of the supported area of pip. > Sorry. Is it necessary for aptitude full-upgrade to withhold information from the user about package destruction or removal? Ideally users would get a loud warning when actions are taken that are expected to impact an installed package. If it’s a mission critical tool, users need to be able to back out of the upgrade and assess the consequences. I would also like to mention a fifth defect I just discovered: ⑤ argostranslate was only /partially/ removed. There are some big language files that were originally installed by argostranslate. The argostranslate executable survived the upgrade but not some of the modules it relies on, leaving it in a broken partially existent state with no information given to the user. The language packs remained in tact. I don’t know where on the filesystem they live, but when I installed argostranslate again the previous language packs were found and automatically available for use. In my case, this happened to be a benefit as it saved me from the effort of refetching the language files. But it’s probably not an favorable general behavior for packages to be partially removed. Ideally the user should receive a warning about pending removal and an option for a clean removal or a partial removal. The pip package manager has an uninstall procedure and since pip is the manager of the argostranslate package, users rely on it to keep track of the objects associated to the application. > Can I suggest using pipx for installing applications instead? It > actually understands this problem and has a "reinstall-all" feature to > help you recover from Python minor-version upgrades. I really appreciate the suggestion. I gave it a try. It had several issues, but it could very well turn out to be the lesser of evils. After several hurdles pipx was able to install argostranslate in the end.
Bug#1070203: python3-pip: app silently lost in upgrade to Bookworm + pip3 lost track of the status (4 bugs)
Package: python3-pip
Version: 23.0.1+dfsg-1
Severity: normal
X-Debbugs-Cc: debbug.python3-...@sideload.33mail.com
In Bullseye, an app was installed as follows:
===8<
$ torsocks pip3 install argostranslate
$ torsocks pip3 install --log-file $logs_dir/pip3-argostranslategui.err --log
$logs_dir/pip3-argostranslategui.log --cache-dir /usr/local/tarballs
argostranslategui
$ torsocks argospm install translate-${lang1}_$lang2
===8<
It ran fine. Then an update to the latest Bullseye point release was
performed, followed immediately with an “aptitude full-upgrade”. A
full log of the upgrade was not kept, but I did note this conflict:
===8<
python3-virtualenv : Depends: python3-pip-whl but it is not going to be
installed
Depends: python3-setuptools-whl but it is not going to be
installed
===8<
It’s probably irrelevant because aptitude’s resolution resulted in a
functioning python/pip - but worth mentioning in case it matters. An
attempt to execute the app after the upgrade to Bookworm went like
this:
===8<
$ /usr/local/bin/argos-translate -v
Traceback (most recent call last):
File "/usr/local/bin/argos-translate", line 3, in
from argostranslate import cli
ModuleNotFoundError: No module named 'argostranslate'
$ pip3 list | grep -i argo
(no output)
$ pip list | grep -i argo
(no output)
===8<
It’s also a bug for pip3 to have lost track of the fact that it
managed the app. Running “pip3 list” should reveal the existence of
the app.
I’m told it would have been wise to have installed the app in a
virtual environment not in the system (likely accurate). But
nonetheless a needed app module should never be silently dropped
without informing the user in the very least. If a needed module
needed to be scrapped for some reason, ideally a signal would be sent
to the apt procedure to block the upgrade and inform the user of steps
needed.
Recapping, I see four bugs:
① (upstream) an app module was dropped/lost
② (upstream) the user was not informed about the loss
③ (debian) the anomaly failed to block aptitude from moving forward
④ (upstream) pip3 lost track of the fact it was managing the app
I did not apply the upstream tag because bug 3 above is debian
related. But this report should probably be seen by upstream devs as
well.
-- System Information:
Debian Release: 12.5
APT prefers stable-updates
APT policy: (990, 'stable-updates'), (990, 'stable-security'), (990,
'stable'), (500, 'oldstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.10.0-28-amd64 (SMP w/2 CPU threads)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages python3-pip depends on:
ii ca-certificates 20230311
ii python3 3.11.2-1+b1
ii python3-distutils 3.11.2-3
ii python3-setuptools 66.1.1-1
ii python3-wheel 0.38.4-2
Versions of packages python3-pip recommends:
ii build-essential 12.9
ii python3-dev 3.11.2-1+b1
python3-pip suggests no packages.
-- no debconf information
