[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Retargetting this bug at just the missing toml. Use LP: #1904945 for the dirty virtualenvs. ** Bug watch added: github.com/pypa/pip/issues #6120 https://github.com/pypa/pip/issues/6120 ** Also affects: virtualenv via https://github.com/pypa/pip/issues/6120 Importance: Unknown Status: Unknown ** Summary changed: - python3 virtualenvs include (broken) bundled pip deps + Ubuntu 20.04: virtualenv: pep517 requires toml, but it isn't bundled in venvs -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: Ubuntu 20.04: virtualenv: pep517 requires toml, but it isn't bundled in venvs To manage notifications about this bug go to: https://bugs.launchpad.net/virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
I think we can get the missing toml resolved. Agreed that that's really ugly that the virtualenvs include all of the pip deps. That was an issue in the early virtualenv 20 days, that took a while to get the necessary upstream support for cleaner de-vendoring. But messyness on its own can't justify broader changes to a stable release. The process for these changes is a Stable Release Update (SRU), see the policy linked below. We need to avoid breaking things for users of stable releases, once they are released. The change has to be proven to be worth the risk. I'll need some more concrete examples of the impact, to push through an SRU. And to show that the SRU resolves the issues. https://wiki.ubuntu.com/StableReleaseUpdates -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
The impact of the pep517 issue isn't too big of an issue for us- we can work around it (installing toml). The "impact" is that we have a test scenario which runs `pip install pep517` (reports already installed) and then tries to do `python -m pep517.build --binary --out-dir dist .` (which fails due to the toml issue). As mentioned, we can work around this by installing toml. I'm more worried about the broader issue in the report, that virtualenvs are dirty in the first place, this limits their utility :( -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
OK, so sounds like we're missing a toml wheel in python-pip-whl. What is the impact of this? It seems to happily install PEP 517 packages as-is. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Stefano, For the pep517 issue, try importing pep517.check or pep517.build: (foo) root@3edb3d8dca37:/# python3 -m pep517.build Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/foo/lib/python3.8/site-packages/pep517/build.py", line 6, in import toml ModuleNotFoundError: No module named 'toml' Upgrading it does indeed work fine, the main issue is that it's reported as being installed already but the installed version doesn't work. This is because it's bundled with pip (and gets injected into the virtualenv even though it shouldn't (the main reason for this bug report)), and this version of pip bundled conflicting versions of libraries. They bundled a pep517 version that wants toml, but bundled pytoml alongside it instead of toml. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Sorry, posted that from the wrong account, that comment was by me (the original bug reporter). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
** Changed in: python-virtualenv (Ubuntu Focal) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
> You will notice there are a plethora of extra packages in the virtual environment that should not normally be there, resulting in a dirty virtual environment. Looking at if this can be changed for Ubuntu 20.04. I'm afraid it may be complex. I think we may need to patch pip to be able to apply these changes to virtualenv. > To make matters worse, the latest release of pip bundles incompatible versions of libraries. The net result is that `pip install pep517` will show that it is already installed, and but `import pep517` will result in an ImportError. I can't reproduce that: 1. pep517 imports successfully. 2. pip can upgrade it, without error. root@actual-krill:~# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 20.04.1 LTS Release:20.04 Codename: focal root@actual-krill:~# python3 -m virtualenv foo && source foo/bin/activate created virtual environment CPython3.8.5.final.0-64 in 590ms creator CPython3Posix(dest=/root/foo, clear=False, global=False) seeder FromAppData(download=False, CacheControl=latest, appdirs=latest, certifi=latest, chardet=latest, colorama=latest, contextlib2=latest, distlib=latest, distro=latest, html5lib=latest, idna=latest, ipaddr=latest, lockfile=latest, msgpack=latest, packaging=latest, pep517=latest, pip=latest, pkg_resources=latest, progress=latest, pyparsing=latest, pytoml=latest, requests=latest, retrying=latest, setuptools=latest, six=latest, urllib3=latest, webencodings=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1.debian) activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator (foo) root@actual-krill:~# pip list Package Version - -- appdirs 1.4.3 CacheControl 0.12.6 certifi 2019.11.28 chardet 3.0.4 colorama 0.4.3 contextlib2 0.6.0 distlib 0.3.0 distro1.4.0 html5lib 1.0.1 idna 2.8 ipaddr2.2.0 lockfile 0.12.2 msgpack 0.6.2 packaging 20.3 pep5170.8.2 pip 20.0.2 pkg-resources 0.0.0 progress 1.5 pyparsing 2.4.6 pytoml0.1.21 requests 2.22.0 retrying 1.3.3 setuptools44.0.0 six 1.14.0 urllib3 1.25.8 webencodings 0.5.1 wheel 0.34.2 (foo) root@actual-krill:~# python Python 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pep517 >>> (foo) root@actual-krill:~# pip install pep517 Requirement already satisfied: pep517 in ./foo/lib/python3.8/site-packages (0.8.2) (foo) root@actual-krill:~# pip install -U pep517 Collecting pep517 Downloading pep517-0.9.1-py2.py3-none-any.whl (18 kB) Collecting toml Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) Installing collected packages: toml, pep517 Attempting uninstall: pep517 Found existing installation: pep517 0.8.2 Uninstalling pep517-0.8.2: Successfully uninstalled pep517-0.8.2 Successfully installed pep517-0.9.1 toml-0.10.2 (foo) root@actual-krill:~# lsb_release -a^C (foo) root@actual-krill:~# python Python 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pep517 >>> Am I missing something obvious? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Still waiting for this bug to be resolve in Ubuntu 20.04.1 Focal Fossa. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Almost half a year now and the LTS version is still not fixed? This is sad, because it keeps Ansible from bringing out their PPA for focal. I guess most users switching to pip installations will never automatically apply released security patches for it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: python-virtualenv (Ubuntu Focal) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
** Also affects: python-virtualenv (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: python-virtualenv (Ubuntu Groovy) Importance: Undecided Status: Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
20.04 is an LTS release, so it's the next step in upgrading from 18.04 and all new server builds. Sadly, this fix did not make it into 20.04.1, according to the Release Notes. As mentioned by Kai, does "latest development version of Ubuntu." mean 20.10+ releases? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
For the record: This is still broken in Ubuntu 20.04 (Focal Fossa). It breaks Ansible being able to support 20.04 in their official PPA. Tracking ticket on the Ansible side: https://github.com/ansible/ansible/issues/69203 It would be awesome if someone with the time and energy would do the work suggested by Kai to get this fix into a Focal Fossa update as well, so we can get these packages working there. Otherwise, people on Focal Fossa are bound to run the repo-provided Ansible packages (i.e. some older version). ** Bug watch added: github.com/ansible/ansible/issues #69203 https://github.com/ansible/ansible/issues/69203 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Thank you for taking the time to report this bug and helping to make Ubuntu better. However, I am closing it because the bug has been fixed in the latest development version of Ubuntu. If you need a fix for the bug in previous versions of Ubuntu, please perform as much as possible of the SRU Procedure [1] to bring the need to a developer's attention. [1]: https://wiki.ubuntu.com/StableReleaseUpdates#Procedure ** Changed in: python-virtualenv (Ubuntu) Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
It would be great to get this fixed as more people are upgrading servers and CI environments to Ubuntu 20.04 and starting to find weird edge cases related to this bug (besides not being able to install many popular packages from PPAs that haven't been updated like Ansible's). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Please backport the fixes, this affects how we install our toolset at my company. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880749] Re: python3 virtualenvs include (broken) bundled pip deps
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: python-virtualenv (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880749 Title: python3 virtualenvs include (broken) bundled pip deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/1880749/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
