On Tue, Dec 3, 2024 at 7:02 PM Miro Hrončok via python-devel < python-devel@lists.fedoraproject.org> wrote:
> Hello Pythonistas. > > > tl;dr I wonder if we should get rid of the last downstream-only patch in > Fedora's Python, > the one responsible for /usr/local/lib(64)/python... installation location. > The removal would bring us closer to upstream, but perhaps cause a > regression > for our users. > I propose to adapt PEP 668 (Marking Python base environments as > “externally > managed”) instead. > > > Details: > > On Python 3.13+ we only ship one downstream-only patch in Python. > That is a patch that is not yet upstreamable, isn't a backport etc. > > The patch is this: > > > https://src.fedoraproject.org/rpms/python3.13/blob/f39/f/00251-change-user-install-location.patch > (Deliberately linking f39 to make the link stable. Rawhide is currently > the same.) > > Or as a commit: > https://github.com/fedora-python/cpython/commit/ac3a9df5f8 > > We carry this patch in various forms for 7+ years. > It originated with > https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe > > The idea is this: > > - sudo pip install etc. installs to > /usr/local/lib(64)/python3.X/site-packages > - RPM packages are installed in /usr/lib(64)/python3.X/site-packages > - RPM installed software ignores the packages in /usr/local (unless they > opt > in not to) > - other software uses packages from both, giving preference to /usr/local > > There are several challenges to this approach: > > One of them is upgrading. Ideally if pyfoo 1 is installed via RPM, > sudo pip install --upgrade 'pyfoo>=2' should install pyfoo to /usr/local > while > leaving > the RPM-installed pyfoo 1 be. > In reality, pip is not ready for this and needs a patch as well: > > https://src.fedoraproject.org/rpms/python-pip/blob/f39/f/remove-existing-dist-only-if-path-conflicts.patch > This means that users who do sudo pip install --upgrade pip lose this > patch. > This also means that the fact that this appears to work in other Python > package > managers (such as uv) is probably purely accidental. > This behavior also tends to differ for RPM-packaged Python packages with > .dist-info and .egg-info metadata. > > Another challenge is the mechanism we decided to use for RPM packages to > ignore > /usr/local. > We use the -s flag in shebangs for this. That leads to problems like this: > "global pip installed modules not visible if user site dir is disabled" > https://bugzilla.redhat.com/2256190 > > We could probably solve this by introducing a custom flag instead of -s, > but that would make the patch even worse -- our RPM-packaged software > would not > even execute > without our Python with our patch. I don't want to do that. > > A result of this problem (which is shared with Debian-based distros) was > PEP 668 – Marking Python base environments as “externally managed” > https://peps.python.org/pep-0668/ > Except we cannot use it as is, because our install locations share a > common > stdlib location. > I described this at > > https://discuss.python.org/t/pep-668-marking-python-base-environments-as-externally-managed/10302/46 > and further. > > At this point, after 2 years of failure, I really don't know how to get > something from this that's upstreamable. > > I spoke about this at PyCon PL '22 as well: > https://www.youtube.com/watch?v=7SBH4PkbMhw > > --- > > Recently it occurred to me that perhaps we could: > > 1. drop the patches from Python and pip > 2. mark /usr/lib(64)/python3.X as externally managed (PEP 668) > > So the implication here is that pip will not be allowed to install a package in the system directories that RPMs are installed in? And declare sudo-pip-installing unsupported. (That means, it only works > with a > custom pip flag and if users nuke their systems by using that flag, they > shot > themelves and we cannot help them.) > Honestly I don't see this working and I believe that ship sailed a long time ago unfortunately. Way too many tutorials, books and even workshops recommend "sudo pip install" and before our patch we always had a constant stream of bugs filled about users nuking their systems that way. Personally, I think this would be the best solution: > sudo-pip-installing is bad and should never be done (no, not even in > containers). > > However, I realize that others might disagree. For them, I can offer pip > --break-system-packages. > > > If we want to do this, I think we need to do this on Python upgrade > boundary. > We don't want users sudo pip-installing packages with Python 3.13 on > Fedora 41 > only to find them > ignored on Fedora 42. > However, when we upgrade Python to 3.14 in Fedora 43, their packages would > still be broken anyway. > > -- > Miro Hrončok > -- > Phone: +420777974800 > Fedora Matrix: mhroncok > > -- > _______________________________________________ > python-devel mailing list -- python-devel@lists.fedoraproject.org > To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue > -- Regards, Charalampos Stratakis Senior Software Engineer Python Maintenance Team, Red Hat
-- _______________________________________________ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
