PYTHONPATH issue analysis - part 3 (was: PYTHONPATH woes)

Hi, here is my third part of the analysis: Result == We can avoid all of the problems related to how Guix is using PYTHONPATH quite simple. This will work for virtual environments, too. Preliminary Proposal === To be able to install different minor versions of

PYTHONPATH issue analysis - part 2 (was: PYTHONPATH woes)

Hi, nex part of the analysis: Result === The venv-hack I posted a few days ago works as expected only for GUIX_PROFILE, but not for virtual environments. Preliminary Proposal === As it stands now, the venv-hack is not a valid solution. It may be the basis for another

PYTHONPATH issue analysis - part 1 (was: PYTHONPATH woes)

Hi, I'm going to analyse the PYTHONPATH issues systematically. This is part 1 of the analysis. Result === PYTHONPATH intented (by upstream) for adding site-packages.  PYTHONPATH elements are added in front of essential packages, while site-packages are added behind. This means the way we

Re: PYTHONPATH woes

Am 24.02.2018 um 11:44 schrieb Hartmut Goebel: >  you may read all of the mail as the techniques are a bit complex. No offense meant, please ignore the sentence. When I started writing the mail I though my explanation would be much more complex. -- Regards Hartmut Goebel | Hartmut Goebel

Re: PYTHONPATH woes

Am 23.02.2018 um 17:59 schrieb Pjotr Prins: > I may misunderstand how you are doing this, but I think it is going to > cause problems. Python makes assumptions about the environment and > virtualenv kinda redirects those. On top of that we have guix profiles > and people who need to redirect

Re: PYTHONPATH woes

On Fri, Feb 23, 2018 at 08:36:46PM +0100, Ricardo Wurmus wrote: > Patching Python 2 is still an option, but I’d like to explore (and > understand) upstream mechanisms first. I don't think it will be radical and something upstream can adopt. We can at least suggest it ;) > > With Ruby we have a

Re: PYTHONPATH woes

Pjotr Prins writes: > Then there are people > using virtualenv on top of Guix ... Pjotr brings up a good point: can virtualenvs be composed? Do users still have a way to override modules via PYTHONPATH or some other mechanism if we were to create virtualenvs by

Re: PYTHONPATH woes

Am 22.02.2018 um 16:30 schrieb Ricardo Wurmus: > (This wouldn’t help us much for wrapper scripts, though.) Attached please fins an approach for solving this issue. The basic idea is make the application/script use a python within a virtual environment. I nee to rethink some details and check

Re: PYTHONPATH woes

Hello, On Thu, Feb 22, 2018 at 9:42 PM, Hartmut Goebel wrote: > (synopsis "Python venc-hack for Guix") Looks like a typo... s/venc/venv/ -- Vincent Legoll

Re: PYTHONPATH woes

Hi, Am 22.02.2018 um 16:30 schrieb Ricardo Wurmus: >> re 2): As soon as Python2 and Python3 are installed in the same profile, >> we put both site-packages for both versions into PYTHONPATH, which >> obviously is wrong. >> >> $ PYTHONPATH= guix environment --ad-hoc python@2 python >> […] >>

Re: PYTHONPATH woes

Hi Ricardo, you are welcome, I've taken the challenge :-) I always have fun digging in the internals of Python - if I find time. Thanks a lot for the data. I started investigating it and this raised some more questions. Some behavior is curious. E.g. why does "python -c 'import statmodules'"

Re: PYTHONPATH woes

Hi Hartmut, I was hoping for you to show up and give your input. Thank you! I appreciate you taking the time. > re 1): This is extremely curious: Why is the older package imported, > where the newer one is expected to be first in path? > > To analyze this in detail, please make a copy of the

Re: PYTHONPATH woes

Hi, thanks for raising the PYTHONPATH issue. Looks like we have a major problem here. Am 20.02.2018 um 11:53 schrieb Ricardo Wurmus: > I don’t know why this happens. I find it puzzling that in this > particular case the user’s profile contains an *older* version of > statsmodels (0.6.1). The

Re: PYTHONPATH woes

On Tue, Feb 20, 2018 at 04:18:40PM +0100, Andy Wingo wrote: > In Guix we don't set LD_LIBRARY_PATH but we do set PYTHONPATH, so it's > not quite the same I don't think. Not exactly the same, but close enough ;). But it is clearer now that mixing is the problem. Ricardos .pth may be an option if

Re: PYTHONPATH woes

Hi Pjotr, > On Tue, Feb 20, 2018 at 11:53:54AM +0100, Ricardo Wurmus wrote: >> Would it be good to make the wrappers for Python scripts stricter and >> not accept any user-set PYTHONPATH? > > I think that is a bad idea. You need to be able to opt out. Also > people need to experiment with

Re: PYTHONPATH woes

On Tue 20 Feb 2018 16:01, Pjotr Prins writes: > On Tue, Feb 20, 2018 at 11:53:54AM +0100, Ricardo Wurmus wrote: >> Would it be good to make the wrappers for Python scripts stricter and >> not accept any user-set PYTHONPATH? > > I think that is a bad idea. You need to

Re: PYTHONPATH woes

On Tue, Feb 20, 2018 at 11:53:54AM +0100, Ricardo Wurmus wrote: > Would it be good to make the wrappers for Python scripts stricter and > not accept any user-set PYTHONPATH? I think that is a bad idea. You need to be able to opt out. Also people need to experiment with modules without

PYTHONPATH woes

Hi Guix, we have a couple of packages that provide scripts that depend on Python modules. We wrap them in PYTHONPATH to ensure that the correct Python modules are found at runtime. This is not enough. We don’t wrap them tightly enough; instead we allow for a user-provided PYTHONPATH value to