Am 23.02.2018 um 17:59 schrieb Pjotr Prins:
> I may misunderstand how you are doing this, but I think it is going to
> cause problems. Python makes assumptions about the environment and
> virtualenv kinda redirects those. On top of that we have guix profiles
> and people who need to redirect PYTHONPATH (temporarily). Adding
> virtualenv into the mix will complicate things. Then there are people
> using virtualenv on top of Guix ...
I have to admit that my description was a bit terse. I understand your
concerns, but don't worry. This answer is a bit lengthy and you may read
all of the mail as the techniques are a bit complex.
0) I'm proposing to replace the wrapper scripts by a private virtual
environment.
1) This only effects the ways scripts are run, the package is still
available as normal.
2) The venv-hack I posted a few days ago is proposing something
different and is aiming to solve a different problem.
3) AFAIK virtualenvs can NOT be layered, they can only share the system
site-packages. This needs to be investigated further, but only effects
the venv-hack I posted a few days ago.
Re 1):
This only effects the way *scripts* are run. Instead of using a wrapper
script, this uses a private virtual env - for the script only! Any
package is still available as normal.
Instead of using the python-executable of the profile, the script uses
the one in its private virtual env. This will make the script being run
in its own environment, which we also could name "profile".
The generated layout is as follows:
/gnu/store/…-my-app-0.1
+ lib/python3.5/site-packages/my_package/__init__.py # unchanged
+ bin/my-app # no wrapper! uses private venv's python
+ share/guix-venv/my-app-0.1
+- pyvenv.cfg # tells bin/python this is a virtual environment
+- bin/python -> /gnu/store/…-python-3.5.3/bin/python
+- lib/python3.5/site-packages/my-app.pth # simulates PYTHONPATH
This comes down to something like
pyvenv-3.5 /gnu/store/…-my-app-0.1/share/guix-venv/my-app-0.1
/gnu/store/…-my-app-0.1/bin/pip install my-app-0.1.tar.gz
ln -s /gnu/store/…-my-app-0.1/share/guix-venv/my-app-0.1/bin/my-app \
/gnu/store/…-my-app-0.1/bin/my-app
ln -s
/gnu/store/…-my-app-0.1/share/guix-venv/my-app-0.1/lib/python3.5/site-packages/my-app
\
/gnu/store/…-my-app-0.1/lib/python3.5/site-packages/my-app
This solves several issues:
* - python-applications in one profile can use conflicting
python-packages, since each application's dependencies are enclosed
in this application's private environment
* no more problems since the scripts file-name is .my-app-real (see
e.g. https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26752)
* packages include references to their dependencies the gc can now
pick up (only valid if the package includes a script, though)
Re 2) and 3):
> I think the problem of mixing module versions has to be resolved
> through profiles. That should just work(tm).
Mixing different package-versions for the same python version can never
ever being solved by profile-means Python supports only a single package
version in sys.path. Python has this
pkg_resources.get_distribution()-stuff, but this never really took of
and was superseded by virtual environments.
> The problem of mixing interpreter versions can be resolved through
> profiles. Though there is the danger that people mix them into one
> profile.
Mixing versions of different Python version can by solved by the
venv-hack I posted a few days ago. This basically makes the profile into
a virtual environment. Thus there is no need for setting PYTHONPATH in
the profile anymore (we can even remove the search-path) and packages of
different Python versions will no longer be mixed.
> For this I suggest we tell Python2 to only use PYTHONPATH2. That way
> there is no interference. Python2 is being phased out (it is obsolete)
> and upstream should consider such a solution too.
No other distribution is using something like PYTHONPATH2, neither does
upstream. Upstream will never introduce such a heavy incompatible change.
Our problems are caused by misusing PYTHONPATH for collection the
system/profile site-packages. We could of course implement search-paths
GUIX-PYTHON-SITE-PACKAGE-2 and GUIX-PYTHON-SITE-PACKAGE-3 (instead of
PYTHONPATH) and change "site.py" to honor them. I'll look into whether
this or my venv-hack are a better solution, esp. regarding "stacked"
virtual environments.
--
Regards
Hartmut Goebel
| Hartmut Goebel | h.goe...@crazy-compilers.com |
| www.crazy-compilers.com | compilers which you thought are impossible |
