These changes work as desired and expected, producing sys.path the same as when starting python from the activated virtual environment. I like it. I hope that it makes it to release.
Bill On Sat, Mar 31, 2012 at 3:03 AM, Graham Dumpleton <graham.dumple...@gmail.com> wrote: > I have made changes in mod_wsgi source code repository for 4.0 which > adds python-home option to WSGIDaemonProcess. > > The way it works is that if you do not specify WSGIPythonHome or > python-home option to WSGIDaemonProcess, then the Python installation > that mod_wsgi was compiled against will be used for everything. > > If you do not set WSGIPythonHome but set python-home option on a > specific WSGIDaemonProcess group, then just that daemon process group > will be initialised with that Python installation and everything else > will use the Python installation that mod_wsgi was compiled against. > > If you set WSGIPythonHome then it will instead be used for everything, > except for the case where a specific WSGIDaemonProcess directive has > an overriding python-home option. > > That it acts as an overriding thing is why went with python-home after > all, even though the Python shared library is always sourced from the > Python installation mod_wsgi was compiled against. > > Now, this only works this way if WSGILazyInitialization is left as its > default, which is On. If you turn off lazy initialisation for Python > interpreters, which you shouldn't, because it causes memory leaks in > Apache parent process due to Python not cleaning up after itself > properly, then any python-home options are ignored completely and will > always be Python installation mod_wsgi was compiled against, or any > specific one set by WSGIPythonHome. > > How does one use this, is a simple matter of doing the following. > > virtualenv --no-site-packages /some/path/venv > > Then set for a specific WSGIDaemonProcess directive, python-home to > /some/path/venv. This is the same value as you would get if you ran in > the interpreter: > > import sys > print sys.prefix > > except that virtual environment sys.prefix isn't necessarily > normalised properly. > > $ /some/path/venv/bin/python > Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) > [GCC 4.2.1 (Apple Inc. build 5646)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> import sys >>>> print sys.prefix > /some/path/venv//bin/.. >>>> os.path.normpath(sys.prefix) > '/some/path/venv' > > What this replaces in way things currently done is that previously you > would also create: > > virtualenv --no-site-packages /some/path/venv-baseline > > and set WSGIPythonHome to that directory, with python-path option to > daemon process group being then being set to site-packages directory > of the original virtual environment. > > /some/path/venv/lib/python2.6/site-packages > > Now you do not worry about the baseline virtual environment, nor set > WSGIPythonHome or python-path. Instead the only thing you need to do > is set python-home to the location of the virtual environment. > > Bill, as the person who asked for it, can you give it a good testing. :-) > > Graham > > On 31 March 2012 09:13, Graham Dumpleton <graham.dumple...@gmail.com> wrote: >> As explained in: >> >> http://code.google.com/p/modwsgi/wiki/VirtualEnvironments >> >> you would create a virgin Python virtual environment, with >> --no-site-packages if old virtualenv version. You would point >> WSGIPythonHome at that virgin Python virtual environment. It should be >> dissociated from site-packages of system wide Python. >> >> You then create Python virtual environments for each daemon process >> group, still with --no-site-packages, and set python-path option to >> the site-packages directory of them. >> >> So, multi layers as alluded to. >> >> If python-virtualenv option were implemented WSGIPythonHome and need >> to have virgin Python virtual environment layer would be skipped, and >> python-path replaced with reference to per daemon process group Python >> virtual environment root. >> >> When using the Python virtual environment with WSGIPythonHome as >> described above what specifically is still being inherited from >> site.py that is causing problems. >> >> The issue of sys.prefix is a contentious and confusing issue. For >> virtualenv as standard part of Python 3.X, my understanding is that >> sys.prefix might remain the parent Python installation and there will >> be a separate variable indicating the location of the virtual >> environment. >> >> Graham >> >> On 31 March 2012 08:58, Bill Freeman <ke1g...@gmail.com> wrote: >>> On 3/30/12, Graham Dumpleton <graham.dumple...@gmail.com> wrote: >>>> On 31 March 2012 05:55, Bill, KE1G <ke1g...@gmail.com> wrote: >>>>> Might there already be a way to have per process group PYTHONHOME >>>>> settings? >>>>> >>>>> And/or, please tell me that/why it wouldn't help. >>>>> >>>>> My purpose is to have several separate Django instances, >>>>> each serving in a separate VirtualHost, and each using a >>>>> separate virtualenv, all be run from a single apache. >>>>> >>>>> We are currently running 5 servers, a front end which does >>>>> the virtual hosting, proxying to the other four separate apaches >>>>> (same executable, separate configuration files, so linked against >>>>> the same libpython.so). >>>>> >>>>> I was very pleased to discover the WSGIPythonHome directive >>>>> recently, since now my sys.path looks correct (and my .wsgi >>>>> scripts are much simpler). >>>>> >>>>> My understanding of WSGIDaemonProcess is that it spawns a >>>>> separate process in which to run python interpreters, and the root >>>>> apache, then proxys for them. Further, I hope that if I put them in >>>>> separate WSGIProcessGroup, that I can arrange to have traffic for >>>>> a particular VirtualHost served by a particular process group (by >>>>> putting the WSGIScriptAlias, the WSGIDaemonProcess, and the >>>>> WSGIProcessGroup directives all within the VirtualHost element?). >>>>> >>>>> If all this is true, and if the python interpreter hasn't already been >>>>> started by the time the daemon processes are forked, then it >>>>> would be very nice to be able to set the PYTHONHOME env >>>>> according to which VirtualHost's WSGIScriptAlias will use the >>>>> daemon process (group). >>>>> >>>>> I did start to play with WSGIPythonHome, but discovered that it >>>>> is not allowed inside of a VirtualHost. Similarly, while there is a >>>>> python-path= option for WSGIDaemonProcess, there is not one >>>>> called, for example, python-home= . >>>>> >>>>> So: >>>>> 1. Would this scheme work if I could set PYTHONHOME per >>>>> daemon process group? >>>> >>>> It is not clear why it is needed when other mechanisms may achieve >>>> what you want. >>> >>> Pointer please? >>> >>>> >>>>> 2. Is there already a way to do this? >>>> >>>> No. >>>> >>>>> 3. Would adding a python-home= WSGIDaemonProcess option >>>>> requre major refactoring? >>>> >>>> With the way mod_wsgi used to work, which was that Python interpreter >>>> initialisation was done prior to fork of daemon processes, yes. >>>> >>>> By default it now only initialises interpreter after fork of process, >>>> so technically it may be possible, I would need to look. It would have >>>> to some how be disabled though when lazy initialisation of >>>> interpreters had been turned off for some reason. That is, old way of >>>> initialising in parent process before fork had been reenabled. >>>> >>>>> 4. Is this a bad idea even if possible? >>>> >>>> It could technically be useful, but using python-path to refer to the >>>> site-packages directory of a second Python virtual environment >>>> probably achieves the same result. >>> >>> Using python path doesn't quite achieve the same result, since the >>> various things that the root python's home's site.py has added to >>> sys.path are there (yes, I can remove them in the .wsgi script, but >>> that makes things messy and brittle) and sys.prefix doesn't point >>> to the virtualenv (though that probably doesn't matter). >>> >>>> >>>> A python-home would would sort of eliminate one level of Python >>>> installation, but not really, as the Python shared library is still >>>> going to be loaded on mod_wsgi being loaded into Apache. It therefore >>>> is still going to be sourced from the Python installation mod_wsgi was >>>> compiled for and so all Python virtual environments still need to be >>>> for same version of Python. >>> >>> I understand. All the virtualenv instances are already using the same >>> python (even though we are running 5 apache's, they all share the >>> same modules - only the configurations differ). >>>> >>>> To make it clearer that daemon process groups wouldn't be truly using >>>> everything from the virtual environment still, rather than >>>> python-home, could be python-virtualenv option and would be given root >>>> of the Python virtual environment. >>>> >>>> Graham >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "modwsgi" group. >>>> To post to this group, send email to modwsgi@googlegroups.com. >>>> To unsubscribe from this group, send email to >>>> modwsgi+unsubscr...@googlegroups.com. >>>> For more options, visit this group at >>>> http://groups.google.com/group/modwsgi?hl=en. >>>> >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "modwsgi" group. >>> To post to this group, send email to modwsgi@googlegroups.com. >>> To unsubscribe from this group, send email to >>> modwsgi+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/modwsgi?hl=en. >>> -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to modwsgi@googlegroups.com. To unsubscribe from this group, send email to modwsgi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
