Thanks for this. It was easier to use this on both projects to get them interopting http://python-future.org/compatible_idioms.html
This is good to know if my hand is ever forced though. On Sunday, April 5, 2015 at 8:05:48 PM UTC-4, Graham Dumpleton wrote: > > Sorry for the slow reply. Has been far from a normal week this week. > > On 2 Apr 2015, at 11:40 pm, Justin Dearing <[email protected] > <javascript:>> wrote: > > > I have two flask apps. One is python 3, the other is python 2. I'd like > to run both through mod_wsgi on the same apache instance on the same > ip/port combo. Is this possible? > > So the basic limitation here is that the one running Apache instance can > only handle either Python 2 or Python 3 code. This is because the Python > interpreter itself is embedded inside of the Apache processes through > indirect dynamic linking of the Python interpreter shared library when the > mod_wsgi module is loaded into Apache. How library linking works means that > it isn’t possible to load two different versions of the same dynamic shared > library into the one process at the same time. > > The only solution therefore is to have two separate instances of Apache > running, one for Python 2 and one for Python 3 code. > > The issue is how one achieves that given that systems are generally set up > with only one Apache installation and for the normal user it is too large > of a challenge to work out how to use that with two separate configurations > and different builds of Apache modules such as mod_wsgi. > > Ignoring that latter point initially, lets consider that if one could > easily run two separate Apache instances, one with mod_wsgi for Python 2 > and another for Python 3, how would you even run them so they appear on the > same ip/port combo. > > Right now, the only way to achieve that is through the use of a front end > web proxy. > > What would normally be done is run a web server which accepts all requests > on the public ip/port combination and which then based on either the > virtual host name if separate sites, or based on sub URL if part of the > same host, forward the request on to a backend instance in which the Python > web application is running. > > As soon as one mentions a proxy, most people would run to nginx and use > that, but Apache itself can be used as a proxy as well since you would > already have that installed and setup to listen on the required ip/port > combination. Whatever proxy is used, the proxy would then normally forward > requests received to the separate instances of the Python web applications > running Python 2 and 3. > > For the case of Apache/mod_wsgi there is another option. Since it is also > already loading mod_wsgi, the primary Python web application can still run > within the main Apache and you only proxy requests which need to be handled > by the secondary application. > > In other words, the main Apache/mod_wsgi instance could be setup for > Python 2 and handle the web application requiring Python 2. Any requests > then destined for the Python 3 web application would instead be forwarded > onto a second backend instance on a separate port running that application. > > Whatever way you do it, you can’t therefore avoid proxying for at least > one of the Python web applications. Even if you were not using mod_wsgi, > this restriction would still apply due to the basic limitation that you > cannot run both a Python 2 and Python 3 application in the same process. > > The issue then is actually how can we easily run both a Python 2 and > Python 3 web application at the same time with Apache/mod_wsgi even if in > separate processes. > > At this point most people would give up and run one of the Python web > applications with gunicorn, running it separately and forwarding on > requests just for that one. This is because of the difficulty of running > multiple Apache instances from the one Apache installation. > > Yes, you could manually install a separate Apache installation with > mod_wsgi, but that is a pain. The better way at this point is actually to > use mod_wsgi-express. > > Where mod_wsgi-express is useful is that it allows you to install mod_wsgi > as part of a specific Python installation or virtual environment. You can > therefore install it separately into both Python 2 and Python 3 > installations. You can then run mod_wsgi-express out of each respective > Python installation and they will use that version of Python. > > Key in this is that it will use your existing Apache installation and it > will not touch or interfere with the system wide Apache configuration. > > The first option therefore is that you remove mod_wsgi from the main > Apache installation and instead configure it simply as a proxy which will > forward requests on to the two separate mod_wsgi-express instances. If you > want you could even disable the main Apache installation, but not uninstall > it, and run nginx as the front end proxy instead. You wouldn’t uninstall > it, as mod_wsgi-express wants to still use it, just not with the system > wide configuration. Instead mod_wsgi-express generates its own Apache > configuration automatically for the purpose it is being used. > > Now I keep mentioning mod_wsgi-express but haven’t actually said what it > is. If you haven’t been following my recent blog posts about it, then have > a read of: > > http://blog.dscpl.com.au/2015/04/introducing-modwsgi-express.html > https://pypi.python.org/pypi/mod_wsgi > > In short, mod_wsgi-express allows you to easily run multiple instances of > Apache/mod_wsgi using different configurations and different versions of > Python. > > So mod_wsgi-express solves the problem of how to run both Python 2 and > Python 3 web applications at the same time, albeit by using separate Apache > instances. > > As explained above, it is then just a matter of gluing them together using > a proxy. > > I am not going to go into specific configurations for setting up the > proxies here, but instead want too focus on one more specific issue. > > This issue is that as soon as you start talking about having multiple web > applications running as distinct instances, you need to deal with how you > get them started up when the system starts. > > Web servers such as Apache and nginx which you might use as a proxy are > usually already integrated into system startup scripts. The issue is how > you get your backend Python web applications running. > > Your choices here are to write the OS specific startup scripts for init.d, > monit, systemd or whatever system is used to manage the processes. > Alternatively, if supervisord is already setup as a secondary process > management system, you could use that instead. > > What I want to talk about here is another trick which doesn’t require you > to write any system startup scripts or use supervisord. > > What you would do is use your existing Apache/mod_wsgi installation to run > your Python 2 application. > > You would then use the daemon mode capabilities of mod_wsgi to define a > new daemon process group with a single process. This daemon process will be > defined so it is run, but you will never setup mod_wsgi to run a WSGI > application directly in it. That is, mod_wsgi itself would not use its own > internal protocols and mechanisms to forward requests to it. As far as > mod_wsgi is concerned it is an unused process. > > What instead we are going to do is use the fact that Apache/mod_wsgi will > manage that process and keep it running. That is, if that process is killed > it will be restarted. If Apache is restarted, the process will be restarted > and if Apache is stopped, the process will be shutdown. Thus we have a > simple process manager. > > What we can now do is tell mod_wsgi to import a specific Python script > file into that special daemon process group process at the time it is > started. In this Python script we can add code which will exec() a separate > application. That separate application will then be managed by Apache and > kept running. > > This separate process could be anything. We could even exec() an instance > of Redis or Memcached and Apache would manage it. > > In this case what we can do though is exec() an instance of > mod_wsgi-express which is installed for your Python 3 application. > > With the main Apache then managing that separate mod_wsgi-express > instance, you would then also setup the main Apache to proxy requests for > just the sub URL for that application, to whatever port the > mod_wsgi-express instance is listening on. > > To summarise, we have your existing main Apache installation still running > mod_wsgi but for Python 2. > > You install mod_wsgi-express against Python 3 and work out how to run it > for your Python 3 application. > > You use a dummy mod_wsgi daemon process group to exec() the Python 3 > instance of mod_wsgi-express for your application listening on a separate > port. > > The main Apache is then setup to forward requests for the Python 3 part of > your overall site, to the mod_wsgi-express instance. > > Now if you hadn’t worked it out, this actually means your main Apache > instance, is via the mod_wsgi daemon process, managing a separate instance > of Apache. This may sound scary, but it actually does work and is simpler > to setup than this description may suggest. > > So I leave it up to you absorb all that I have said and work out what you > might want to do. If you are interested in using mod_wsgi-express then try > experimenting with it and get it running with your Python 3 application. I > can then explain what you need to do to configure your existing Python 2 > Apache instance to run the managed mod_wsgi-express and setup proxying for > it properly. > > Graham > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
