Thank you _very much_ to all of you for your detailed responses and your patience: this has been like taking Pylons 101 (plus "the Apache you should have known before getting into this"). Using the proxy approach works just fine.
I am pasting below some notes I took to document what I did right and wrong, and links to several of your messages. Maybe this can be of some use to other newbies like me? (In addition to my ignorance, several of you pointed out correctly that I was getting lost because of the wide variety of options). As soon as we have our setup fully up and running, we will provide a complete document where we detail other stuff (how we deal with MPI, dependencies from R, etc). Best, R. P.S. Is there any "canonical way" of citing Pylons? We are writing a paper where we want to cite Pylons. ************************************** (This uses emacs org-mode.) ****************************************** Notes for how to deploy our apps. They run via Pylons and paster. What about more complex set-ups? Most of these notes come from the thread [[http://groups.google.com/group/pylons-discuss/browse_frm/thread/4bb9b20d4724d8ae?]] Starting on 2006-12-07 I tried to get Pylons and Apache to work together. Tried a bunch of things, unsuccessfully. (Some historical traces are available either on the current code base or in the repository, with names such as dispatch.cgi, asterias_disptatch.cgi and asterias_dispatch.fcgi, and sample_httpd.conf). I ask on pylons-discuss. Lots of extremely useful answers!!! First, I am obviously confused about how Pylons, Apache, FastCGI, SCGI, CGI, etc, all talk to each other. Now I see that the info is in the docs and wiki, but I just didn't see it (or read it right). Important background info in the messages (which are like a compressed "Pylons 101"): Jose Galvez's 2006-12-13: [[http://groups.google.com/group/pylons-discuss/tree/browse_frm/thread/4bb9b20d4724d8ae/12e938531d04af61?rnum=1&_done=%2Fgroup%2Fpylons-discuss%2Fbrowse_frm%2Fthread%2F4bb9b20d4724d8ae%3F#doc_7f4d89a8ce679d09]] James Gardner's 2006-12-13: [[http://groups.google.com/group/pylons-discuss/tree/browse_frm/thread/4bb9b20d4724d8ae/12e938531d04af61?rnum=1&_done=%2Fgroup%2Fpylons-discuss%2Fbrowse_frm%2Fthread%2F4bb9b20d4724d8ae%3F#doc_08e4e54d15682cb1]] Shannon -jj Behrens's, 2006-12-13: [[http://groups.google.com/group/pylons-discuss/tree/browse_frm/thread/4bb9b20d4724d8ae/5e184a674437f6f2?rnum=11&_done=%2Fgroup%2Fpylons-discuss%2Fbrowse_frm%2Fthread%2F4bb9b20d4724d8ae%3F#doc_723ce609ea1f2c3a]] Ben Bangert, on why having Apache before paster is good: [[http://groups.google.com/group/pylons-discuss/tree/browse_frm/thread/4bb9b20d4724d8ae/5e184a674437f6f2?rnum=11&_done=%2Fgroup%2Fpylons-discuss%2Fbrowse_frm%2Fthread%2F4bb9b20d4724d8ae%3F%5D%5D%3D%26#doc_723ce609ea1f2c3a]] On 2006-12-13 I thought it was for sure gonna be CGI. The most attractive thing here is that I would not need to start paster (i.e., I would not need to monitor it), FastCGI seems unwarranted in our case, and mod_python is not an option (using Apache 1.3) and SCGI I hade never heard of.However, I run into the problem I describe in: [[http://groups.google.com/group/pylons-discuss/tree/browse_frm/thread/4bb9b20d4724d8ae/12e938531d04af61?rnum=11&_done=%2Fgroup%2Fpylons-discuss%2Fbrowse_frm%2Fthread%2F4bb9b20d4724d8ae%3F#doc_797f64ff3f099bf2]] I ask again in the list, and I get a whole bunch of very useful answers. Including Ben Bangert, who now seems to be using Apache as proxy and paster in the back. So I decide to try using paster in the back, and Apache with mod_proxy. Instructions and details in this email by Jose Galvez: [[http://groups.google.com/group/pylons-discuss/tree/browse_frm/thread/4bb9b20d4724d8ae/5e184a674437f6f2?rnum=11&_done=%2Fgroup%2Fpylons-discuss%2Fbrowse_frm%2Fthread%2F4bb9b20d4724d8ae%3F%5D%5D%3D%26#doc_0367b653aa1d43cb]] A few things to watch out (from JJ's message): If you're proxying, you have to make sure that Paster gives a response to Apache within a reasonable amount of time, otherwise Apache will time out. For long running tasks like this, I like to return a response immediately that tells the user "Yeah, yeah, I'm going to do what you asked." Then I can do the processing in the background. Then, you can use AJAX to poll the Web server to see if it's ready to produce some output. I think we should be OK here, with our "checkdone" approach, that periodically has the server send the client a message saying "we are working on it, this page will be autorefreshed". Ben Bangert, on supervising the Paster processes ([[http://groups.google.com/group/pylons-discuss/tree/browse_frm/thread/4bb9b20d4724d8ae/5e184a674437f6f2?rnum=11&_done=%2Fgroup%2Fpylons-discuss%2Fbrowse_frm%2Fthread%2F4bb9b20d4724d8ae%3F%5D%5D%3D%26#doc_723ce609ea1f2c3a]] ) I should also note that since then, I've gone to a proxy setup. I have djb's supervise process running which ensures my paster serve process is always up and running (mostly following the how-to James Gardner put up on the wiki about supervise). It works wonderfully, and paster serve has been incredibly solid, to date it hasn't even crashed (which supervise would instantly restart). This solution is also rather resource efficient, when your app is loaded in mod_python, it typically means the apache process handling your request increasing in ram. So if you want to handle 100 simultaneous requests, under Apache pre-fork, you'll have 100 apache processes.... which means 100 Pylons apps loaded into ram. The link to the supervise stuff is: [[http://pylonshq.com/project/pylonshq/wiki/DaemonTools]] >From what I see, we could try to integrate this with the scripts that automatically check that MPI is up, that nodes are accessible, etc. Maybe better to integrate into our LVS setup? ------ I tried the proxy thing. Works just fine. In httpd.conf I add: #################### LoadModule proxy_module /usr/lib/apache/1.3/libproxy.so ProxyRequests off <VirtualHost adacgh.bioinfo.cnio.es> ServerAdmin [EMAIL PROTECTED] ServerName adacgh.bioinfo.cnio.es ErrorLog /http/Asterias-Pylons/log/adacgh_error.log TransferLog /http/Asterias-Pylons/log/adacgh_access.log ProxyPass / http://localhost:5000/adacgh/ ProxyPassReverse / http://localhost:5000/adacgh/ </VirtualHost> ########## and the asterias.ini file has: ####################3 [server:main] use = egg:Paste#http host = localhost port = 5000 [app:main] use = egg:asterias ########### and, sure enough, when I do: http://adacgh.bioinfo.cnio.es the thing works. On 12/14/06, Jose Galvez <[EMAIL PROTECTED]> wrote: > > Ramon Diaz-Uriarte wrote: > > On 12/13/06, Mike Orr <[EMAIL PROTECTED]> wrote: > > > >> On 12/12/06, Ramon Diaz-Uriarte <[EMAIL PROTECTED]> wrote: > >> > >>> On 12/12/06, Mike Orr <[EMAIL PROTECTED]> wrote: > >>> > >>>> On 12/12/06, Ramon Diaz-Uriarte <[EMAIL PROTECTED]> wrote: > >>>> > >>>>> I see that DocumentRoot is a directory where, I think, the pylons egg > >>>>> cannot have been installed; the eggs will install under either the > >>>>> system-wide site-packages or some other location, as provided by > >>>>> virtual python or workingenv. But, as far as I know, none of the > >>>>> Pylons' project file will be left under, say, > >>>>> "/var/www/example.com/htdocs" (as in the FastCGI doc from the wiki) or > >>>>> similar. Thus: > >>>>> > >>>>> a) are people making a sym link from, say > >>>>> ~/somewhere/lib/python2.4/my-project.egg/my-project/public to > >>>>> /var/www/example.com/htdocs > >>>>> > >>>>> b) are people just copying (or linking) just a few selected py files > >>>>> to /var/www/example.com/htdocs? If so, which ones and why not a) > >>>>> > >>>> Neither one. If you put Python files in the document root or symlink > >>>> to them, Apache will serve them as ordinary static files and users > >>>> will see your Python source code, not the webpages your application > >>>> produces. > >>>> > >>>> Instead, set up your application outside Apache's space, and adjust > >>>> Apache's config file (httpd.conf) to forward requests to your > >>>> application server. The easiest way is to use the HTTP server that > >>>> comes with Paste, and use Apache's mod_proxy to forward requests to > >>>> the other HTTP server. Something like this: > >>>> > >>>> ProxyPass / http://localhost:5000/ > >>>> > >>>> See http://httpd.apache.org/docs/1.3/mod/mod_proxy.html . > >>>> You may need ProxyPassReverse too. > >>>> > >>>> Alternatively, you can use FastCGI, SCGI, mod_python, etc. The > >>>> directives are different but again it's all in the Apache config file, > >>>> not in the document root. > >>>> > >>>> > >>> I think I understand what you are saying here, but it seems very > >>> different from what the above documents suggest (if I am understanding > >>> correctly). I see no mention of using ProxyPass (although it might be > >>> implicit). > >>> > >> I was wondering that too, but it's simply a case of that page not > >> mentioning that alternative. I would have added it myself but I want > >> to get some more experience with flup first so I don't put anything > >> incorrect in the wiki. > >> > >> > > > > Sorry, Mike, I am getting lost here: so the ProxyPass approach is an > > alternative to what they recommend in those pages? So can they use > > FastCGI (as in http://pylonshq.com/project/pylonshq/wiki/FastCGI) or > > plain CGI (as in > > http://pylonshq.com/project/pylonshq/wiki/CgiOnNoFrillsHostingSvc) > > without using ProxyPass? > > > > Don't they still need to have paster up and running? > > > > > > > Pylons is a wsgi compliant web framework, and as such it can be served > ans stared lots of ways. The way most of us do it is to use paster > because frankly Ian has made it really easy to run lots of wsgi > applications and middle ware together with his very nice code. If you > look in your development.ini file you will find the [server:main] > section which describes what server you want to run. If you run pasters > own http server then you can use = egg:Paste#http and serve your pylons > app directly on port 5000 (or what ever port you want). If you use > fastcgi then you would use = egg:PasteScript#flup_fcgi_thread which uses > the flup fastcgi server to serve your application. If you want to use > mod_python then you can ignore the [server:main] since the pylons > application will be served via mod_python. If you want to use > mod_python the wiki has good instructions on how to set that up and it > runs very nicely. If you don't use mod_python then you will have to > chose one of the servers in (http, fcgi, scgi, ajp). Assuming you are > using the http server on port 5000, then you would turn on mod_proxy in > apace and add the proxy pass code to your apache config so that your > users can get to your application. For example if you wanted your users > to get to you pylons app from say http://somehost.com/myapp then in your > apache conf you would have: > > ProxyPass /myapp http://localhost:5000 > ProxyPassReverse /myapp http://localhost:5000 > > From one of yor other emails it looks like you are trying to install > your pylons application under your httpdocs folder so that it is > visiable to apache. If this is correct you really don't need to do that > and its probably not what you want to do because you won't be running > the code in the same way as cgi runs and so apache really don't need to > know exactly where your files are. > > I think the main problem your having is that there are to many choices > (a problem that lots of new users have). I personally would either pick > mod_python or the paster http server with mod_proxy. Of those two I > think mod_proxy is the eaiser of the two to set up so I recommend that > for new comers. > Jose > > >> By the way, I'm using Apache only because my Pylons sites will have to > >> share the port with existing applications that are being served by > >> Apache. > >> > > > > Yes, same situation here. > > > > > > > >> Otherwise I'd consider one of the Python HTTP servers > >> instead. > >> > >> > > > > I am all for making my life simple here! > > > > Best, > > > > R. > > > > > >> -- > >> Mike Orr <[EMAIL PROTECTED]> > >> > >> > > > > > > > > > > > -- Ramon Diaz-Uriarte Statistical Computing Team Structural Biology and Biocomputing Programme Spanish National Cancer Centre (CNIO) http://ligarto.org/rdiaz --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
