[TurboGears] Re: Lighttpd as a simple proxy to TG
Thanks for all the posts in this thread, would be vitally useful to me. Some real novice questions: - flup is not available? Does it imply we can no more have SCGI/WSGI configuration? - As with LightTPD, will SCGI/WSGI be faster compared to simple proxying in NgineX? - Any tutorial/guidance on how to configure SCGI/WSGI with Nginex would help. - By This method will not work with the standard visit model..., I understood that we can still use the identity package, but with visit.on=False in app.cfg. Is it not so? Or, we can't use the identity package althgether? Sanjay --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
Sanjay wrote: - flup is not available? Does it imply we can no more have SCGI/WSGI configuration? It is now available... Sadd's website was offline then... - Any tutorial/guidance on how to configure SCGI/WSGI with Nginex would help. No tutorial that I know of, but between: http://www.cleverdevil.org/computing/34/deploying-turbogears-with-lighttpd-and-scgi http://docs.turbogears.org/1.0/LightTPD and http://trac.turbogears.org/turbogears/wiki/NginxIntegration you may find enuf info to get it to work. /venkat --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
On 8/26/06, Damjan [EMAIL PROTECTED] wrote: As I said, it's only serving static content, there is no application. And lighttpd only uses mod_alias, mod_access, mod_accesslog. My point was that the memory leak is not in the core of lighttpd but in some of it's modules. Since there is a choice which module to use to connect to WSGI applications, we should see what modules have leaks. mod_proxy definitely leaks. When I attempted to deploy lighttpd, that was the *only* module I was using (no static content even) and it leaked memory like a sieve. nginx does not leak at all (at least for proxy, fcgi, rewrite, static content). The config language is simpler, and it also uses less memory and CPU than anything else I've tried (and this is at a load of 500 req/sec at times). I don't think scgi is a choice with nginx at the moment, but fcgi and proxying definitely are. -bob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
On Thu, 2006-08-24 at 18:44 -0700, Cliff Wells wrote: On Thu, 2006-08-24 at 15:43 -0700, Bob Ippolito wrote: The only solution I know of that's extremely high performance that offers all of the features that you want is nginx [2], but its documentation is largely in Russian. I can't read Russian, but I was able to figure it out (the configuration language isn't Russian, neither is C source). I currently have nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a *single server*. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6). Here's a nice document for those interested. http://blog.kovyrin.net/2006/04/17/typical-nginx-configurations/ I've added a page to Trac showing a basic (well, a bit more than basic) setup for using Nginx with TurboGears for anyone interested. http://trac.turbogears.org/turbogears/wiki/NginxIntegration -- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
As I said, it's only serving static content, there is no application. And lighttpd only uses mod_alias, mod_access, mod_accesslog. My point was that the memory leak is not in the core of lighttpd but in some of it's modules. Since there is a choice which module to use to connect to WSGI applications, we should see what modules have leaks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
Interesting, I have an instance of lighttpd serving only static files for more than month and a half and it's total memory usage is less than 5MB. So it must be same of the modules you used? mod_scgi or mod_proxy? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
Hi Damjan, Could you pl. list the lighty modules you have running in your app. If possible, if you could also provide snippets of your lighttpd.conf file, it would be useful. Thanks, /venkat --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
On Thu, 2006-08-24 at 11:11 -0700, venkatbo wrote: Hi folks, Since flup is not *available* for some reason, Im looking to evaluating lighttpd as a simple proxy to TG, based on the steps in: http://www.turbogears.org/preview/docs/deployment/lighttpd.html#proxy Could anyone tell me what the limitations are to this approach as compared to the SCGI/WSGI method in: http://www.turbogears.org/preview/docs/deployment/lighttpd.html#proxy Also in the SCGI/WSGI method, there is a reference to the statement: This method will not work with the standard visit model... Could anyone explain what the standard visit model is... Googled around, but couldn't get any more info for this context. I'd recommend not using Lighty for this purpose. Take a look at Pound instead: http://www.apsis.ch/pound/ What I typically do is run both Lighty and TG behind the Pound proxy, with TG serving dynamic content and Lighty serving everything that's under /static. An example pound.conf might look like: ListenHTTP Addressyour.external.ip.address Port 80 Err500 /etc/pound/err500.html Err503 /etc/pound/err503.html HeadRemove (X-Forwarded-For|X-SSL-Connect) Service HeadRequire Host: .*\.yourdomain\.com.* Url /(static|images|css|javascript|flash)/.* BackEnd Address 127.0.0.1 Port8080 End End Service HeadRequire Host: .*\.?yourdomain\.com.* BackEnd Address 127.0.0.1 Port8081 End End End Then setup Lighty to run on port 8080 and your TurboGears app to run on 8081 and you're set. Regards, Cliff --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
Thanks Cliff. I just checked out Pound. May be I'm missing something, but other than Load Balancing Reverse Proxy, I don't see it offering anything more than what lighty alone can already do as per: http://www.turbogears.org/preview/docs/deployment/lighttpd.html#proxy I'll not be building a hi traffic site, so may not benefit from pound. Ligthy can already be compiled for SSL, pcre etc., and be setup to talk to TG (even over a UDS, if I'm right). What I need is all the extra http functionality lighty provides over and above what CherryPy (TG) can provide. Pl. let me know what I would gain by using pound in my scenario. Thanks, /venkat --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
On Thu, 2006-08-24 at 12:36 -0700, venkatbo wrote: Thanks Cliff. I just checked out Pound. May be I'm missing something, but other than Load Balancing Reverse Proxy, I don't see it offering anything more than what lighty alone can already do as per: http://www.turbogears.org/preview/docs/deployment/lighttpd.html#proxy I'll not be building a hi traffic site, so may not benefit from pound. Ligthy can already be compiled for SSL, pcre etc., and be setup to talk to TG (even over a UDS, if I'm right). What I need is all the extra http functionality lighty provides over and above what CherryPy (TG) can provide. Pl. let me know what I would gain by using pound in my scenario. The real question is: what would you gain by using Lighty? You are asking about a proxy solution and Pound is a proxy. Lighty can *act* as a proxy, but it's really a web server (in fact, the reverse proxy support is new to the latest point release). Since CherryPy is already a web server, what is the point in putting another web server in front of it? In short, Pound gains you simplicity and singleness of purpose. Also, FWIW, I switched to Lighty around a year ago and I've found it pretty disappointing. Jan Knetche appears to have more interest in catering to the Rails community and going to conferences than fixing outstanding bugs. Lighty's config syntax is dismal. At first it appears quite elegant, but soon you realize it's actually a gross hack and is broken in several key ways. There are obvious misfeatures in its syntax and some parts are outright broken and have remained so for quite some time. It's pretty sad when bugs stick around so long that they acquire names (The Nested Conditional bug being one that grates on me to no end). I still use Lighty for its original purpose, that is, serving static content, but find that maintaining complex configurations to be more of a pain than it's worth and so avoid it whenever possible. Frankly I'm hoping that one of the other lightweight web servers steps up to take Lighty's place soon. Regards, Cliff -- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
On Thu, 2006-08-24 at 12:36 -0700, venkatbo wrote: What I need is all the extra http functionality lighty provides over and above what CherryPy (TG) can provide. Sorry, somehow in my haste to reply I didn't see that last sentence: what functionality are you hoping for? Cliff -- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
Thanks Cliff, for your observations. Basically, I needed to provide support for: - HTTP/1.1 - SSL (openssl) - (Fast)CGI - chroot() - sessions - static content I thought lighty would be providing all except the sessions part, which I was hoping to make it disk/file-based. The # of simultaneous users will be in the 20's. Initially no db invloved (data coming from other backend apps), but may have a db later. /venkat --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
One problem with Lighty is that it leaks memory like a sieve [1]. I audited it for a little bit and I gave up, it's a mess. I'd steer clear of it, it will quickly ruin your day if you throw a lot of traffic at it. The only solution I know of that's extremely high performance that offers all of the features that you want is nginx [2], but its documentation is largely in Russian. I can't read Russian, but I was able to figure it out (the configuration language isn't Russian, neither is C source). I currently have nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a *single server*. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6). Under the same kind of load, apache falls over (after using 1000 or so processes and god knows how much RAM), pound falls over (too many threads, and using 400MB+ of RAM for all the thread stacks), and lighty *leaks* more than 20MB per hour (and uses more CPU, but not significantly more). [1] http://trac.lighttpd.net/trac/ticket/758 [2] http://sysoev.ru/en/ -bob On 8/24/06, venkatbo [EMAIL PROTECTED] wrote: Thanks Cliff, for your observations. Basically, I needed to provide support for: - HTTP/1.1 - SSL (openssl) - (Fast)CGI - chroot() - sessions - static content I thought lighty would be providing all except the sessions part, which I was hoping to make it disk/file-based. The # of simultaneous users will be in the 20's. Initially no db invloved (data coming from other backend apps), but may have a db later. /venkat --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
On Thu, 2006-08-24 at 15:43 -0700, Bob Ippolito wrote: The only solution I know of that's extremely high performance that offers all of the features that you want is nginx [2], but its documentation is largely in Russian. I can't read Russian, but I was able to figure it out (the configuration language isn't Russian, neither is C source). I currently have nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a *single server*. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6). I'm glad to hear of someone I trust trying this server. Someone was espousing it on the #cherokee channel, but I hadn't gotten around to trying it out. Guess I have to now. Thanks for the info. Cliff -- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
Bob, Thanks for that insight :-) I somehow got the impression lighty was the rage these days, second only to apache. With this kink of a leak issue, unclear how it attained that position ;!) Will give nginx a try. Thanks. /venkat --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
[TurboGears] Re: Lighttpd as a simple proxy to TG
On Thu, 2006-08-24 at 15:43 -0700, Bob Ippolito wrote: The only solution I know of that's extremely high performance that offers all of the features that you want is nginx [2], but its documentation is largely in Russian. I can't read Russian, but I was able to figure it out (the configuration language isn't Russian, neither is C source). I currently have nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a *single server*. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6). Here's a nice document for those interested. http://blog.kovyrin.net/2006/04/17/typical-nginx-configurations/ Cliff -- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups TurboGears group. To post to this group, send email to turbogears@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears -~--~~~~--~~--~--~---
